Back to the glossary
Crawl budget
The number of pages a search engine is willing to fetch from a site in a given period.
It is a real constraint only on large sites, and it is invoked far more often than it applies. A site of a few hundred pages will not run out.
Where it bites is on sites that generate URLs: faceted navigation, search result pages, session parameters, endless pagination. A crawler spends its allowance on those and never reaches the pages that matter.
It is one of the arguments for keeping a generated URL set small and deliberate rather than publishing every combination a template can produce.
We do this
SEO 360°
Strengthen your organic visibility with an end-to-end SEO strategy across technical, on-page, and content pillars
See the service