Google is embarking on a mission to reduce its crawling activity, emphasizing quality over quantity. Google analyst Gary Illyes reveals his goal of simplifying the crawling process to focus on the most relevant URLs. The search engine thus wishes to improve the efficiency of its exploration while reducing its data consumption.
The quality of the content becomes a determining criterion for the crawl, thus disproving the myth of a fixed “crawl budget”. According to Gary Illyes, search demand influences crawl limits, meaning that producing quality content is essential to being better crawled and indexed.
This dynamic approach allows Google to adapt to search trends and ensure more efficient indexing of relevant content. The objective of reducing crawling and data consumption is part of a vision of sustainability and efficiency of the web.
At the same time, discussions around the IndexNow protocol were launched, with Bing and Yandex in the lead. Google announced in November 2021 its intention to test this protocol, but it remains to be seen whether they will make this initiative a reality. In any case, the search engine industry seems to be moving towards a more sustainable and efficient web, where quality takes precedence over quantity.
