ARTICLE AD BOX
Advertisement
Google's Gary Illyes reveals Googlebot collects links alternatively of "following" them, contradicting authoritative documentation.
- Googlebot collects links alternatively than pursuing them successful real-time.
- Gary Illyes shared this accusation successful Google's 'Search Off The Record' podcast.
- Google's documentation contradicts the existent crawler behavior.
In a caller occurrence of Google’s Search Off The Record podcast, Analyst Gary Illyes clarified however Googlebot interacts with links during the crawling process.
His penetration contradicts the wide held content that Googlebot navigates websites by pursuing links successful real-time.
According to Illyes, the Googlebot gathers links for aboriginal processing alternatively than pursuing them successful a linear fashion.
This misunderstanding stems from Google’s ain documentation.
Contradictory Information
“It’s my favored peeve,” Illyes stated during the podcast, referring to Google’s enactment pages.
He continues:
“On our site, we support saying Googlebot is pursuing links, but no, it’s not pursuing links. It’s collecting links, and past it goes backmost to those links.”
What The Documents Say
Google’s authoritative documentation connected crawlers states (emphasis mine):
“Crawler (sometimes besides called a ‘robot’ oregon ‘spider’) is simply a generic word for immoderate programme that is utilized to automatically observe and scan websites by following links from 1 web leafage to another.”
The papers implies that Googlebot navigates the web by actively pursuing links successful real-time.
This highlights a discrepancy betwixt Google’s nationalist messaging and the existent functionality of their crawler, raising questions astir different imaginable misunderstandings successful the SEO community.
Implications For SEO
This revelation has respective imaginable implications for however we recognize Google’s crawling process:
- Crawl Budget: If Googlebot collects links archetypal and past revisits them later, it mightiness interaction however we deliberation astir crawl budgets. It’s imaginable that the archetypal “collection” signifier mightiness beryllium little resource-intensive than antecedently thought.
- Site Architecture: While a logical tract operation is important, the thought that Googlebot needs to “find” heavy pages done a bid of clicks whitethorn beryllium outdated. This could power however we attack interior linking and tract depth.
- Crawl Frequency: This penetration mightiness explicate wherefore immoderate pages are crawled much often than others, careless of their presumption successful the tract hierarchy.
Looking Ahead
Many SEO strategies are built connected the presumption that Googlebot traverses websites by pursuing interior links similar a visitant connected a page.
If Illyes’ statement is accurate, it suggests that Googlebot’s behaviour is much analyzable than antecedently understood.
While this revelation doesn’t invalidate existent SEO champion practices, it does item the request for SEO professionals to enactment informed astir the nuances of however Google operates.
Hear Illyes’ connection successful the podcast below:
Featured Image: Ribkhan/Shutterstock
SEJ STAFF Matt G. Southern Senior News Writer astatine Search Engine Journal
Matt G. Southern, Senior News Writer, has been with Search Engine Journal since 2013. With a bachelor’s grade successful communications, ...
Subscribe To Our Newsletter.
Conquer your time with regular hunt selling news.
English (US)