Google’s New Search Console URL Inspection API: What It Is & How to Use It

2 years ago 90
ARTICLE AD BOX

Diagnosing technical issues connected your website tin beryllium 1 of the astir time-consuming but important aspects of moving a website.

To marque things worse, Google lone allows you to inspect 1 URL astatine a clip to diagnose imaginable issues connected your website (this is done wrong Google Search Console).

Luckily, determination is present a faster mode to trial your website: participate the Google Search Console URL Inspection API…

What is the Google Search Console URL Inspection API?

The Google Search Console URL Inspection API is a mode to bulk-check the information that Google Search Console has connected URLs. Its intent is to assistance developers and SEOs much efficiently debug and optimize their pages utilizing Google’s own data.

Here’s an illustration of maine utilizing the API to cheque whether a fewer URLs are indexed and submitted successful my sitemap:

In array  form, API showing Chris' URLs are indexed but not submitted successful  sitemap

What benignant of information tin you get from the Google Search Console URL Inspection API?

The Google Search Console URL Inspection API allows you to propulsion a wide scope of data. Below is simply a database of immoderate of the astir discussed features:

lastCrawlTime

With this field, you tin recognize precisely erstwhile Googlebot past crawled your website. This is highly utile for SEOs and developers to measurement the frequence of Google’s crawling of their sites. Previously, you could lone get entree to this benignant of information done log record investigation oregon spot-checking idiosyncratic URLs with Google Search Console.

robotsTxtState

With this field, you tin recognize whether you person immoderate robots.txt rules that volition artifact Googlebot. This is thing you tin cheque manually, but being capable to trial it astatine standard with Google’s ain information is simply a fantastic measurement forward.

googleCanonical and userCanonical

In immoderate situations, Google has been known to prime a antithetic canonical from the 1 that has been specified successful the code. In this situation, having the quality to comparison some (side by broadside and astatine scale) utilizing the API is utile for enabling you to marque the due changes.

crawledAs

This tract allows you to recognize which idiosyncratic cause is utilized for a crawl of your site: Mobile/Desktop. The effect codes are beneath for reference:

  • DESKTOP – Desktop user agent
  • MOBILE – Mobile user agent

pageFetchState

Understanding the pageFetchState tin assistance you diagnose server errors, not recovered 4xxs, brushed 404s, redirection errors, pages blocked by robots.txt, and invalid URLs. A database of responses is beneath for reference.

FieldWhat it means
PAGE_FETCH_STATE_UNSPECIFIEDUnknown fetch state
SUCCESSFULSuccessful fetch
SOFT_404Soft 404
BLOCKED_ROBOTS_TXTBlocked by robots.txt
NOT_FOUNDNot recovered (404)
ACCESS_DENIEDBlocked owed to unauthorized petition (401)
SERVER_ERRORServer mistake (5xx)
REDIRECT_ERRORRedirection error
ACCESS_FORBIDDENBlocked owed to entree forbidden (403)
BLOCKED_4XXBlocked owed to different 4xx contented (not 403, 404)
INTERNAL_CRAWL_ERRORInternal error
INVALID_URLInvalid URL

indexingState

The indexing authorities tells you the existent presumption of indexation for the URLs. Apart from the much evident Pass and Fail responses, determination are different responses:

  • NEUTRAL is equivalent to the “Excluded” connection successful Search Console.
  • PARTIAL is equivalent to the “Valid with warnings” connection successful Search Console.
  • VERDICT_UNSPECIFIED means that Google is incapable to travel to a decision astir the URL(s) successful question.

coverageState 

This gives you item connected whether a URL has been submitted successful your sitemap and indexed.

referringUrls

This allows you to spot wherever each leafage is linked from, according to Google.

Sitemap 

This enables you to recognize which URLs are included successful the sitemap(s).

Other uses for the API

You tin besides usage the API to inspect your AMP site—if you have one.

How to usage the Google Search Console URL Inspection API measurement by step

Using the Google Search Console URL Inspection API involves making a petition to Google. The petition parameters you request to specify are the URL that you privation to inspect and besides the URL of the spot successful Google Search Console.

The petition assemblage contains information with the pursuing structure:

Structure of information

If you are funny to larn much astir however to usage the API, Google has extended documentation astir this.

Below is an illustration of the benignant of effect you tin get from the API:

Code effect   from API

If you’re not comfy with codification oregon conscionable privation to springiness it a spell consecutive away, you tin usage valentin.app’s escaped Google Bulk Inspect URLs tool. The instrumentality provides a speedy mode to query the API without immoderate coding skills!

Here’s however to usage it. You can:

  1. Go to https://valentin.app/inspect.html, authorize entree to your Google account, and prime the Search Console spot you privation to test. Then paste your URLs into the container below. (The information volition beryllium processed successful your browser and not uploaded to a server oregon shared with anyone.)
  2. Click the “Inspect URLs” button. The information volition commencement to propulsion from the API.
  3. Export the information arsenic a CSV oregon Excel record by clicking the button.
  4. Analyze the information and cheque for immoderate imaginable issues.

How tin you usage the Google Search Console URL Inspection API successful practice?

In theory, the Google Search Console URL Inspection API seems similar a large mode to recognize much astir your website. However, you tin propulsion truthful overmuch information that it’s hard to cognize wherever to start. So let’s look astatine a fewer examples of use cases.

1. Site migration – diagnosing immoderate method issues

Site migrations can origin each kinds of issues. For example, developers tin accidentally artifact Google from crawling your tract oregon definite pages via robots.txt.

Luckily, the Google Search Console URL Inspection API makes auditing for these issues a doddle.

For example, you tin cheque whether you’re blocking Googlebot from crawling URLs successful bulk by calling robotsTxtState.

Here is an illustration of maine utilizing the Google Search Console URL Inspection API (via valentin.app) to telephone robotsTxtState to spot the existent presumption of my URLs.

In array  form, API showing presumption    of Chris' URLs arsenic  "allowed"

As you tin see, these pages are not blocked by robots.txt, and determination are nary issues here.

TIP

 Site migrations tin sometimes pb to unforeseen method SEO issues. After the migration, we urge utilizing a tool, specified arsenic Ahrefs’ Site Audit, to cheque your website for implicit 100 pre-defined SEO issues.

2. Understand if Google has respected your declared canonicals

If you marque a alteration to the canonical tags across your site, you volition privation to cognize whether oregon not Google is respecting them.

You whitethorn beryllium wondering wherefore Google ignores the canonical that you declared. Google tin bash this for a fig of reasons, for example:

  • Your declared canonical is not https. (Google prefers https for canonicals.)
  • Google has chosen a leafage that it believes is simply a amended canonical leafage than your declared canonical.
  • Your declared canonical is simply a noindex page.

Below is an illustration of maine utilizing the Google Search Console URL Inspection API to spot whether Google has respected my declared canonicals:

In array  form, API showing Google respects Chris' canonicals

As we tin spot from the supra screenshot, determination are nary issues with these peculiar pages and Google is respecting the canonicals.

TIP

To rapidly spot if the googleCanonical matches the userCanonical, export the information from the Google Bulk Inspect URLs instrumentality to CSV and usage an IF look successful Excel. For example, assuming your googleCanonical information is successful Column A and your userCanonical is successful Column B, you tin usage the look =IF(A2=B2, “Self referencing”,”Non Self Referencing”) to cheque for non-matching canonicals.

3. Understand erstwhile Google recrawls aft you marque changes to your site

When you update galore pages connected your website, you volition privation to cognize the interaction of your efforts. This tin lone hap aft Google has recrawled your site.

With the Google Search Console URL Inspection API, you tin spot the precise clip Google crawled your pages by utilizing lastCrawlTime.

If you can’t get entree to the log files for your website, past this is simply a large alternate to recognize however Google crawls your site.

Here’s an illustration of maine checking this:

In array  form, API showing "last crawl" day  and clip  for each   URL

As you tin spot successful the screenshot above, lastCrawlTime shows the day and clip my website was crawled. In this example, the astir caller crawl by Google is the homepage.

Understanding erstwhile Google recrawls your website pursuing immoderate changes volition let you to nexus whether oregon not the changes you made person immoderate affirmative oregon antagonistic interaction pursuing Google’s crawl.

FAQs

How to get astir the Google Search Console URL Inspection API limits?

Although the Google Search Console URL Inspection API is constricted to 2,000 queries per day, this query bounds is determined by Google Property.

This means you tin person aggregate properties wrong 1 website if they are verified separately successful Google Search Console, efficaciously allowing you to bypass the bounds of 2,000 queries per day.

Google Search Console allows you to person 1,000 properties successful your Google Search Console account, truthful this should beryllium much than capable for most users.

Can I usage the Google Search Console URL Inspection API on immoderate website?

Another imaginable limiting origin is you tin lone tally the Google Search Console URL Inspection API on a spot that you ain successful Google Search Console. If you don’t person entree to the property, past you cannot audit it utilizing the Google Search Console URL Inspection API.

So this means auditing a tract that you don’t person entree to tin beryllium problematic.

How close is the data?

Accuracy of the information itself has been an contented for Google implicit the past fewer years. This API gives you entree to that data. So arguably, the Google Search Console URL Inspection API is lone arsenic bully arsenic the information wrong it.

Tim Guillot shares tweet to John Mu astir  information  issues; Tim adds the issues are "not fixed"

As we person antecedently shown successful our study of Google Keyword Planner’s accuracy, information from Google is often not arsenic close arsenic radical presume it to be.

Final thoughts

The Google Search Console URL Inspection API is simply a large mode for tract owners to get bulk information straight from Google connected a larger standard than what was antecedently imaginable from Google Search Console.

Daniel Waisberg and the squad down the Google Search Console URL Inspection API person decidedly done a large occupation of getting this released into the wild.

But 1 of the criticisms of the Google Search Console URL Inspection API from the SEO assemblage is that the query complaint bounds is excessively debased for larger sites. (It is capped astatine 2,000 queries per day, per property.)

For larger sites, this is not enough. Also, contempt the imaginable workarounds, this fig inactive seems to beryllium connected the low side.

What’s your acquisition of utilizing the Google Search Console URL Inspection API? Got much questions? Ping maine on Twitter. 🙂