Page Level Query Analysis at Scale with Google Colab, Python, & the GSC API [Video Instructions Included]

[Guest Jason Melman]

The YouTube playlist referenced throughout this blog can be found here:

 

Anyone who does SEO as part of their job knows that there’s a lot of value in analyzing which queries are and are not sending traffic to specific pages on a site.

 

The most common uses for these datasets are to align on-page optimizations with existing rankings and traffic, and to identify gaps in ranking keywords.

 

However, working with this data is extremely tedious because it’s only available in the Google Search Console interface, and you have to look at only one page at a time.

 

On top of that, to get information on the text included in the ranking page, you either need to manually review it or extract it with a tool like Screaming Frog.

 

You need this kind of view:

 

 

…but even the above view would only be viable one page at a time, and as mentioned, the actual text extraction would have had to be separate as well.

 

Given these apparent issues with the readily available data at the SEO community’s disposal, the data engineering team at Inseev Interactive has been spending a lot of time thinking about how we can improve these processes at scale.

 

One specific example that we’ll be reviewing in this post is a simple script that allows you to get the above data in a flexible format for many great analytical views.

 

Better yet, this will all be available with only a few single input variables.

 

A quick rundown of tool functionality

 

 

The tool automatically compares the text on-page to the Google Search Console top queries at the page-level to let you know which queries are on-page as well as how many times they appear on the page. An optional XPath variable also allows you to specify the part of the page you want to analyze text on.

 

This means you’ll know exactly what queries are driving clicks/impressions that are not in your