If you’re an SEO nerd, like me, you’ve probably heard of Google’s Quality Rater Guidelines.
The Quality Rater Guidelines (or QRG) is the document Google give their human evaluators when they’re tasked with testing the latest Google Search algorithm updates.
There are still lots of people out there that don’t even know this documentation exists. To be fair, Google used to keep them secret.
But now they publish them openly and they’re frequently updated too.
It’s a weighty read though. The latest version comes in at 170 pages and is just shy of 60,000 words. Phew. If you’re so inclined you can download the Quality Rater Guidelines. It makes for a punchy bedtime read!
So I decided to get Google’s Notebook LM to summarise it into a podcast for me. And it did a pretty decent job — wrapping up in under 15 minutes. Give it a listen:
Next time you need to get your content team, copywriters (or your boss?) on board with some of the important tenets of the QRG, this post might come in handy!
Making the QRG more digestible
If you’re looking for ways to make this more digestible lots of smart folks write about them. Lily Ray and Marie Haynes have written about them extensively. Their work is always worth a read.
When I gave a talk at the #DigitalHustle event in Dublin one of the other speakers was AI Consultant Karen Howley who spoke about some of her processes around creating content and how she uses AI during the various processes.
We talked about AI, LLMs and all the various tools out there — and it occurred to me that, I don’t think anyone has used any of these tools to summarise the QRGs (if they have, I’ve missed it).
So I put all 170 pages of the most recent version of Google’s Quality Rater Guidelines (March 2024) into Google’s own Notebook LM model.
The Egon Spengler in me is slightly worried about the “Don’t cross the streams…” here of getting Google tools to summarise Google information — but I think we’re safe from the Stay Puft Marshmallow Man… for now at least.
You may have heard of this tool before. It had a ‘viral’ moment when the tool launched and anyone could get the two American-sounding hosts to ‘chat’ about… well, anything: a blog post, a video, uploaded documents and more. Including (in)famously when someone uploaded a 1,000 page document which repeated just the words “poop” and “fart” with the ‘podcast hosts’ getting into a discussion around the perception of art and what is or isn’t ‘profound’.
Notebook LM’s attempt at summarising the QRG is actually not bad. It certainly gives a good gist of the Guidelines, some of the factors it considers when evaluating websites and online content and why those things are important too.
OK, it’s not a replacement for reading and understanding the entire document (two very different things and chunky tasks in and of themselves) but I thought it was interesting. If you’re looking for a gentle introduction to these sorts of ideas (perhaps for your colleagues… or your boss?) it might be useful.
AI and LLMs still have their limitations (I asked Bing’s Copilot to generate the image of the two hosts that I’ve used for this!) again, not exactly flawless but… not bad!
And yes, I had to prompt it not to make both the hosts Caucasian (because all podcast hosts are white, right?) and Copilot’s spelling leaves a lot to be desired too! But it’s not bad.
Key takeaway
Notebook LM’s ‘podcast hosts’ came up with this as the “Key takeaway our listeners should ponder…”
Now that you know how Google thinks about high-stakes topics, what are some ‘Your Money or Your Life’ searches you make regularly?
How can you apply these quality standards to evaluate the information you find? Especially when it comes to those crucial areas of your life?
Not a bad summary!
My summary is that tools like these are — like any tool — only as good as the people wielding them. They can be great time-savers, used in the right way. Using them to summarise extensive documents or complex topics can make things more digestible.
But it’s like when a friend tells you about their favourite film. You may get a good idea about it but you still won’t have seen the film. Or if you watch a YouTube video on how to run like Usain Bolt. You may learn some good tips, you may even be better at sprinting than you were before you watched the video… but you won’t be endangering any world records any time soon!
That said, I hope you found this article and the audio useful or interesting (or both!). Feel free to let me know below if you did!
If you’re looking for some help with your SEO (maybe some training for your content team, where we introduce them to some of the ideas in the QRGs?) I’d love to hear from you. Do get in touch.
Qualitative insights wrapped up on Quality Raters Guidelines. Good to consume content this way, though I am a reader. I just tried, and its overall fine. I am amazed with tone clarity of thoughts shared and some perspective at the raters end is something every publisher needs to look into on improving content.