One of the reasons (many) I started the Optimisey SEO events was to share knowledge. The SEO community is good like that.
Another reason was to share the great knowledge I’ve picked up from others sharing theirs — to pass it on as it were.
A while back I was working on a site and really toiling with the on-site search. The site was huge, the on-site search was unwieldy and I didn’t know where to start. I’d interacted with JP Sherman on Twitter and when I asked if I could pick his brain, maybe send him an email he said: ‘Why not a call? Let’s talk.’
Long story short JP gave me over an hour of his time, talking through his experiences, helping out with my queries and just being really bloody lovely. When I finished the call I had this: ‘Wow… what a nice guy.’ my next thought was: ‘I wish I’d recorded that…’
Fast forward a little bit and I managed to persuade JP to add Cambridge to his itinerary when he was coming to the UK (he’s based in North Carolina, in the US) and get him on the Optimisey stage.
JP’s almost painfully smart but extremely generous with his knowledge and the chance to get him to share some of his insights into on site search (an all too often overlooked opportunity) was too good to miss.
Good news if you missed it – the video, transcript and JP’s slides are all here.
Get the good stuff first
Want to be among the first to get this good stuff? Here are your options:
- Attend the Optimisey events
- Sign-up to the Optimisey newsletter
- Subscribe to the Optimisey YouTube channel
Now over to JP (usual cop-out here: JP’s words have been mangled by both YouTube and me (in that order) so it if’s nonsensical that’d be me, not JP. Best way to avoid this? Attend the events yourself and hear it first hand!).
JP Sherman on Site Search
Download JP’s slides (4MB PPTX file)
My name is JP Sherman and I work at Red Hat. My job is basically this: I connect people to the information that they’re looking for in a completely agnostic way, whether it’s SEO, voice chat or any other method that people are looking for information.
So I don’t just do SEO. I do findability.
So I’m gonna be a little existential here: Who am I? I started my career in search while working for a US Army PsyOp. It’s not that scary.
A lot of what I would what I did was humanitarian. Things like malaria prevention in Southeast Asia – so disseminating information.
I studied biological anthropology with a focus in behavioural science. I’ve worked for large agencies and small agencies. I’ve been in-house.
I started getting into on-site search when I was recruited build a search engine for video games. It failed.
Now I run search and findability for Red Hat. So we’re gonna start with the fact that:
What is essentially the behaviour behind search? In general it’s a journey – and the idea is that a person is looking for something that they don’t know or something that they need and that they want.
So it ultimately starts with desire. So I’m gonna reach out and say: How many of you right now use data to measure, analyse your SEO or your PPC?
Should be everybody. All right how many of you actually look at your on-site search data on a fairly regular basis?
That’s great but it’s still not enough.
I kind of see on-site search as the little sibling of SEO because it’s useful but not a lot of people are talking about it or even using the information that it can provide to make better decisions.
All right – throughout this I’m going to do some shameless ‘Twitter plug’ moments.
The reason that on-site search is so important is that if a user is using your site search they’re engaged. They think you have something they want and they are expressing that intention by actually using your on-site search. They’re demonstrated a basic level of trust in the sense of they’re not ditching they are continuing their journey and they’re in an ‘actively looking’ phase.
When it comes to behaviour post-search behaviour of users they’re:
- more likely to return
- they’re more likely to add to the cart
- they’re more likely to consume your content and
- they’re more likely to convert
Users who don’t find what they’re looking for they know that Google is sometimes less than a click away and in a lot of things they remember the miss more than they remember the hit – and they’re less likely to browse your site.
So I’m using a little bit of fear tactics on you guys right now. So across the industry about approximately 30% of visitors use on-site search and this is a really, really fantastic scenario because this means that people actually want to, when they enter your site, they may have come in through a link from social, from a social network, a friend, even Google and if they’re continuing – if they want to continue their engagement with your site they can either use the navigation, they can use internal links or they can search.
And this is the big number research shows – and I do need to be clear that this route there’s not there has not been a lot of research on this so this data is about two years old it may be better, may be worse but honestly because nobody’s doing a lot of research on this it might be out of date but still it’s fairly significant:
We do know that people who search the site these are people who are 5 to 6 times more likely to convert.
Only about 15% of companies have resources dedicated and this is this is the really kind of foundational aspects of search behaviour.
The two different types of search: Active & Passive
My background is in behavioural science and biology so I look at the active, the passive and the quantitative and the qualitative behaviours that people do, that we can measure in on-site search.
So there are two basic types of search: There’s active search and there’s passive search.
Passive search is very similar to if you’re driving, you’re on the road you are lightly scanning for any threats any cars moving in either erratic ways. When it comes to Google or your own on-site search people switch from a passive mode to an active mode this means that they’re actively looking for something.
They may not even know what they’re looking for but they’ll know it when they see it.
So this starts with desire. They want something.
This desire expresses itself as a query and the tricky part is that that query puts inside the user’s head acceptance and rejection criteria.
So if you’re considering about the ten blue links on Google – I am processing acceptance and rejection criteria for every result that I see, that I’m scanning in less than a second to say: ‘Yes this could potentially be useful. Yes this is exactly what I’m looking for’ or ‘No this doesn’t look right’ and this is why it’s so critical because people look on the search engine results snippets for less than a second on average.
The imperative part for SEOs and people who are working on on-site search is to create an immediate perception of value.
This is done through title tags and metadata but I’m getting ahead of myself – so I got way ahead of myself I, apparently I’m very excited about this…
So we’re talking about the immediate perception of value in your content. This can be expressed through title tags, meta tags, even micro-data structured markup – so if it’s an image search there are ways to convey to the user what this is about – and the perception of value.
So now we’re gonna talk a little bit about measuring.
Now we have the foundation we know why on site search is important and we know the the kind of background behaviours that humans use when they’re searching.
How do we measure this?
So search behavior is the context for measurements and what do I mean by measurements? It basically falls into three categories:
- what are the words people use are using
- what are the behaviors they elicit or exhibit when they accept a query or a result and
- what do they do when they reject it?
So the critical thing is identify the things that trigger the acceptance.
This could be well-crafted titles…
Quick aside: one of the experiments that I ran when I was working at a large international online bike retailer was my competition was Target, Walmart in the US, REI so really big names, even Amazon and every single one of them had the title tag ‘”Road bikes | [brand]’
Our site had “Road bikes | [brands]”.
I thought ‘This is crap’ so I went in, without permission, and changed it to “Lightweight and fast bikes for the open road”.
We went from position seven to position one in three weeks
Because, when we’re talking about the perception of value, I tried to optimise for emotion and optimised for the click-through as opposed to optimizing for that one perfect word that everyone else is using.
Identify the conditions that result in a failure state. What’s the failure state? Not a click, even a search with no results.
A long time ago the this company called Think Geek and for their April Fool’s joke they mocked up a tauntaun sleeping bag.
My friend who worked there said that they went from zero instances of “Taunton sleeping bag” in there on site search to thousands.
And every single time it was there was nothing comes up with this information. So thousands of people were searching that site for something that they thought was really cool and could potentially exist and getting no feedback.
Well we all know what nerds do when they get mad.
So at this point the searcher accepts/rejects the criteria this behavior is based on the internal criteria influenced by the results of that search and this is great until you start actually collecting the data.
And this is me on a regular basis: I have more failures than successes when it comes to this. When we talk about data we talk about the cast, the characters. These are clicks, these are impressions, these are instances, these are searches with null with no results.
How we turn that data into something, a little into a story is ultimately more useful when it comes to measuring.
So let’s start with the keywords and again – I kind of expressed this earlier – but like:
Don’t think of keywords as just a bag of words. Consider keywords as the vehicles through which intention is expressed.
When you’re looking at on-site search: what keywords drive the most engagement?
What are your KPIs? Is it a conversion? An engagement? Is it shares? Is it comments? Is it some sort of user-generated content?
Red Hat – we don’t actually sell anything – like I said we sell free software so all of our content, none of it has a conversion attached to it.
So we look at consumption: are people actually using that in a predictable way? What’s your top converting keywords and content?
When you start to understand, when you start looking at ‘This piece of content drives a high conversion rate through on-site search’ you can improve that content, you can make it more prominent, you can display that. In fact you could use that information to create features and promotions and things like that because, you know, now that this content converts well for users.
We have another shameless Twitter moment: this is a fun thing that we did when I was in e-commerce. We took the keyword that the user searched for in on-site search, dropped a cookie and used that information for re-targeting.
That way, as they search the web, if they were looking for ‘tauntaun sleeping bags’ they encounter our ads as they continued on through their internet journey.
What are the Different types of keywords?
So let’s talk about different types of keywords.
The high frequency low intent, low frequency high intent, there are long tails and then there are unicorns.
Unicorns are basically defined as a keyword that shows up once in a given period of time.
These can be – I’d say about 80% of them – typos but at the same time when you could start looking through these unicorns you can find some really good stuff.
So when it relates to these, when we were looking at quantitative metrics associated with these things, we want to look at: How many clicks? What was the click-through rate? How many conversions? What was the conversion rate? What was the consumption rate?
And the negative CTR – obviously – is one minus the click-through rate.
But I like to use the negative CTR as a way to look at my search failures it’s a way to kind of re-contextualize the information that I’m sharing.
Say we had a 45% click-through rate – is that good? Well we had a 55% negative click-through rate so we still have lots of room to improve.
So when we’re talking about high frequency and low intent keywords these are things like: bikes, software, college, Finland.
This is the top of the funnel. This is people searching for information. These are people rarely looking to actually purchase anything.
They’re looking for information or knowledge.
When we’re looking at the middle of the funnel: download latest iTunes, stream The Clash, what does coulrophobia mean – when it comes to this type of thing the user wants to take an action. They want to understand a specific thing.
This is the difference between high intent low frequency and low intent high frequency.
So whilst click-through rates are really good to understand kind of the overall health of the click-through rate it’s very much like taking the temperature. It’s not going to tell you anything specific.
It’s gonna tell you ‘Is it doing okay? Yeah or no’.
Just a fun fact:
When people right click on a search result and open a new tab sometimes you’ll get weird things like a greater than 100% click-through rate. Google Analytics hasn’t quite figured this out perfectly yet.
So that doesn’t mean you’re amazing it just means that this is a behavior that people are doing.
So when you start seeing patterns of people, patterns of keywords getting a higher percent click-through rate you could actually identify a very specific behavior that a user is doing. And if you can actually use that behavior to increase your your engagement that’s fantastic.
What is a good click through rate on search?
All right, people asked me ‘What is a good click-through rate for on-site search?’
I’ve done some research in knowledge base, I’ve done some research in e-commerce and generally what I find is that people have a high expectation of search. It is just supposed to work.
Thanks Google they don’t tolerate bad results.
And if your search results are bad then you are bad and you should feel bad.
And so what I found is on average if you have less than a 75% click-through rate people notice.
They don’t like it. They think you suck and they know Google is click away.
But if you’re above 75% you are a magical wizard. They don’t understand what you do.
There’s very little gray room in this. This is one of those rare things where it is almost binary.
So when we talk about conversions it is quantifiable. It is an action taken: it is a click, it is a purchase.
When I talk about consumptions this is something that gets a little trickier.
And people are still trying to kind of figure out in a consistent and predictable way because not all content is consistent across the web. There’s news, videos, articles documentation, technical documentation, troubleshooting documents you really can’t say that there is one type of consumption that’s going to fit them all.
But generally it [content consumption] boils down to ‘Scroll plus time’. Did a user scroll? How much time do they spend on the page based upon an average read speed of the content? That’s on us, on the page.
When you squish those two things together you get a good idea of how your content is being consumed.
I generally avoid looking at data that’s about two standard deviations from the mean. Mainly because these are your edge cases.
And what you want to do with the consumption data is understand how people are using your content not how many people are using it, or is consuming your content – and that’s really kind of the key differentiator.
When we talk about consumption it is 40% likely that this person read this article not this person read 40 percent of the article – so we’re talking about a little bit of nuance, a way of describing what consumption is and what how its measured.
So again I always go back to failure because it’s something that I’m really good at.
Understand that SEOs really tend to look for the top position and almost habitually we forget that there are nine other pieces of information on that page, on Google, on your search, most likely on your results. Because ten seems like a magical number that people really like.
So we talked about how the hell things fail. There are several different ways to actually provide assistance to the user when performing an on-site search. It can be as simple as ‘Did you mean?’ it can be an auto suggest such as, when you’re typing, a drop-down comes to either specific pieces of content in your library or similar keywords that people have searched for.
And what happens is that when you give a user a way to actually help them search along they do better.
And this is simple. Amazon do it: “Did you find what you’re looking for?”
The trick with that is that you get generally get such low user engagement that it’s not always statistically significant.
The last piece of research that I did that in was 15,000 content consumptions. The search feedback on our page was filled out four times.
The other funny thing is that if my page is about apple pies and I get a I get a person coming in from looking for ‘apple pies’: ‘Yes this was helpful’.
If I get a user coming to that same page from ‘apple pie recipes’ no, no. ‘Apple pie gallery’ nope, not helpful.
You can have the same piece of content with a ‘This was helpful/this was not helpful’ so understand that when we’re talking about ‘helpful’ search it’s really only valuable in the context of the keyword given.
It’s a really long-winded way of saying ‘Eh… it’s kind of useful’.
How to improve your on-site search
So these are some ideas: Auto-suggest – this is very common this drives engagement, this drives a better search experience for the user. If they don’t know exactly what they’re looking for they can find it quickly and easily they don’t have the type as much.
And one of the things that we look at is the time it takes for a user to from query to conversion.
If you can make their experience faster to get them to the point where they interact with your site they are happier.
Some sort of language recognition: When I search for ‘telescope fishing’ Amazon knows I’m looking probably for a telescopic fishing rod.
So having some language logic in your search can actually assist users in this.
Having some sort of auto-suggest can increase your click-through rate and conversion. So when we have no kind of assistance we get about a 42% click-through rate and a 3.9% conversion rate.
With it you’re looking at about a good 10 to 20% increase in those core KPIs.
For queries that have zero or anemic results – this is one of the really cool things that you can do if it’s valuable to you. Send that to your content team to create a content pipeline.
When people ask ‘What should I write about? I don’t have anything to write about?’ look through your search queries.
And understand: ‘We get this query ten times but nobody clicks’ or ‘We get this query 20 times and we don’t get any results’ this is perfect information to send your content team.
Context is king – not content
Now contextualization and disambiguation.
Every single one of these images is associated with the word ‘halo’ so it can get a little tricky.
Fortunately most the time if you’re in a niche and somebody is – if I’m on a video game site and I type in ‘halo’ probably mean that.
But if I’m on Amazon which sells music, games, toys well not that… but it’s understand it’s critical to understand what are the keywords in your universe that need to be contextualized.
One of the things that we’ve done at Red Hat is we’ve created a knowledge graph. So one of our keywords is one of our products is called JBoss. It’s a middleware application.
And what we’ve done is added this knowledge graph, that is at this point manually curated by me, to show users more information about what they’re looking for.
And again ‘JBoss’ I don’t know, like I can’t tell if this person is looking to download something. I can’t tell if they’re looking to fix a problem. I can’t see if they’re looking for just installation or registration issues.
So based on my keyword data I can see that when, it comes to JBoss, downloads are important information or important documentation is important – so I base this knowledge graph on the top searches surroundingthe low frequency high intent keywords, from that low intensity high volume keyword.
And again 10 blue links may not even work for your particular business.
And this is the thing: don’t be Google please don’t be Google. You can actually be better than Google because you know your product you know your audience you know a lot more about the nuances of what your people do and what your products are.
The other great thing is that you’re also working with a smaller set of information so you could be much more specific. If images are important you could organize your search results like this. Blended search results are really fantastic specifically in the area of high feature products and these there’s just some other examples of how search engines create a user interface that supports their users’ needs.
And I want to repeat that UI/UX is absolutely critical in this stage. You want to be able to test this. You want to be able to change your layouts and be able to measure how these things have, how what the changes you made have any effect on your core KPI’s.
So does the experiments group have a higher click-through rate? Are there more purchases or conversions? Does more contact and views after a search is performed?
These are all things that you can start looking at to figure out what’s important.
To me, and how can I design an experiment, with a control group in an A/B setting to test exactly for what you want what is the answer that you want from a specific question that you have?
Oh yeah mobile’s super important too. And that’s just the thing. I’ll tell you a quick story.
We made some changes on our search page and we found that on desktop we got a increase of 4% click-through rate. Yay!
We went to the mobile section and we had a 7% drop in click-through rate. And part of it was because of the way that the fonts rendered.
So we started creating more ‘if an’ statements about specifically your fonts, about snippets separators little things like that.
We were able to get the the mobile click through back up but as you do these tests always remember that people are searching on desktop and people are searching on mobile and the chance that people are going to be searching on mobile is going to increase over time to the point that the mobile experience is going to be a more important experience experience for the user.
Even if they’re the same result. Even if you get the exact same results the mobile UI and UX is going to be absolutely critical if it isn’t already.
So when you start – and this is a fun thing that I like to do – measure the queries that get no clicks.
I take those queries and I search Google. Do we show up in Google? Because if we don’t we just lost twice.
I take a look at the competition: and see why are they ranking for these particular terms? What content do they have that is more relevant in the user’s eyes?
8 bit, 16 bit and 32 bit – making internal site search better
So I’d like to use the metaphor of resolution and this is the really important thing.
Start at 8 bit resolution. Start with Pong. Understand that you just need to start looking at the data.
You need to start understanding the click-through rate; understand that like the click-through rate is not the best indication of how well your search is doing.
Hyper localize your internal search – and I’ll use a really quick example for that:
When I worked at the bike retailer, for the same keyword ‘bike tires’ we would get a horrible click-through rate, because it was just the results were bad.
So I did some testing and I found that, in Los Angeles for people looking for ‘bike tires’ they were most likely looking for road bike tires. In Colorado Springs, where the Rocky Mountains are, people looking for ‘bike tires’ were likely looking for mountain bike tires.
So one of the things that we did was we started biasing search results by geo. So when a person in Colorado Springs looks for ‘bike tires’ mountain bike tires results were biased.
We started doing more personalisation. We found that the types of people who are looking for specific types of jerseys fit into specific groups and we were able to use that kind of information to bias search results.
You don’t have to be pure as Google. You can use this information. Use this kind of personalisation to make your search results even better.
Now 16-bit: measure the time from query to conversion.
Create conversion events for content tests. And this is when you start testing the UI changes.
What do I mean by create conversion events in the content? When a user comes to a page there can be a non-transactional conversion event, kind of like a micro-conversion. Look for ways to send signals to your analytics that a user performs a beneficial action.
Now we’re getting into 64-bit:
Search oriented micro-data and content using structured markup and schema.
Start using that. Use that as a relevancy tuner for your search engine personalisation of other data and integrate search data in CMS so that your writers, when they look at a piece of content, they know that these are the ten key words that drive content to this page.
Here is the user behavior from this content; this is the level of conversions that you get from this content. Integrate the data that you get from search into the writer’s experience and the SEOs experience so they can clearly see the value of what they’re doing.
All right. The barriers really are minimal. Start with the data that you already collect. Refine that data with greater accuracy contextualize it with the KPIs that are important to you and then turn that data into intelligence that can be connected, integrated and shared.
How to get started with onsite search
Now there was a couple of different ways that you can do this.
Generally most ecommerce come with a search app installed. There are other options I use: Apache Solr, there’s Swift Type there’s Amazon Cloudsearch, Lucid Works just came out with a really good plug-and-play cloud search where you basically paste the code into a page and run search from a cloud.
So there are a lot of ways that you can actually get a better search than just the basics because most people when they do build their site they turn on search and then it’s plug-and-play. And then they rarely look at it again.
Thank you so much. I really appreciate, I’m humbled and honored to be able to speak in front of you.
