NETTING IT OUT

Is your site search good enough, or do you need to invest in making it better? There are plenty of opinions in any organization about the quality of the search experience offered by its Web site or intranet, ranging from “it stinks” to “it’s great.” We’ve prepared a self-assessment guide to help you move past opinion and into the realm of actual data.

Our guide, with questions, explanations and charts, leads you through four key tasks:

• Identify site audiences and scenarios
• Review current search experience data
• Assess performance against audience expectations for search
• Gather key stakeholders’ expectations and aspirations

Using our guide, you could spend mere days getting a back-of-the-envelope analysis, or spend months exhaustively collecting an irrefutable assessment of capabilities. It’s up to you how much resource you spend on this process: pick the easy steps in the guide, go with partial data, or assemble a team and kick off a research project. Either way, you’ll gain significant insight into your current search experience.

POTENTIAL FOR IMPROVING SITE SEARCH

We talk to many people who are uncertain whether their Web site search is “good enough” or needs improvement. They aren’t sure whether search experience is a drag on customer experience, and therefore depressing revenues and customer loyalty. They also aren’t sure where the gaps are in the search experience, what the potential value of improving the search experience might be, and what should be done to improve site search.

These questions can provoke lots of opinion, but you don’t have to settle for opinion and anecdotes. You can gather some data and perform some analysis. There are three key questions to be answered, and if you don’t have the answers to any of them, address them in this order:

• Is our site search experience good enough?
• If not, what actions would improve site search?
• What resources does my organization have to improve site search?

In this report, we provide a self-assessment guide for the first step, assessing the quality of your search experience. If you haven’t already heard from your customers, via Web logs and Web Feedback rants, that your search is terrible, or you have but are still in denial, we’ll guide you to an irrefutable answer. You can spend a few days on this self-assessment, picking the low-hanging fruit, and discover enough to determine where you stand. If you need the irrefutable proofs to convince others in your company, you could spend a few weeks or a few months on this self-assessment, gathering every last morsel of information.

IS OUR SITE SEARCH GOOD ENOUGH?

There are four key areas to investigate to perform your assessment:

• Identify site audiences and scenarios
• Review current search experience data
• Assess performance against audience expectations for search
• Gather key stakeholders’ expectations and aspirations

The assessment draws on information from a number of sources, including Web search statistics; interviews with key stakeholders; and customer feedback, including surveys and emails.

IDENTIFY SITE AUDIENCES AND SCENARIOS

Our first question, “Is search good enough?” needs some upfront analysis, or you won’t get anywhere. Good enough for whom to do what?

You have more than one audience for your site, and more than one outcome audiences are chasing. At a minimum, you should list your key audiences and their top three to five outcomes. Use anecdotes, interviews, or Web logs to identify actual scenarios. For example, a scenario might be, “A new employee at a top account, unfamiliar with our equipment, wants to learn how to set up a new application.” Or, “a high net worth client wants to adjust his portfolio in response to decline in price of oil.” See Table A for more examples of audiences and scenarios.

Key Audiences and Scenarios
(Please download the PDF for the formatted guide and self-assessment tables.)
Table A. This table presents examples of key scenarios by audience. Your scenarios and audiences may differ.

REVIEW CURRENT SEARCH EXPERIENCE DATA

Most likely, your company already collects data that will be useful to this analysis.

Customers and other searchers measure search success in terms of time to answer: how long did it take to connect to the information they need? Unfortunately, in many cases it is not possible to actually track this key metric. We have to consider things that can be measured, and use them as proxies for the customer metric. The most effective proxy is the average number of searches performed during a visit. Customers lose patience and confidence after two or three failed searches, defined as a search with no results or a search with no useful results.

There are also important operational metrics for search, which identify the specific areas where improvement is needed, and give you some insight into actual visitor experience. We recommend analyzing Web and search statistics to estimate the four key metrics ...