February 2010 – Cover Story: Make Site Search More Personal

Subjectivity is the Key to Driving Site Search Success
By Dr. Scott Brave
If your site search is failing both you and your customers, you aren’t alone. Marketers, website managers and search professionals across all types of organizations struggle every day to produce sets of meaningful search results that truly help their user base–rather than inundating them.
Enterprise search has long been the de facto standard for websites and corporate knowledge databases, predominately employing text-matching algorithms. While statisticians might be impressed with the outcomes, users who initiate searches find themselves drowning in a sea of homogenous data.
Search professionals and merchandisers are in no better position, with product catalogs and content that is constantly growing and changing.
Site search has come a long way over the past few years, and some traditional vendors are even making impressive strides to improve relevance without having to rely as much on manual tuning. Even so, the single most important ingredient to produce relevant search results is still often lacking: subjectivity. In other words, typical search engines still have no reliable way of accurately interpreting users’ intent or determining what results are actually useful in your ever-growing sea of content.
“What exactly is so wrong with objectivity?” you might ask.
For one, the critical information needed to determine relevance is simply not in our content. The content that makes up our websites–whether HTML, flash, video, PDFs or some other type–seems like the best place to discover a match to a user’s search terms, but often it’s not. Frequently, there’s a mismatch in how a user phrases his or her query and the words content creators use–and that’s just for content that the engines are able to spider.
If we make an assumption that the content is in a format that’s spiderable and that it contains the user’s search terms, it still doesn’t mean that the documents that are returned in the results are truly relevant. The key is to understand what content is useful, when it’s useful and who it’s useful for. However, that critical information is in users’ heads, and it’s based on their experiences and needs. That’s why introducing subjectivity into search is so important.
The best approach for surfacing this subjective insight might seem to be to explicitly ask the users. This approach seems good on the surface, but upon further scrutiny it’s flawed from a standpoint of coverage, bias and inaccuracy. Some enterprise search providers are beginning to move in this direction, but have generally failed to recognize a fundamental rule: actions speak louder than words. As social science has taught us all along, if you really want to understand people, you need to watch what they do, not what they say.
A few players in the enterprise search space have even figured this out. That’s the good news. The bad news is that they neglect to look beyond search. What users are doing in search only skims the surface of what’s really going on. Anytime someone comes to your website, they are looking for something. Many times they will have done a search on Google before they came to your site. Their Google search terms are your first clue. Once they get to your site, not all users search. And even if they do, there are many steps and actions that they take after the search is performed. Every action, be it search, navigation, engaging with content–or even all of the above–is providing you with valuable clues about what’s useful, why it’s useful and who it’s useful to.
Consequences of Objective Search
Site search is one of the single most important features on websites, and a primary method for your users to find the products and content they need. The consensus among analysts is that while there have been great improvements in site search technology, the current options aren’t making the grade–on the user experience side or the search manager side. To really move the needle on the search experience, companies will have to look at new methods that fall outside of the traditional search technology realm. Independent analyst firm Gartner predicts that by year-end 2013, more than 30 percent of the most popular websites will use search technology to better target content by adapting to the user’s context.
Poor search experiences aren’t just an inconvenience for users, they are also costly. Marketing budget is spent attracting visitors to your site. Opportunity cost is sunk into the hours spent manually tuning search results and content. Customer service costs rise as users pick up the phone or send e-mails. Revenue is lost as prospects and customers abandon your site in frustration. This clearly isn’t lost on companies. A recent study by Internet Retailer reports that 35.9 percent of online retailers have plans to upgrade their site search.
Creating a Personalized Search Experience
Improving site search isn’t just about producing better search results. It’s about personalizing the search experience to each user based on their specific needs. It’s about making search subjective.
If continuing on with the status quo is no longer an option because of both the negative impact on customer experience and the high associated costs, then you need some concrete, objective points to consider when looking for a solution to the search problem:
Think not only about content, but also about context - Users come to sites with intent. That intent might be indicated in a Yahoo or Google search that brought them to your site. After reaching your site, they might then express intent or context in the pages they visit and engage with, the links they click and maybe the site searches they perform.
What else are they doing to express intent and even engagement on the site? Consider how long their mouse hovers over a piece of content. Are they engaging with multimedia content? Do they spend time comparison shopping, hopping back and forth from one document to the other? This expression of interest may span multiple searches and actions, and finding content that engages them and holds true value for the user’s specific intent or context may also take multiple steps.
By paying attention to the context of users’ visits, you can begin to fine-tune their search experiences and identify what web pages and content will be most relevant to them.
Make search social - In recent years, explicit user actions such as click-throughs, ratings and feedback were introduced to solve the issue of search relevancy. We’ve seen this on the likes of YouTube ratings, Yelp reviews and Amazon product commentary.
However, today’s advanced social search techniques have evolved to take into account how humans search for, find and consume information and products in the physical world. In this sense, the implicit feedback users are giving you as they navigate both the web and your site are much more important and telling than explicit feedback–on many levels. These implicit actions take into account all traffic, not just a small percentage. They also include information such as how a visitor arrives at a site and every action they take once they arrive, including navigation patterns, search behavior, browsing behavior and even interactions with non-spiderable content like video and PDFs.
Remember, actions speak louder than words or, in this case, search queries as well as navigation on the web and throughout your site. It’s amazing what a little listening will tell you about your visitors and what will interest them.
On a larger scale, you should also consider the macro data that can be collected and analyzed from all your website visitors. As this data is continuously distilled, virtual communities of like-minded visitors begin to emerge. Actions, patterns and tendencies associated with these communities form the basis of a collective prospective that you can use to deliver better search.
Automate the fine-tuning process - When companies do find flaws in their search results, the approach most take is to manually tune and tweak the search algorithms or the content. With this approach, companies can only focus on the most popular products and content, and they miss out on the highly profitable long tail. Simply put, it isn’t timely, scalable or all-inclusive.
Furthermore, search administrators are rarely subject matter experts and possess little knowledge about how visitors think about a company’s products and services, making it difficult to establish rules to actually help users. When subject matter experts are deployed to tune the content, it takes valuable time away from more high-value activities they could be focusing on, and still only addresses the most popular searches.
Companies need a solution that automates the fine-tuning of results in real-time, and allows the site to change the results as the behaviors of visitors themselves change, continuously optimizing the conversion potential of search.
Give online marketers “community plus control” - If we rethink how we approach fixing search and let our users or online community automatically surface the right content in the right context in real-time, we can really take our sites to the next level. Yet with that automation, companies still need the capacity for control.
Automation should do the heavy lifting for marketers, but it must have flexibility built into it. Online marketers and merchandisers will still need to create business rules to augment and sometimes even override what the community is doing.
Think outside of the box engine – Remember, search should be more than just a box. The engine is just one piece of the navigation framework that guides visitors to your products and content–no matter where the content lives or the format it takes. Perhaps it’s on the homepage, internal product pages, customer support areas, a microsite or elsewhere. It could be HTML, multimedia, a PDF or even some other format.
If companies begin to accept the limited nature of search engines and the concept of intent, they can do much more than fix search. They can start recommending products and content the second they see users expressing intent. When companies create these personalized experiences–be it personalized search results or personalized product and content recommendations across the site–they create a user experience that is engaging at every touch. And driving users to the best content every time they visit ultimately improves sales and reduces costs.
Dr. Scott Brave is co-founder and CTO of Baynote. He holds a Ph.D. in Human-Computer Interaction from Stanford University. Dr. Brave can be contacted at scott@baynote.com.


Many companies dismiss live chat under the assumption that if a visitor wants to communicate, they’ll just pick up the phone. For our first foray into this investigation, we went directly to the source and asked the Internet shoppers themselves. Our quantitative project with over 250 regular e-commerce buyers revealed several findings, including some that surprised us.
Scott and his entire team are able to beat the benchmarks for two primary reasons. The first is their impressive and continuous improvement processes. Allergy Buyer’s Club is constantly applying and re-applying website analytics in order to maximize chat’s effectiveness.
What about personalization?
As recommendations engines matured, online retailer adoption increased. Analyst estimates vary, but most suggest that 20 to 30 percent of electronic retailers today employ some type of automation for product cross-sells (either their own technology or a commercial solution). Sites like Amazon and Netflix have brought recommendations to the forefront and a growing number of commercial solutions have made the technology available to virtually all online retailers, regardless of size.
Another retailer is using its recommendations engine to automate its online best-seller and gift-guide sections. Sophisticated, automated merchandising campaigns instruct the engine to select and display the best-selling products or gift suggestions most relevant to each shopper. If I were shopping on this site, I’d see best-selling products and gift suggestions more tailored to my tastes and needs, while you would see products tailored to yours–making both of us more likely to find and buy the right product or gift. In addition, the site’s merchandising team has more time to focus on data analysis to make sure its assortment and gifts are relevant to customers.

Recalibrate your bids for non-holiday traffic. During the holidays, online merchants typically see increased traffic, conversion rates and sales, as well as higher-than-normal average order values. As a result, competition and bids for popular keywords usually increase during the run-up to the holidays. In the post-holiday period, you need to anticipate the shift downward in terms of conversion rate and average order values that results from consumers returning to their everyday habits. If you don’t, you may find yourself bidding more than you should for keywords based on ROI data from the period of holiday over-exuberance. An easy way to avoid this problem is to either look back to the period one year earlier to see how keyword prices changed as a guideline, or simply to exclude the most recent data from your bid calculations. While this means starting over, if you are using a tool which factors recent data heavily into bid calculations, you will be able to adjust your bids within just a day or two to quickly take advantage of the change in season.
Clean up your campaigns for the new year. The shift from holiday to post-holiday is a tremendous opportunity to clean house and start fresh with search marketing. Take this opportunity to remove inactive ad copy that resulted from holiday testing, and identify ad groups that are missing creative altogether and associate new copy to them. Typically, after months of testing and optimization, your campaign structure may have grown unwieldy, and will need to be reworked. Review your ad groups with an eye toward relevance and the number of keywords per group. As a rule, you should have no more than 50 keywords per ad group. Restructure your ad groups to ensure associated copy is highly relevant. This will not only help your quality scores, but it will also make it easier to perform testing and optimization in the new year.

One of the best ways to boost online sales is to personalize offers; research shows that if shoppers are presented with products that interest them, they convert 10 to 30 percent more often than those presented with one-size-fits-all offers. By personalizing offers for individual shoppers based on real-time data like clicks, shopping preferences and browsing behavior–and automatically modeling that real-time knowledge against CRM data, a customer’s individual past purchase information, top sellers, commonly purchased pairs of products and other historic data–you’ll present shoppers with highly relevant offers that interest them in that moment, increasing per-session conversion rates by 10 to 20 percent.
Geo-targeting goes hand-in-hand with multichannel analysis. By automatically analyzing both parameters, you can make extremely accurate product recommendations to holiday shoppers. For example, if you automatically analyze purchase trends at local stores and find that a certain item is sold out, you could then set up your system to recommend that shoppers coming to your site from that zip code buy the product online, mentioning that it’s sold out in stores. Alternatively, you could recommend a similar product that’s still in stock if your purchase data shows the majority of shoppers complete their purchases in-store. In addition, use geographical sales data gathered online to print in-store circulars recommending top-selling products in that zip code. There are sophisticated recommendations platforms that analyze POS, CRM and purchase data–and measure this against zip-code-level geographical trends–to recommend the best products for shoppers in certain areas.




There are millions upon millions of ways to effectively combine the respective strengths of the various digital communications channels to improve customer relationships and increase sales. In fact, simply contemplating where to start is enough to intimidate some marketers from seeking the new innovations necessary to keep their businesses ahead of the competition. However, these marketers will miss out on huge segments of the ever-evolving pool of consumers and prospects by failing to employ a multichannel strategy in their direct digital marketing.
While multichannel strategies may seem expensive and complicated, they are quite cost-effective and surprisingly simple if the right solution is available. Software solutions that combine multichannel capabilities with a marketing datamart specially designed for developing cross-channel, targeted segments is ideal.


Here are three good times to do a survey:
Give customers a reason to watch and react to your online videos by showing them how to tackle a complex home-improvement project (step-by-step) or how to spot antiques whose value will increase over time. Limit these how-to videos to three to five minutes in length, advises Rodger Roeser, president at Cincinnati-based public relations consultancy Eisen Management Group, Inc. “Make them any longer,” he says, “and your customers will lose interest and move on to another site.”