August 2009 – Cover Story: A Hyper-Efficient Market

Transitioning from click-fraud detection to traffic-quality assessment

By Richard Sim

Online advertising–a medium where a tremendous amount of data is readily available and performance can often be measured down to the penny for a given ad–is on the verge of a revolutionary shift. Ad buying and selling is rapidly evolving toward a “hyper-efficient” marketplace driven by near-perfect information and real-time transactions.

Buyers and sellers are increasingly leveraging technology to make informed decisions to meet their particular performance requirements. Meanwhile, data storage costs are going down, database technologies and analytical tools are becoming more powerful, processing speeds are getting faster and online ad spending is growing year over year. These trends are creating a “perfect storm” for online advertising and revolutionizing the way the industry buys and sells inventory.

Armed with high-powered, data-driven systems, cutting-edge machine-learning techniques and predictive-modeling methodologies, a new class of ad sellers, marketers and solutions providers is rapidly emerging to capitalize on this new opportunity. Much like the emergence of hedge funds and the automated trading desks of Wall Street in the 1990s, the online advertising industry is experiencing a dramatic transition in which access to data and the tools to leverage that data are separating the sophisticated, efficient clearing houses from the outdated, speculative arbitragers.

One of the key enablers of the development of a truly efficient ad marketplace is the increasing use of analytics to predict user intent. Predicting user intent has historically been a core component of measuring traffic quality for the purpose of eliminating invalid or fraudulent traffic, but the focus is rapidly shifting toward optimizing the positive end of the quality spectrum to drive better ad performance.

In the Beginning, There was Click Fraud
Click fraud can be defined as the act of producing clicks or impressions that have no economic value to the advertiser due to malicious intent on the part of the clicker. A click can be fraudulent when the clicker has no intention of converting, giving the advertiser no chance to reap a return on the investment in that click. Click fraud clearly reduces marketers’ ROI, as ad spend is funneled to malicious perpetrators in lieu of facilitating the acquisition of legitimate customers.

To date, ad sellers have addressed the issue of click fraud with varying degrees of success. Each of the major search engines–including Google, Yahoo, Microsoft and Ask.com–has settled or has pending litigation on click-fraud-related cases brought by advertisers. Almost all ad sellers have an in-house filtering system to rid their network of fraudulent click activity. However, the amount of time, energy and resources required to keep up with the dynamic nature of online threats is an increasing burden, especially as it forces trade-offs between growing the business versus investing in cost centers.

In recent months, click fraud has re-emerged as a top concern for consumers and marketers alike. Facebook is facing widespread advertiser complaints about undeterred click fraud in its network. Microsoft recently filed a $750,000 lawsuit against a family of three for committing competitive click fraud. The FTC, in an unprecedented move, recently shut down the operations of the Pricewert/3FN ISP, known as a safe harbor for extremely malicious entities involved in online crimes such as child pornography, click fraud and spam. It’s clear that click fraud remains a serious threat to online advertising. And as ad dollars continue to shift from traditional media to online, this threat will only loom larger.

Why is click fraud such a difficult challenge for the industry to address? There are several reasons. First, click fraud, like any online threat, is highly adaptive in nature. While fraudsters commonly take the path of least resistance when targeting vulnerable networks, they do adapt as needed when options run dry. Customers have reported blacklisting IPs and user agents leveraged for malicious attacks only to find that new ones pop up within minutes.

Second, the black hat community is highly networked and collaborative, and readily shares data about vulnerabilities and successful fraud strategies. Evidence of successful fraud campaigns (such as checks received from ad networks) are commonly touted in black hat forums and websites, along with detailed reports on how these campaigns were executed. This level of collaboration and information sharing is yet to be replicated on the security and defense side. Finally, click-fraud detection is simply very difficult to accomplish. Click-fraud detection fundamentally boils down to answering the question, “What is the intent of the person who is clicking on this ad?” Clearly this is not an easy question to answer, and the answer is difficult to validate.

These challenges have created a market opportunity for companies (like mine, Anchor Intelligence) to emerge and provide enhanced protection and security for ad buyers and sellers. Anchor Intelligence’s flagship product, ClearMark for Traffic, is a click fraud detection system, which, at its core, has a sophisticated machine-learning and rules-based system designed to predict user intent. ClearMark enables search engines, networks and publishers to protect advertisers from fraudulent click activity by identifying invalid ad clicks in real time. Customers are empowered to reduce these threats by withholding payouts to the fraud’s perpetrators and evicting fraudulent publishers from their networks while preventing advertisers’ budgets from ever being negatively impacted.

LookSmart Increases Advertiser Satisfaction

Company Background Founded in 1997, LookSmart helps marketers reach their audiences online. The San Francisco-based search advertising network and management company provides targeted, pay-per-click search advertising products and services to advertisers via its award-winning LookSmart AdCenter platform.

Approach In an effort to drive higher advertiser satisfaction and improved ad performance, LookSmart engaged Anchor Intelligence to leverage its ClearMark real-time traffic-scoring system. LookSmart originally licensed ClearMark to score all advertising traffic within its network, further strengthening the company’s efforts to safeguard advertisers against illegitimate and fraudulent traffic. LookSmart is now leveraging ClearMark’s traffic quality scores to offer value-based pricing to its advertisers and partners.

Results Through its collaboration with Anchor Intelligence, LookSmart began to see positive results almost immediately; since implementing ClearMark, LookSmart has made significant enhancements to the structure of its network and has improved the quality of traffic available to advertisers. The company is able to better monitor and evaluate traffic in its network to ensure that suspicious activity is addressed and that distribution partners who undermine the performance of LookSmart’s network are dealt with promptly. “ClearMark has given us further insight to make confident decisions about our traffic sources, so we can protect our advertisers and partners,” says Michael Schoen, LookSmart’s vice president and general manager of advertising platforms. By reducing advertisers’ exposure to invalid traffic, LookSmart has also seen improvements in advertiser ROI and, correspondingly, advertiser satisfaction with its network. “A number of our advertisers have recently expressed their pleasure at the enhanced performance of their campaigns,” affirms Schoen.

Going Forward LookSmart’s partnership with Anchor has paved the way for new opportunities. LookSmart is now working to leverage Anchor’s data and analytics in order to understand and capitalize on the entire spectrum of traffic quality. By utilizing Anchor’s user-intent prediction capabilities, LookSmart hopes to further improve the quality of network traffic and the ads that it serves in real-time.

The Next Generation: Traffic Quality and Predictive Analytics
As data and analytics have grown in sophistication and scale, cutting-edge marketers have realized that addressing click fraud is only the first step toward optimizing ad spend. Marketers have recognized that applying the same data and analytics toward identifying, understanding and capitalizing on the entire spectrum of traffic quality is the most lucrative opportunity available to their organizations. As a result, Anchor has recently been encouraged by our customers to leverage the predictive analytics of ClearMark to assess the entire spectrum of traffic quality, including high-value clicks.

Traffic quality is a measure of the value of a click or impression along a continuous spectrum from good (likely to convert) and bad (invalid and/or fraudulent). Traffic quality depends on a combination of interdependent factors including user characteristics, the context in which a user is interacting with an ad, the ad creative and, of course, timing. The more information that is available about each of these factors, the better the optimization potential.

Companies like ours with the ability to detect whether a user who is clicking on an ad has malicious intent can also apply this technology to predict whether the intent of a user is authentic and furthermore, if that authentic intent is relevant for the advertiser. As customers have recognized the game-changing opportunity present in applying technology to assess and optimize the entire spectrum of traffic quality, companies have begun to enhance their intent-prediction capabilities to change the way online advertising is bought and sold.

In the not-too-distant future, an advertiser will be able to make ad-purchasing decisions based on the predicted value of a given click or impression to that advertiser, as opposed to how it’s done currently–based on general prices set by disparate marketplaces. Many of today’s standard targeting frameworks are founded on the assumption that applying known characteristics about populations of users enables advertisers to improve the performance of these ads over untargeted ads. These frameworks have been successful not only because they improve ad performance, but also because they make sense. It would be hard to find anyone who would argue against the notion that males between the ages of 35 and 45 who make over $100,000 per year are more likely to buy a luxury car than the average user.

However, those obvious dimensions are unlikely to have sole predictive value when measuring expected performance. Going forward, we believe that advertisers will also be able to target inventory based on “value tiers” (groups of traffic dynamically identified based on their propensity to convert), rather than limiting themselves to standard targeting frameworks like demographic, geographic and behavioral targeting. These value tiers will incorporate standard targeting parameters in addition to a limitless number of other dimensions that have a direct impact on a particular ad event’s propensity to lead to conversion for a given advertiser. And the more access to data and analysis about both obvious and non-obvious dimensions an ad server has, the greater the expected lift it can generate.

While this concept may sound somewhat audacious, Anchor Intelligence and its customers have already begun to partner together to seize the opportunity of traffic quality assessment and targeting–improving the performance of ad inventory by leveraging intelligence about traffic quality.

For ad sellers, this has not only helped eliminate poor quality traffic from their networks, but also enabled them to capitalize on their highest quality traffic sources. Leveraging data and analysis about various dimensions such as IPs, user agents, ad placements, behavioral profiles and websites, the performance of advertising on ad networks and search engines can be improved by scoring ad events on a continuous spectrum of traffic quality. In fact, by leveraging data typically unavailable to traditional ad buyers and sellers, Anchor Intelligence has enabled search engines to enhance ad-matching decisions to improve quality scores from advertising partners and ultimately improve advertiser ROI.

Characteristics such as the ratio of the ad footprint to the content footprint on a referrer site, the types of browser plug-ins installed on a user’s machine and the ISP that provides a user’s Internet access are all data points that impact the likelihood that a given ad event for a individual advertiser will lead to a conversion, but are essentially ignored by standard targeting methods.

Not only can ad sellers leverage traffic quality intelligence, ad buyers can improve performance by supplementing standard targeting dimensions with non-obvious metrics that also impact conversion probability. As most advertisers and ad servers know, historical conversion is one of the most reliable indicators of expected performance. However, conversion data is not always readily available for marketers to leverage. We have worked with some of the largest online advertisers to optimize their bidding strategy for head, belly and long-tail keywords by building out conversion probabilities even in cases when historical conversion data does not exist.

By leveraging a machine-learning algorithm using our extensive reputation database, proprietary network intelligence and traffic-based observations to score each inbound ad click and optimize keyword bids, the performance of advertisers’ ad spend can be improved five to 10 percent in a matter of weeks.

Our company and others have just scratched the surface of leveraging predictive analytics to assess traffic quality for ad buyers and sellers, but early results are extremely promising. As search marketers around the country scrutinize advertising expenditures and opt out of low-performing and stale campaigns in an effort to maximize ROI, ad sellers and buyers can now capitalize on the traffic quality of each ad to ensure that every click or impression results in positive value for the business.

The days of media buying through personal relationships and handshakes over expensive rounds of golf are quickly disappearing. The market as a whole is embracing a shift toward a hyper-efficient marketplace. A new crop of data providers, audience-measurement companies, yield optimizers and ad exchanges have introduced new capabilities to facilitate information exchange and real-time decisioning.

Companies like BlueKai, Acxiom and Experian are offering access to their unique datasets to optimize targeting. Other audience-measurement companies like Quantcast, Compete and AudienceScience are providing detailed profiles of audience types to facilitate enhanced targeting. Yield optimizers like Pubmatic and Rubicon Project are introducing real-time decision-making capabilities at the time of an ad event to optimize publisher payouts. And finally, ad exchanges like ContextWeb and AppNexus are creating open platforms to facilitate data sharing and more efficient marketplaces.

Each of these camps promises to introduce efficiencies to the existing ad buying and selling process. Anchor Intelligence intends to leverage its technology platform, data footprint and its proprietary analytics to predict user intent and capitalize on the emergence of a hyper-efficient marketplace. For search marketers seeking to optimize their online spend in the wake of an apparent resurgence of fraudulent attacks on their ad dollars, a holistic understanding of traffic quality will enable them to not only survive these challenging conditions, but in fact thrive while their counterparts lag behind.

Richard Sim is vice president of product and marketing at Anchor Intelligence, a traffic-quality solutions provider utilized by ad networks, search engines and advertisers. Sim holds an MBA from the Wharton School of Business as well as a BA in Political Economy from U.C. Berkeley. He can be reached at richard.sim@anchorintelligence.com.