Users have fought back with ad-blocking add-ons to their browsers. So far, the only browser that fully supports such add-ons is Firefox. Google Chrome will also introduce an extension scheme in version 4 of their browser (already in beta and available for download), and there are a couple of ad-blocking extensions already available for it (AdThwart, AdBlock) , though neither is as good as Ad Block on Firefox just yet. The market penetration of such extensions remains poor – it’s estimated that only 4% to 8% of Firefox users actually install and use Ad Block, though it’s been around for years. And Firefox (all versions combined) is installed on less than a third of all computers out there. So at this time, ad blocking extensions are not a major threat to ad revenue.

However, it seems likely that the popularity of such ad blocking extensions will continue through word-of-mouth advertising. Some of their success is linked to the success of the Firefox browser itself, which has continued to gain market share. Google Chrome is also a fast growing browser, and when it starts to offer extensions (in a non-beta version, which will be in a few months), it will likely also carry some ad blockers along with it. Anyone who has used an ad blocker knows for himself how immediately and profoundly it alters the web-surfing experience for the better, making surfing much more pleasant. So it’s not like ad blockers are a hard sell. I think that even if ad blocking isn’t a revenue threat to the ad industry now, it will become a threat in time.

This has hurt revenues in the web publishing industry. It’s not just extensions like AdBlock, which has a fairly negligible market presence – it’s more that users are simply ignoring ads. Coupled with the fact that non-ad revenue is also down, since fewer people buy newspapers or magazines; the result is that publishers are facing serious problems, shown in the number of bankruptcies, closures and layoffs in the industry.

Now I am not one of those who have a pathological hatred of  ads. I realize that much of the valuable content on the web is not produced by part-time hobbyist bloggers; it’s produced by people to whom publishing is a profession. These people don’t have other day jobs – they feed themselves and their families through the content they create for the web. They deserve to be paid for their work. And if they are not paid, they will be forced to turn to some other line of work to pay their bills, and the web will be poorer for it. For example, although news bloggers provide a lot of value added service, without the original news gathering efforts from paid journalists, there is only so much bloggers can do to recycle each other’s stories.

So far, very few people have figured out how to make money off a pay-to-view site. This may eventually change if Rupert Murdoch has his way. Any site that makes money off the internet would probably love to put their content behind a pay wall if they could, if they thought it would increase their revenue. But this has not worked for the large majority of people who have tried – it has only made viewers turn to alternative sources for similar content. Alternative sources are made viable by news aggregators like Google, who undermine pay wall protection by leading users to competitors who aren’t behind a pay wall. Clearly, things will not change until publishers band together in significant numbers and act together.

I don’t know when or if this will happen. However, I know that markets are driven by money, and eventually, a business model that produces poor revenues will be supplanted by one that produces better revenue. Whether that will be pay walls or court injunctions against the likes of Google or something else, I can’t say. But something will change.

For decades, ads supported a large fraction of the publishing industry. Over time, they became a proven way that worked for both publishers and readers. Publishers got their revenue; readers got cheap content, subsidized by ads. This worked well in the print medium, but somehow it has not translated well to the internet. I’d like to ask why, and suggest ways in which it could be made to work again.

I think the primary reasons why ad revenues have started to fail are:

1. Ads have become intrusive to the point that they are an annoyance even to those readers who might be inclined to buy, and people are starting to tune them out.
2. There is no good system for pricing internet ads, as there is for print and TV. Simple pay-per-click schemes don’t go far enough. They don’t sort out the public into marketer-friendly demographics where it’s possible to know well in advance how much an ad should cost, and what’s the most effective place to place it.

The first is relatively easy to deal with, but it requires some coordination and a great deal of work by someone with credibility enough to make it work. This is what I would propose. First, work out a standard that takes the annoyance factor out of ads. Second, create a certifying authority that vets ads and approves those that are non-annoying, and gives them a digital certificate of approval. Third, create a mechanism in ad-blocking products that permits certain ads to go through. Fourth, create a scheme that lets content providers serve content only to browsers which allow certified ads to appear.

Each of these steps has major technical and organizational hurdles to cross, and even if they are crossed, it will never be 100% effective. However, it does not need to be 100% effective. It only needs to be reasonable enough to convince a majority of users, and non-threatening enough that people don’t care to expend a great deal of energy to circumvent it.

Let’s take each of these steps one at a time.

Taking the Annoyance out of Ads

Here are some rules for ads that would make them much less annoying for me. No doubt others could add items I have forgotten:

1. No pop overs or pop unders. Nothing that would open another window or tab in your browser unless you explicitly and deliberately click on the ads
2. Ads must not cover up or obscure content that the user visited the page to read. You must not require the user to acknowledge the ad by clicking a “close” or “go away” button before he is allowed to read the content. That only makes him hate you, and hate the ad provider.
3. No flashing, no animation, no movement of any kind. This is like asking a person to read a page in a book while ambulance lights flash and flicker in the background. They are distracting and make it harder to read the page.
4. No audio. Pages should never load with any audio playing. Audio should only be enabled after a deliberate and unmistakable click from the user, specifically requesting the audio.
5. Goes without saying, but I’ll add it anyway. Ads must be vetted for malware, browser hijacking, and similar exploits.
6. No mixing ads with copy. In other words, don’t use fake underlines and highlights on content, trying to fool the reader that such highlights or underlines are relevant to the content, when in fact they only point to ads.
7. Privacy concerns: although it’s useful for ad servers to keep track of users, and I would not suggest that they do away with this powerful mechanism that the internet provides for targeting ads, there should be some easy way for users to opt out of it. If you don’t provide it, users will build their own, and then you will be worse off than if you had just provided it in the first place.
8. 8. Ads should not unduly slow down page loads. More specifically, content should never have to wait for a slow ad server to be displayed.

I’d like to mention one more point, but separately from the list. That point is ad ratio. If your page is 75% ads and 25% content, your users will notice that. They are not stupid. This is not something that should be enforced by the ad certifying agency; it’s up to whoever owns the web pages in question. This is where that second agency I mentioned, the one that figures out ad prices, comes in. It may well be more profitable for some little company to create pages which are 4/5th ads and 1/5th content, in the hope that the greater number of ads will offset the decreased number of users. But this should reflect in the cost of those ads, which pay-per-click systems don’t always do. Print and TV have long-established means to price ads, with companies that survey and measure such things down to the last penny. The internet needs the same.

Certifying Agency

Not much to be said here. It could be a private corporation, or it could be a volunteer effort. There is really very little work to do, so long as the guidelines are very clear. Either an ad passes those guidelines and receives the certificate, or it doesn’t. Checksums and hashes can make sure that the certificate is paired with a specific ad, and that the ad producer can’t modify the ad after it’s been approved. Again, the point is that the guidelines should be clear enough that even a machine could figure out whether to approve or reject an ad.

Ad Blocking Software

This is the easiest part. Ad blocking browser extensions (and perhaps someday the browser itself) would have a built in mechanism that allows the user to select which ads will come through, simply by checking a box in the configuration. The browser or extension downloads the certificate from the authority, and vets each ad to make sure it’s been signed with the proper certificate.

Ad Serving

This is the hardest part to solve, technically. This is what the web server needs to implement at the content end. The goal is to serve content only to browsers which allow certified ads to go through. To figure out how this might be done, consider how ad blockers work now.

There are several mechanisms, depending on the browser and extension. The best implemented is Ad Block on Firefox. This mechanism is based on a feature built into the Gecko engine, which is used by browsers like Firefox or Thunderbird. The feature is called “content policies”, and it’s simply an object that gets called whenever the browser is asked to load a page. It looks at the content (the address, plus some other stuff), and based on that, it decides whether to allow the content or not. Extensions like Ad Block simply define content policies. How they do so varies. One way is by maintaining a list of ad providers, and simply blocking their IPs. This is a really good way, because it prevents the browser from even downloading the ad, so you save bandwidth too, and the page loads faster. Another is by searching the text (through regexps) for stuff that is likely to be found in ads and not in content.

Chrome extensions at present don’t allow a mechanism for ad content not to be downloaded, since Chrome’s engine (Webkit) doesn’t have that feature from Gecko. So the ad content is downloaded, but after download the extension checks it against similar publicly available lists and also examines the text, and hides content that it thinks is an ad.

There are problems with both approaches, and also with the implementations. At present, it’s possible to create scripts on web pages that can detect the presence of Ad Block, and simply refuse to serve pages to browsers which have Ad Block installed. However, it’s still fairly uncommon to see a site that actively refuses to serve to Ad Block-enabled browsers. More often, it’ll just post a notice to the web page with something like “we know you’re using Ad Block, if you enjoy the content on this site, please consider turning off Ad Block or donating – here’s our PayPal button”.

It’s really not practical to expect users to turn Ad Block on and off individually for different sites, depending upon how much they like the content. Some will do it, but most will just keep Ad Block on, and ignore your message like they ignored the ads previously.

It’s also possible for users to circumvent the ad-block detection. But this also leads nowhere – it just produces a constant state of war, where content providers continue to develop ever smarter ways to detect Ad Block and users continue to invent new ways to circumvent it. This process never ends, as we’ve seen with various DRM schemes. No one really wins in the end.

Instead, we need to come to a consensus. This will only happen if content providers understand the very real problems faced by users, and take them seriously. And if users understand that content providers need to make money to continue working, and ads are a much better alternative than pay walls, or killing the goose that lays the golden eggs.