Solving The Feed Search and Partial Feed Puzzle

One of the nice things about all this “Web 2.0” stuff is that we get some new problems to solve. Let me outline a problem that’s been bugging me for a while now that was brought top-of-mind by the launch of Google Blog Search a few weeks ago. I’ve also got a possible solution I want to put out there for feedback.

Here’s the situation:

  1. Ad-supported sites rely on people seeing the ads on their site. That’s how they make money. That’s good.
  2. Feeds allow ad-supported sites to notify past readers (subscribers) that there is new stuff at the site to see (along with the ads that support the content). That’s good.
  3. If an ad-supported site publishes a full feed with all the content, ad-free, they don’t make any money. That’s not good.
  4. If you put the ads in the full feed it kills much of the value of the feed to the subscriber and becomes very hard to measure. That’s not good.

So a partial feed (while not the preferred choice of subscribers) is the logical compromise. Subscribers are notified of new relevant posts and can easily click-through to see the ad-supported content. A compromise, but a good thing.

And here’s the problem with that situation:

  1. Feeds (through ping services) also act as notifiers for aggregators and search services. Because this makes it possible for prospective readers to find a publisher’s content, this is good.
  2. But a new class of services only looks at what is in the feed to assess the content. So anything not mentioned in an ad-supported site’s feed is not crawled and therefore not searchable by users. For both publisher and reader, this is a bad thing.
  3. The problem is made worse by the fact that one of the best new services, PubSub only reads feeds, not the full related posts. That’s bad.
  4. But what is really bad is that Google Blog Search is only crawling feed content, not the original posts.

So the essential problem we’re faced with is you need to produce a full feed so that people who might be interested in your content can find you when using ping-centric search tools. But producing a full feed means that regular readers can avoid ads on your site by viewing your content only in their feed reader.

And finally, my suggested solution:

  • Create a “Public Partial Feed” that is easily available and conspicuous on your site. Make it so that auto-discovery can find this feed.
  • Make a “Ping-friendly Hidden Full Feed” that is hard to find unintentionally and have that feed sent to ping-centric search tools.

Is anyone doing this? Are there any issues I’ve missed in using this approach?


Originally published at www.onedegree.ca on September 28, 2005.