Friday, September 10, 2004

Scoble on RSS costs

Scoble talked about the costs of RSS the other day. He compares the traffic costs associated with RSS readership to the costs associated with HTML readership. For the few reading who don’t know what I’m talking about: web pages that you view through a web browser are in HTML. Many weblogs (and an increasing number of news sites) publish a variation of their website in an XML file called an RSS feed. The RSS feed is designed to be subscribed to by an aggregator, a program that will periodically check the RSS feed for any new content. If there’s new content, it will download it and preserve it for later viewing by you.

Scoble points out one of the challenges inherent in the RSS model: whereas someone might visit your site once a day, their aggregator will often visit you once an hour (to see if you’ve said anything new). Over time, this disparity can add up.

I noticed this on my site as well: half of my bandwidth each month was RSS traffic. Back in May, I decided to check out a new service called FeedBurner. It’s a service that will host your RSS feed for you, as well as provide you with some great readership statistics. In the past three months I’ve been using FeedBurner, here are some statistics for you:

  • In April, 60% of my page views at my site were my RSS feeds. This represented nearly 1 gigabyte of bandwidth consumed. (About 30% of my traffic that month.)

  • Today, FeedBurner updates my RSS feed only when I ping it to let it know I’ve updated; all requests for my RSS feed from subscribers are redirected to Result? In August, just 3% of my page views were to my RSS feeds (all from and the bandwidth consumed was almost nonexistent.

  • Courtesy of FeedBurner, I know that I have approximately 400 people who subscribe to my RSS feeds (they can tell this by analyzing queries to my RSS feeds, applying known patterns of how often certain readers request files, etc.). Just based on raw numbers of accesses in the server logs, there is no simple way to discern this otherwise.

  • FeedBurner also tells me how many click-throughs I have (people who click through from the RSS feed to read the post on my site); this helps identify what content is most interesting to readers.

  • FeedBurner does some nice optimization of your feeds, tweaking the formatting of the feeds dynamically, so if FeedReader is pulling the feed, they might strip out some markup that they know FeedReader can’t handle. If NewsGator pulls it next, they’ll add in some functionality they know NewsGator can take. It’s very slick.

Bottom line: FeedBurner addresses the bandwidth consumption issue in a simple manner. They transfer all bandwidth consumption from you to them; the higher-traffic your site, the more dramatic the impact. In addition, you get some nice reporting on the readership of your feed, which help identify how your readers actually use your content. FeedBurner also addresses interoperability — if your aggregator supports only certain syndication formats (say, ATOM), FeedBurner can seamlessly address that by converting from one syndication format on the fly to another.


  1. Does bloglines register as one subscriber where, in fact, you have 91 readers subscriber THROUGH bloglines?

  2. Barry - They actually have some ways at guessing re: Bloglines subscribers, but no, they can't get a specific tally. (At least, that's what I think. I'll ask one of the FB guys to comment.)

  3. Barry - actually, specifically with Bloglines, we know exactly how many subscribers you have because they are nice enough to send it down in the user agent as part of the request, so if you have 91 bloglines subscribers, we count it as 91 bloglines subscribers.

    Where it gets tricky is when desktop client aggregators do not let us know when they are making a request on behalf of a new user versus making the same request over and over again (the protocol for retrieving RSS/Atom specifies this, but most clients don't respect it) - and counting requests by IP address isn't an exact science either because there may be many clients behind one firewall that all look the same to us.

    All that aside, we've catalogued all 600+ of these RSS/Atom clients and know their particular capabilities, so where it's exact, we count it exactly, and where it's not exact, we've developed algorithms for calculating "circulation" that we think are really accurate based on a lot of testing and a lot of statistics and data we have.

    Hope this helps - feel free to give the service a try at

  4. I agree with that, Feedburner is a nice service for those not being able to manage their own RSS for diffrent reasons.

    However, giving out-house the crucial item of generating rss might readers to loose control on an important part of their website.

    In fact, instead of propagating your own domain in the RSS URL, you have to propagate the feedburner domain if you use feedburner to generate the feeds.

    Controling bandwidth in your own feeds is not that hard: just make sure that your web server answers with a "304 not modified" when a request is coming too often from same client. Enabling compression does the rest to limit bandwidth consumption.

  5. Marcel - I disagree. I have not changed the URL for my feed, just use .htaccess to redirect requests for my feed's URL to FeedBurner. If I ever decide to stop using FeedBurner, I just remove that line from .htaccess.

  6. Serving RSS Feeds With Feedburner

    Rick Klau points out several advantages to using FeedBurner for making RSS feeds available, especially for popular sites where bandwidth costs are an issue. The RSS feed owner uses .htaccess to redirect requests for their feed to FeedBurner. Here's Ric...