There are so many people and good topics in the Blogosphere, its very easy to get overloaded with postings. This article is my investigation of how to blend and filter RSS feeds using Digg, how much of Digg accomplishes what I want, and thoughts of how Digg might in the future extend its functionality to use Bayesian methods to maximize the functionality to pull in the wisdom of non Digg-users.
http://www.subtraction.com/archives/2006/1129_all_feeded_u.php:
When it comes to actually keeping up with [blogs], though, I fall down. [...] On more occasions than I’d like to admit, I’m simply behind the curve.
Fazal Majid replied:
I wrote my own web-based aggregator, Temboz, precisely to deal with the problem of information overload The way I handle it is to flag particularly noteworthy posts with a “Thumbs up”. My aggregator keeps statistics on which feeds have a high ratio of thumbs up (adjusted for time, so recent good posts count more than old ones in a feed that has lost its edge). The big publishers like Apple or Microsoft are only now making RSS mainstream, but I am pretty sure the next versions will focus on managing information overload, possibly with advanced techniques like Bayesian classifiers, so the aggregator learns by itself what you find interesting and adjusts accordingly.
This led me to search for RSS+Bayesian Classifier. For which, a posting, RSS Aggregators Should Offer Bayesian Classifiers, by Bill Lovett on October 12, 2005 offers:
I was just reading Ben Kamen’s paper on applying Bayesian sorting techniques to document categorization beyond spam versus not-spam.[..] What if the next step forward was really a step from the world of email to the world of RSS and ATOM? I can’t think of any RSS application offhand that considers the potential of Bayesian sorting. You might say that it’s unnecessary in the first place, since consumption of a syndicated feed presumes interest and trust in the source.
I’d like then, a feedback loop between my RSS reader, presently GreatNews, and some upstream RSS filter. This would distinguish what I consider good vs. not good, and learn to filter in more of what I am interested in (including aspects that I find tangentially intriguing), and filter out content known to be uninteresting to me, or to people I trust. To some degree, collaborative filtering is what digg.com accomplishes. This suggests I need to subscribe less to the RSS feeds from individuals, and more to feeds surfaced by digg’s tags RSS emissions. How to do that? Well, at the rawest level, its a case of going to http://www.digg.com/ taking the URL their icon. That’s http://www.digg.com/rss/index.xml at the least granular, “what’s generally popular” level. But I want to train this. I want more of what I’m interested in, not just what the world’s general population finds amusing. I might be interested in what 4 people have dugg, rather than what 300 people have dugg, if they are the right 4 people. There are many reasons why I add feeds to my RSS reader. Friends and family: I’m always interested, others I’ll add if someone I respect talks about them. I’ve a large periphery of feeds I’m subscribed to but am only occasionally interested in. Deeper, is that its not just down to keywords, as I might not know the name for a concept. What I really want to know is the topics interesting to people like me, and how they intersect and extend my interests. This is a hard call though, and truly a goal for the semantic web, Web 3.0. Today, Digg’s general population popularity feeds is no more likely to address my particular interests than does the evening news. GreatNews, the RSS reader I use does have the facility to rate a post with digg. Digg.com, does have the concept of producing a feed dugg from your friends. This could be better. Yet, none of the people I thought might make candidates, Dave Pollard, Dave Snowden, Bill Ives, and David Crow are listed as users, so I’d want to tell it to go look at their blogrolls. Combined with my blogrolls, and the resulting network, it could filter according Dugg weightings. After 1.5 hours of experimenting, hypothesis and writing this post, I’m ready to call it a night. In the meantime I’m going to have a go at installing a digg plugin for wordpress, such as this Digg suggested javascript, this plugin by Avrian or the DiggClick improvement to show a gadget next to each posting to help/encourage my readers to promote to digg any posting as worthy candidates.
Its just occurred to me that actually Digg on my site looks pretty, but as I expect most people are reading my blog through the RSS feed they are not seeing the Digg gadget. Hmm. There must be a way.
Hi Martin,
I agree, have Blogs ranked by popularity does not necessarily help with one is interested in or finds intellectually stimulating when reading it. It goes back to the issue of relevance vs. precision in searches which to date has not been adequately been measured. Unfortunately, there is a little of loop process that only the human mind can do and software subsitutue can't.
The problem with ranking like Google or Digg by votes, which at the end of the day is the same, one can get into group think and relevance becomes low. Over time I found that I usually read 6-12 really good blogs consistently, and others when I have time.
You really need to be honest to one self…is it nice to know, hence just information, or is it truly value add to what you need to know. This opens the whole PKM box.