wanderingstan

We are born into this world naked and helpless.

The same could be said for pages on the web today. Like the one you’re reading right now.

When a page is born, no one is pointing to it. The newborn page has no links. It has a PageRank value of exactly zero.

If it is an insightful and talented link, others may notice it and eventually link to it. It’s PageRank will grow. The proud papa will beam in admiration.

…

But this takes time. Time matters. If you’re looking for breaking news, if you’re looking for insights into what’s happening now…you won’t wait around for all those links to build up, like so many layers of sediment that will one day form the Grand Canyon.

This is why authority matters, this is why trust matters. In my (somehow very popular!) last post, I talked about Subscription Links and Identity Links as different from normal links. It’s about predicting the future, it’s about time. If I link to the the Long Tail blog in my blogroll (bottom right), I’m telling the world that I predict that future content from this source (Chris Anderson) will be worth reading. If you trust me, you should trust my recommendation.

So what you need is a system that understands these predictions. Once you can tell the future, you can give newborn pages, and all new content, some value as soon as they are born. Now they’re not so helpless. Suppose lots of people who care about knitting subscribe to Bob’s blog. Now if Bob writes a new post about knitting, we should expect/predict that will be quality knitting material. Even before the knitters of the world start to link to it.

This is one of the many uses for authority, for credibility. Thus my explorations into using blogrolls for ranking.

…

How have others tackled this problem?

One way is to ignore the problem and value pages purely on age. Which gives you basically a ton of splogs, which is what Technorati was being reduced to a couple years ago. (Collective Intellect CTO Tim Wolters told me that he figures a third of all blogs are splogs. Another third are political blogs, and the final third is the rect.)

Technorati stemmed the tide, somewhat, with their authority alorgoithm. It uses counts of past cross-links as a proxy for trust, a prediction of future worth from the same source. But this method has many problems, which a subject for another post.

Digg and Reddit and their many many clones are essentially mechanisms for a newborn page to rapidly acquire new links. A sort of time fast-forwarder. This is not about authority. (Except perhaps in the secret ways that some Digg users’ votes (links) may be worth more than others. More on this in a yet another post.)

Regular Google search mitigates this problem a little by treating domains as sources. E.g. lots of links to various pages of xyz.com lend more credibility to future pages created on xyz.com. I suspect this is why the big blogging platforms put each blog on its own subdomain. E.g. “wanderingstan.livejournal.com” instead of “livejournal.com/wanderingstan”. This is not a concept built into PageRank, but something added into the secret Google search recipe. (Which, come to think of it, is also fodder for another post!)

…

In summary, not all links are created equal, nor are all pages created equal.

Who are the influential bloggers? Which blogs matter? What metrics would you use to even begin to answer these questions?

I’ve been exploring alternate methods of ranking in the past months. The best results are coming from examining Blogrolls. When you think about it, blogrolls compromise the links in a huge implicit trust network. For now I’m calling the calculated score “PeopleRank”. It’s kinda like PageRank, in that blogroll links from higher PeopleRank-ed blogs count more. E.g. if Om Malik has you on his blogroll, that counts a lot more for your ranking than the blogroll of your niece on Livejournal. (No offense to your niece.)

So here are the top 50 blogs as ranked by the preliminary algorithm: (Commentary and caveats follow)

Caveats of this calculation:

• Results with ~5K blogs crawled.
• Blogroll Count = Number of blogrolls this blog appears on = How many people publicly admit to reading this blog.
• The interesting datapoints are where the PeopleRank ordering puts a blog higher in the list than one with a higher blogroll count — those fewer subscribers must be “more important”.
• This crawl took Lijit user blogs as the starting seeds giving an overall tech bias.
• However, there was a period when the crawler went unchecked into what can only be called “The Mommy-o-sphere” so there is an over representation of Mom-blogs in teh dataset.
• Our blogroll detector algorithm still gets false positives, thus the high rank for “Subscribe to Feedburner” and the multiple ColoradoStartups.com listings.
• Some blogs use a Blogrolling widget for a “Web Ring” functionality, thus erroneously appearing as blogrolls. This explains most of the 100+ blogroll counts.
• We need better de-duping. Several blogs appeared until multiple URL’s, reducing the overall score.

So how is this different from existing rankings? Til now, the most common methods have fallen into one of two camps:

1. Number of subscribers. I.e. a pure democracy. Use some combination of Feedburner (for RSS readers) and some web analytics (for web readers) to count the raw number of people reading a blog.
2. Raw number of incoming links (citations). This is similar, except that links are counted instead of subscribers.

Note that neither method discriminates between the blogs “casting the votes”. It doesn’t matter if that 24th reader of your blog happens to be Scoble. Nor does it matter if those 3 citations to your blog in the last month (Technorati defines this as “very low authority”) came from Seth Godin, Fred Wilson, and Guy Kawasaki.

Initial results are encouraging, and I hope to do more analysis this week. What do you think? If you have any suggestions or ideas, please get in touch with me.