Tech Article of the Month

by phil on Monday Nov 21, 2005 10:54 PM

If you have to read only one tech-related article this month, read this: The Great Google Wipeout: Chronicle of a corporate death foretold.

You'll have to ignore, for a minute, the obvious pandering that any anti-Google article will have. But the article isn't about Google. It's about the shape of the Internet.

The concepts about the "deep web" or "invisible web" as created by China, myspace, A9, iTunes, facebook, World of Warcraft, XBOX Live, Skype, and forums is a very important one. All these closed sub-networks are, to many people, more important than the rest of the services provided by the Internet. Many Internet users spend more than 80% of their online time engaged in one of those networks, and wouldn't mind surrendering the rest of the web to exclusively live within that pool.

If Web 2.0 is the Light Side of the Force, the "Invisible Web" is the Dark Side.

All the research I've been doing about myspace due to my involvment with my own Social Networking Service (Leetster) has shown this concept to be true. You have musicians investing more attention into a myspace profile than their own websites because inserting yourself into the myspace web gets you more of the right kind of visibility. In the myspace universe, fan bases spread like viruses in ways that a regular website by itself can't accomplish.

And all of this content can't be found on Google. Just look at http://www.myspace.com/robots.txt -- nothing on myspace can be searched by Google. And even though robots.txt is voluntary compliance, myspace will just block the Google robots from spidering.

Sure, I have a soft spot for Web 2.0, open-source, Apache, Mysql, Perl, Ajax, Googley--all that kind of hippie digital magic. In an ideal world, that's how the Internet would function.

But I'm agnostic and face the music about spam, Digital Rights Management nazis like Sony, idiot users spreading viruses by using insecure browsers, monopolies like Microsoft ruining web standards, etc.

This is important to pay attention to, because unlike the Light Side of the force, this Dark Side is invisible in the mainstream press. And that's precisely how the purveyors of that side like it. Myspace is very poorly designed, breaks so many user interaction rules, is incredibly slow, yet the web community was noticibly quiet when it got bought for $500 million. It registered a small blip in the blogosphere, as the bloggers then went on to yapping about a new Google Maps + Ajax Remix with del.icio.us integration.

Comments

ker stalin said on November 22, 2005 7:41 AM:

Content is not king or something like that.

Travis Vocino said on November 25, 2005 2:51 PM:

Interesting read.

Also, I was prepared to be very surprised by MySpace blocking Googlebot but it appears their robots.txt is only blocking the Web Archiver now -- unless I'm reading it incorrectly.

Philip Dhingra said on November 25, 2005 11:15 PM:

Right, not specifically targetting Google.