Link Directory: the history and future

Link Directory: The History and Future

“Quality information can be found through people, not machines.”

The link directory was the forerunner of search. But yesterday’s announcement about the shutdown of the Yahoo! Directory marks the end of an era. Why did the link directory become so popular in the first place? How did it evolve? And is there any future for a new link directory?

When the Internet was young, and there were fewer websites, it was easy for people to keep track of information. As a hobby project, Jerry Yang and David Filo started categorizing websites. They called it ‘Jerry and David’s Guide to the World Wide Web’. In 1994, they officially launched Yahoo! (‘Yet Another Hierarchical Officious Oracle’). This might feel out-dated, but back then, this personally curated approach was very important. It stood out among the inferior search engines of the time. Due to the success of Yahoo!, other directories soon followed, such as

In 1999, Yahoo! reached its peak, becoming the number one destination to start searching for information. But as the number of websites grew, it became impossible for Yahoo! to keep the directory up to date. And site owners would get frustrated that submissions sometimes seemed to go into a black hole, never to appear. Or, if they were listed, a Yahoo! editor might alter their carefully worded description — which caused them not to show up at all in response to some keyword searches. There were a number of other complications with the ‘old’ link directory systems:

  • Spam
    • Link directories were spammed because back linking resulted in higher search engine rankings.
  • Speed of search
    • Finding a website on a directory took many clicks, in comparison with search engines.
  • Paid listing
    • The link directory required you to pay in order to be listed. That made it impossible to provide a complete (and fair) overview of quality websites about a certain topic.
  • Ranking
    • The listings were unranked. The first website in the link directory listing was not necessarily the best, most viewed etc.
  • One editor per topic
    • Most directories only had a single curator who was responsible for listing all the websites, which made it a daunting task to keep the listing up to date. Additionally, the identity of the curator was not displayed, making it hard to verify his credibility.
  • Broken links / Outdated
    • Single curators, now overwhelmed by the task at hand, caused the listings to become outdated and the links to become broken.

As a result, the people-curated link directory was replaced by an algorithm. It became the fastest way to search for an overview of sites on a specific subject. It became the age of search engines.

The Present of the Link Directory

But everyone knows the limits of algorithms. Smart SEO specialists can bring a site to the top of a search engine result in no time. And that doesn’t say anything about the quality of the information that you’ll find on the sites. Today, there’s no way to tell the quality difference between the #1 search result, and the #3. Or the #20. Even Google admits that search is far from solved:

“You often hear people talk about search as a solved problem. But we are nowhere near close.”

Eric Schmidt, Google

And, despite the advent of the algorithm-based search engine, the act of categorizing and ranking websites (content curation) still remains popular. People like lists of links. In fact, 30% of all blog posts are ‘Top 10 lists’. So it is no surprise that the link directory as an information source is still operational.

The modern day link directory still faces the same challenges challenges that Yahoo! did: objectivity of listings, paying for ranking, spam and broken links. A problem that the modern day link directory can’t seem to solve.. Only multiple curators can reduce complications, and help avoid manipulation, monopolies, and missed opportunities.

Multi human curation is needed if you look at the three major problems on the Internet today:

  • Information overload and the search for quality information
  • Trustworthiness of online information
  • Inability to qualitatively rank search engine results

As often is the case, content curation finds strength in numbers. Multiple curators share responsibility for finding the latest, best and most reliable information available online. And if one curator fails, another curator can pick up the slack.

And  The Future of the Link Directory

Does the link directory have any future? Yes… if they learn from the past. The basic idea of finding information through people instead of machines is still important. It’s human nature to categorize and rank information. And, when it comes to topics they know little or nothing about, people still refer to the expertise of those in their network who know more. The guys at Facebook are well aware of this.

So, how do we provide a platform for the next (r)evolution in search? Is it possible to restore the former glory of the link directory? Is there a future beyond the search engine? Of course there is. And it’s a future that combines the power of the algorithm with the necessary human touch.

Of course, the search still begins with search engines. But then, with human curation, an actual person sifts through those search results, filters out the best content, and ranks it according to quality. To be successful, a future link directory should be:

  • Maintained by active human editors with verifiable credentials – manipulation and fraud may still occur, but everyone will know who the ‘bad guys’ are
  • Categorized and ranked by knowledgeable people (curator and crowd)
  • Searchable and easy to use
  • Equipped with a social review ranking system
  • Open (to suggestions) and freely accessible to edits and suggestions of links
  • Extensive and broad-scope to provide complete information
  • Localized for relevance to specific communities
  • Listings can be distributed via widgets on other websites

Building this kind of a link directory takes time. But by learning from the past, we will be able to build a next generation of directories that will help us navigate through the world wide web. And by working together, we can achieve maximum results from combining algorithm speed with human sensibility.

We are ready to take on this challenge!


Share this blogpost

, ,

No comments yet.

Leave a Reply

Time limit is exhausted. Please reload CAPTCHA.