The ordering problem
Finding pages that contain the words in a query is one task. Deciding which of the thousands of matches to show first is another, and often the harder one. In the mid-1990s, many search tools ordered results mainly by how often the query words appeared in a page and where. This was simple and fast, but easy to game. A page could repeat a popular word dozens of times, and it might outrank a more helpful page that mentioned it naturally.
Human-edited web directories offered one answer for broad topics, but crawlers needed something automatic. Search builders needed a clue about quality that did not depend entirely on what the page said about itself.
An idea borrowed from scholarship
The web’s links offered one. Each link is a choice made by a person who decided to point to another page. In academic publishing, researchers had long counted citations to estimate how influential a paper was, and a field called bibliometrics studied the patterns. The same logic could be applied to the web: a page many others link to may be worth reading.
The comparison is imperfect. A link can mean “this is good”, but it can also mean “this is wrong”, “this is an example” or “this is my friend’s page”. Still, in large numbers, links often carried useful signal.
Not just counting
Counting links alone is easy to manipulate, since anyone can create many pages that link to a target. A more careful approach also considers where the links come from. A link from a page that is itself widely linked is usually treated as more meaningful than a link from an obscure page.
This leads to a circular-sounding definition: a page is important if important pages link to it. It can be calculated by repeating a process in which each page passes a share of its score to the pages it links to, until the scores settle. The general method is related to ideas in linear algebra and to how researchers modeled someone clicking links at random.
A simple illustration, with invented numbers: imagine three pages, A, B and C, where A and B both link to C, and C links to A. Page C has two incoming links and A has one, so C starts with a higher score. Because C passes its score to A, A benefits as well. Real systems apply the same idea to enormous numbers of pages, with extra adjustments.
Several people, parallel work
Credit for this approach is shared. Researchers in the mid-1990s published work on using link structure to judge pages. One line of work, by Jon Kleinberg, distinguished pages that serve as good sources of information from pages that act as good lists of links to such sources. Another, by Larry Page and Sergey Brin at Stanford, described PageRank and a prototype search engine in the late 1990s, and they went on to found Google in 1998. Other groups worked on related ideas in the same period. The story is sometimes told as a single invention, but the underlying concepts developed in parallel.
Why it improved results
Link-based ranking helped for a few reasons.
- It used outside opinions. The ranking drew on what other page authors did, not just what a page claimed.
- It resisted simple repetition. Stuffing a page with keywords did little to change the links pointing at it.
- It fit the web’s structure. The web was already a network of links, so the data was available to crawlers as they collected pages.
Because crawlers already collected links while building an index, the extra data came cheaply, as the earlier generation of search engines had shown. Users often found that for common queries the best-known and most useful sites appeared near the top. Search companies combined link signals with many others, including text matching, page quality and freshness, so link analysis was one factor and not the whole system.
The arms race
Once links affected rankings, they gained commercial value. Some site owners began buying links, trading them, or creating networks of low-quality pages to link to their sites. Others posted links in comments or guestbooks across many sites. Search companies responded with algorithm updates, penalties, and tools such as an attribute that tells engines not to count a given link. The details change frequently and are not fully public.
This cycle is part of why the field of search engine optimization exists. Some of its practices simply help pages be clearer and better organized, while others try to exploit the ranking system. The line between them is debated.
Side effects and criticisms
Link-based ranking has trade-offs.
- Popularity can reinforce itself. Pages already well linked are easier to find, so they gain more links, which can make it harder for new voices to surface.
- Links are not truth. A false or misleading page can attract many links.
- Not all subjects fit. New or niche topics may have few links at all, so other signals are needed.
These concerns are discussed in current debates about how search affects what people see, and they are one reason that modern systems use many signals rather than links alone.
A broader legacy
The idea that connections reveal importance spread beyond web pages. Similar thinking appears in analyzing social media networks, recommending content, and studying citations in research. It also changed expectations. After link-based ranking, users came to assume that a short query would often produce a useful first result, which made search a daily habit and set the stage for later changes, from personalized results to systems trained with neural networks.
The core lesson is modest. Links are not a perfect measure, but treating the web’s own structure as evidence gave early search a more reliable way to order results than counting words alone.