Search Engine Manifesto

April 3, 2019

Early Search Engines held an index of a few hundred thousand pages and
documents, and received maybe one or two thousand inquiries each day. Today,
a top Search Engine will index hundreds of millions of pages, and respond to
tens of millions of queries per day.
Before a Search Engine can tell you where a file or document is, it must be
found. To find information on the hundreds of millions of Web pages that exist, a
Search Engine employs special software robots, called spiders, to build lists of
the words found on Web sites.