Let's compare "fee web" and the "free web." Many of the academic information resources provided on the library's web site are subscription services ("fee" web) and are discussed in the Scholarly Research Tutorial. The information within Fee-based resources are not found by "free" search engines because they are hidden by a password or some other means; the spider programs can't crawl in and see what is there.
What about the "Free" web, what can we find there? Why has anyone bothered to create search tools for the free web and what do the companies that produce them get out of it? Why do "free" web pages even exist?
Ultimately, as with so many things, search tools and web sites exist because of the main motivating forces in our society: Money, Power & Influence (Bwah hah hah hah!!!!!!!)
Let's consider search tools first. They offer you something useful (a way to retrieve "relevant" pages from the free web) in exchange for ... what?
How about the providers of the web pages themselves? Why does anyone put up a web site?
Traditional domain endings provide a clue:
But domain endings aside, there are two main reasons:
Sites also exist for a variety of other personal motivations (e.g. family reunions), but the sites that show up high in the search engine rankings have often used Search Engine Optimization (more on that in module 5) to get there and have organizational motivations.
Government sites are of great value in academia, but not emphasized in search engines. The U.S. government has a mandate to provide information about itself and about what its agencies are doing. This information, referred to as "Government Documents," is outside the usual commercial interests and blatent advocacy of many other types of web sites and is considered generally both credible and reliable and is worth looking for for academic use.