Cyberia Communications, Inc. - Newsletter Articles: Searching for Needles in the Cyber-Haystack

Searching for Needles in the Cyber-Haystack
from: Fall '96 Issue 3

The Internet continues to grow at mind boggling rates. As the amount of content on the World Wide Web increases at a rate of 1% per day, it is becoming more and more difficult to find the sites containing the information you need.

In an ideal world there would be one main index of all of the Internet sites on the World Wide Web. Unfortunately, this is not the case. Currently, there are hundreds of different search utilities out there, some are better than others.

Directory Engines vs. Spiders
There are two main types of search engines on the Web directory engines and spiders. Knowing the difference and figuring out what type you are using may save you a lot of time and trouble.

Directory engines keep a database of different Web sites. The information in a directory engine usually includes a category of some sort, the title and address of the Web site, and keywords that describe the Web site. Most Directory Sites stay up to date by providing a form on their Web site to add a new listing to the directory. Examples of the top directory style search engines include Yahoo and Lycos.

Spiders, on the other hand, are automated search engines that follow links and index some or all of the text found on sites they come across. This indexed information is then stored with the title of the page and Web site address to be searched online. Examples of the top spider style search engines include AltaVista and Webcrawler.

Knowing which type of engine you are connected to can make a big difference in the type of information that you will find. Searching the directory style engines is usually a good idea if you are looking for a specific home page, product, or company. Searching the spider-style engines is usually better when you just have a few key words and are looking for sites that may have what you are looking for. Sometimes where you end up in a spider style search can be deep within a company's Web site on additional sub-pages or product information pages. A directory-style search usually lands you directly on a company's home page.

When you find your favorite search engine, be sure to click on search help and options on that site for specific tricks on finding what you are looking for. For example, in AltaVista putting a + before your search words means the search must include those listed words!

Yahoo:
http://www.yahoo.com
WebCrawler:
http://www.webcrawler.com
AltaVista:
http://www.altavista.digital.com
Lycos:
http://www.lycos.com
Cyberia's Search List:
http://www.cyberia.com/searchtop10.htm