Seek and Ye Shall Find...Maybe

by John Mather

One of the most difficult things for newcomers to the Web to come to grips with is the sheer scope of the Internet. You've been told that everything you want is there...but how do you find it?

On many of the other online services, there is some feeling of centralization—a place to start, a place where you can find direction. The Web, on the other hand, has more than a touch of electronic anarchy. No one's really in charge, or even in control. No one knows where everything is and, even if they did today, they won't tomorrow. The Internet continues to grow at 10% per month and each new server, each new user, is an independent entity, linked only by their ability to communicate.

Actually, the title of this article is not intended to imply that finding information on the Web is a matter of luck, serendipity or "brute force" browsing. It implies that you can find what you need, but only if you know how!

So, how do you find what you are looking for? Luckily, because of the very nature of the Web, a number of innovative mechanisms—usually known under the generic name "search engines"—have been developed to solve the problems.

Of course, the Web being what it is, there are now quite a few search engines and not every engine has efficient access to all the information you might want or need so, to ensure that your research has given you access to everything (or almost everything), you may want to interrogate several engines as your search progresses.

But, if the Net is so huge and amorphous, how do the engines work? Strictly speaking, they fall into two categories—directory-based and Spider-based. There are probably other types, and many that combine features of both but, for the sake of explanation, let's limit the description to these.

Directory-based search engines consist of a huge tree of topics, starting with very broad-based categories such as Arts, Science, Business, etc., with each topic leading to more detailed topics until a point is reached where the topic is narrow enough to start displaying actual information. Moving down the topic lists is performed by executing normal Web hyperlinks so searching a directory-based engine is extremely simple.

Examples of directory-based engines are:

Spider-based engines are really quite clever. A Spider (sometimes called a Worm but, given that we are talking about a Web, Spider sounds more appropriate somehow) is a software robot that crawls around the Web poking its nose (if spiders have noses) into everything it finds. Each time it discovers a new Web site, it requests the pages from the server and takes them back to the engine. The engine then extracts all the key words that it thinks might be of interest and stores them in its search mechanism along with the reference to the relevant Web pages. The next time you search for a key word, up will pop the new reference.

Examples of spider-based engines:

Depending on the browser you are using, you probably won't have to go looking for the engines. Many of the most popular browsers have links to both directories and search engines directly from their "front panels." For example, Netscape has buttons marked "What's New," "What's Cool," "Net Search" and "Net Directory."

What's New and What's Cool are direct links to Web sites which have just appeared (note that, with the volume of additions to the Web, "just" may mean three or four weeks) and Web sites which the people at Netscape have judged to be "Cool."

Other "What's New" sites abound, including NCSA's site and Yahoo, which is one of the largest and best-known directories on the Web.

For out-and-out searching, it really doesn't matter where you start because, increasingly, the engines all reference each other. Netscape's Net Search button, for instance, leads to a page which prominently features InfoSeek but also gives access to almost all of the other major engines. Similarly, the Net Directory button leads to a page which features Excite but includes access to many of the other major directories.

So let's say we reached InfoSeek and we want to get information about J.R.R. Tolkien, the writer. Enter "Tolkien" in the search box and press Enter, or click the button marked "Search." After a few seconds, a new page appears (usually headed with an advertisement) announcing that so many references to Tolkien were found and displaying the first 10.

Each reference is formatted as a title (which also acts as a hyperlink to the underlying Web site) plus a couple of lines describing the site. Often the description ends with "See also...similar pages." An example is:

The J.R.R. Tolkien Information Page (Score 69, Size 18k)

Everything you want to know about Tolkien—on the Internet. Includes answers to frequently asked questions, mailing lists, online texts, and pointers to archives and games. (See also Tolkien, J.R.R., Fantasy, Similar Pages) [Author's note: By the way, this site is really neat.]

At this point you have a number of choices:

Out of the 10 items displayed, you may find one or more that interest you. Click on the item title to proceed immediately to the relevant Web site. Click the browser's Back button to return to the search list.
Alternatively, you may decide that one of the items is interesting, and that you would like to explore it and pages that have a similar focus. Click one of the "See also" references.
You may decide to continue with the next 10 references. Click the link at the bottom of the page marked "Next 10 Titles (11 to 20)." Or "...(21 to 30)." You get the point.
You may wish to refine your search. This means adding changing or adding to your key words in such a way as to achieve a narrower search. For instance, you may decide to add "hobbit" to Tolkien as a key word.

At this point, we should take a look at how InfoSeek interprets the key words. This information can be obtained by clicking on the "Search Tips" link which is displayed next to the search box.

In brief, if you enter a single word, InfoSeek will search for that word in the specified data sources (more about that in a minute). If it is a proper name, you should capitalize the first letter; if two words are entered with their first letters capitalized, InfoSeek will treat them as a single name. Use commas between the words to defeat this combining effect. To combine words into a single phrase, surround them with quotes. Link, with a hyphen (-), words that must be next to each other.

If a word must appear in a document in order for that document to be found, precede it with a plus sign (+); if it is not necessary for the word to appear in the document, precede it with a minus sign (-).

A set of option buttons beneath the search box allow you to limit the sources to be searched to "All Web Pages" (default), "NewsGroups," "UseNet FAQs," "Reviewed Pages" and "Topics." Topics can be picked from a set of directories below the search box.

While each engine differs in detail, it allows access to a link describing the specific rules which are used to make a search. None of the engines I have looked at require the user to have a degree in Rocket Science.

In the next issue of WWWiz, I will expand the topic of searching the Web to more advanced search techniques, searching other parts of the Internet and finding the Internet addresses of individual people— yes, it's possible.

In the meantime, start using the engines to really start exploring the Web for the information you are looking for, and take the "maybe" out of "Seek and ye shall find."

John Mather is the President of Winformation Software, which markets Appeal, a Windows data base utilizing Winformation's revolutionary AutoRelational technology. Appeal combines the power of Relationality with unprecendented ease of use. Visit Winformation's Web site.