Since 1996 the EServer has employed the commercial search application Phantom, which permits us to create searches of local or remote websites. It employs an HTTP search crawler, similar to those used by Google, Yahoo and MSN. We can customize our searches almost completely, providing custom (X)HTML headers and footers to the entire search interface.
Our search crawler starts from one or more URLs?, reads those pages, adds all the links on those pages to a "to do" list, then proceeds through the pages in the "to do" list to read successive pages. It can limit the pages it crawls to within a specific domain or folder structure, and will return to update its catalogue as often as needed--as often as once per day, if desired.
Our search engine provides full-text searching, searching by keywords and other HTML META data, allows a phonetic searching option (which can correct for many spelling errors), can do "begins with" searching to find terms beginning with certain prefixes, offers "what's new" searching to find only documents updated in the past day, week or month, and custom searches only of contents from pages' URLs?, titles or headers, if desired.
If you're an EServer member who needs search capabilities for either a local or an offsite project, contact us and we can discuss whether the EServer's search crawler would be an effective asset for your project.