Sphider

Sphider is a "lightweight search engine" that was created using PHP and uses MySQL as the database. It can be used for accommodating small to medium search engine functionality that can be added to your website.
Sphider does a database dump of the Sphider MySQL database. it has the ability to create searching functionality instead of browsing lots of pages.
Sphider is not full-featured, and not as comprehensive as most search engines. This searches within and across given websites.
Sphider works in a LAMP application. There is a server working behind the scenes.
When the PHP code executes, it will grab web pages from the internet and store it in MySQL.





You give it a list of sites and then it goes out and indexes it. It grabs the text from the web pages and stores it into a database.
It also gives options to fully index or to-depth indexing. This means Sphider wants to know what to do when it gets to a link. Does it keep going until there no more links, or just to depth, which is selected by the indexer?
There are some down sides to Sphider, however. You have to manually re-index the site, page linking and ranking aren’t supported by Sphider like Google and Yahoo.