This article was taken from the PHP Architect May2005 Control Search Engine Indexing Do you have a robots.txt in your webroot? All web sites should have a valid robots.txt file even if every page on the site is meant to … Continue reading →