Google Answers the Question – Why Robot.txt Blocked URLs are Showing up in Search Results
The latest Google Webmaster Central video has been released. This time it deals with why robot.txt blocked Urls still sometimes show up in the search results. Matt Cutts from Google says that they get lots of complaints from webmasters on this issue. Webmasters use the robot.txt to keep the search engine crawlers from crawling [...]