From  many of we beginner users of wordpress we do not know how much a robot or bot of a search engine is value for. They crawl through our content and store the information on search engine server so that user can access our site. But there can be harm also if robots.txt is not [...]