Robots.txt can be a file that exists on the basis directory of every website and can be employed to instruct serps on which directories/information of the web site they could crawl and incorporate of their index. This is certainly what engines like google use to find out the place to https://news.rafeeg.ae