Robots.txt file found in root directory of any website. As its name indicated Robots, It is being made for robots, crawler or web robots. It gives instructions to