Tag: robots.txt

We had already discussed about Robots.txt in websites and their importance. Now, i am going to share some online websites that can help to create robots.txt for the websites.

Robots.txt file found in root directory of any website. As its name indicated Robots, It is being made for robots, crawler or web robots. It gives instructions to