Build a robots.txt file by defining crawl rules for search engine bots.
Runs in your browser — no data leaves your device
Rules
Preview
User-agent: * Allow: /