User-agent: GPTBot Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: CCBot # Common Crawl bot, also used for training data Disallow: / User-agent: Google-Extended # Google's AI training bot Disallow: / User-agent: anthropic-ai # Anthropic AI bot Disallow: /