User-agent: * Allow: / # Install / uninstall scripts are operational assets curl'd from the CLI # bootstrap flow — they're not pages and shouldn't appear in search # results (also avoids look-alike domains piggy-backing on the canonical # install URL via Google searches). Crawlers can still discover them # from the README / docs that link to them. # # The blanket User-agent: * block already covers GPTBot / ClaudeBot / # PerplexityBot / Google-Extended: every bot we want to allow is # allowed by default ("Allow: /"), and every bot is denied the four # script paths below. No need for per-bot sections — those would # actually defeat the Disallow because each User-agent block is # independent under the robots.txt spec (the AI-bot blocks here used # to re-Allow / without re-Disallow'ing the scripts, leaking them). Disallow: /install.sh Disallow: /install.ps1 Disallow: /uninstall.sh Disallow: /uninstall.ps1 Sitemap: https://agentkey.app/sitemap.xml # llms.txt is the emerging convention (llmstxt.org) for LLM-driven # crawlers — distinct from sitemap.xml (pages-for-Google), this is a # curated content brief for retrieval-augmented agents like ChatGPT # Search, Perplexity, Claude. robots.txt parsers ignore this comment; # we document it here so anyone editing the file knows it exists. # llms-txt: https://agentkey.app/llms.txt