Read a site's robots.txt and the owner's stated goals for each AI crawler (OpenAI's OAI-SearchBot and GPTBot, Google-Extended, and others
No bots match
