https://crawlercontrol.com weekly 1 https://crawlercontrol.com/checker weekly 0.9 https://crawlercontrol.com/aprende monthly 0.8 https://crawlercontrol.com/bots weekly 0.8 https://crawlercontrol.com/bots/oai-searchbot 2026-07-11 monthly 0.6 https://crawlercontrol.com/bots/chatgpt-user 2026-07-11 monthly 0.6 https://crawlercontrol.com/bots/oai-adsbot 2026-07-11 monthly 0.6 https://crawlercontrol.com/bots/claude-searchbot 2026-07-11 monthly 0.6 https://crawlercontrol.com/bots/claude-user 2026-07-11 monthly 0.6 https://crawlercontrol.com/bots/perplexitybot 2026-07-11 monthly 0.6 https://crawlercontrol.com/bots/perplexity-user 2026-07-11 monthly 0.6 https://crawlercontrol.com/bots/applebot 2026-07-11 monthly 0.6 https://crawlercontrol.com/bots/applebot-extended 2026-07-11 monthly 0.6 https://crawlercontrol.com/bots/googlebot 2026-07-11 monthly 0.6 https://crawlercontrol.com/bots/google-extended 2026-07-11 monthly 0.6 https://crawlercontrol.com/bots/googleother 2026-07-11 monthly 0.6 https://crawlercontrol.com/bots/google-cloudvertexbot 2026-07-11 monthly 0.6 https://crawlercontrol.com/bots/gptbot 2026-07-11 monthly 0.6 https://crawlercontrol.com/bots/claudebot 2026-07-11 monthly 0.6 https://crawlercontrol.com/bots/ccbot 2026-07-11 monthly 0.6 https://crawlercontrol.com/bots/meta-externalagent 2026-07-11 monthly 0.6 https://crawlercontrol.com/bots/meta-webindexer 2026-07-11 monthly 0.6 https://crawlercontrol.com/bots/bytespider 2026-07-11 monthly 0.6 https://crawlercontrol.com/bots/claude-web 2026-07-11 monthly 0.6 https://crawlercontrol.com/bots/anthropic-ai 2026-07-11 monthly 0.6