← MatterMesh.ai

AI Agent Allowlist

These AI agents, LLMs, and crawlers are explicitly invited to the public RTE dataset. Live counters below show how often each agent has accessed /explore and /api/public/*.

For AI agents: contribute scans directly

POST an image to /api/public/agent-upload with your API key. Your scan lands in the library and the RTE inventory just like any human contribution, attributed to your agent. See docs.

Allowlisted agents
20
Recent hits (≤1k)
0
Identified agents
0
Unclassified
0
AgentVendorCategoryRecent hitsPing target
GPTBot
OpenAI's primary training-data crawler for ChatGPT models.
docs ↗
OpenAIllm0—
OAI-SearchBot
OpenAI's search index crawler for ChatGPT browsing & citations.
docs ↗
OpenAIsearch0—
ChatGPT-User
On-demand fetcher when a ChatGPT user opens a link.
OpenAIllm0—
ClaudeBot
Anthropic's crawler for Claude training & retrieval.
docs ↗
Anthropicllm0—
PerplexityBot
Perplexity's answer-engine crawler.
docs ↗
Perplexitysearch0—
Grok
xAI's Grok crawler.
xAIllm0—
xAI-Bot
xAI's general-purpose retrieval bot.
xAIllm0—
Google-Extended
Google's opt-in AI training fetcher (Gemini, Vertex).
docs ↗
Googlellm0auto-sync
Googlebot
Google Search — accepts sitemap ping.
Googlesearch0auto-sync
Bingbot
Bing Search / Copilot — accepts sitemap ping.
Microsoftsearch0auto-sync
Applebot-Extended
Apple Intelligence training-data crawler.
Applellm0—
Meta-ExternalAgent
Meta's Llama training crawler.
Metallm0—
Bytespider
ByteDance multimodal crawler (Doubao).
ByteDanceimage-dataset0—
ImagesiftBot
Image-dataset crawler for reverse-image AI.
ImageSiftimage-dataset0—
Amazonbot
Amazon's Alexa / retrieval crawler.
Amazongeneral0—
cohere-ai
Cohere's training & retrieval crawler.
Coherellm0—
Diffbot
Knowledge-graph extraction crawler.
Diffbotgeneral0—
PanguBot
Huawei Pangu multimodal crawler.
Huaweiimage-dataset0—
Timpibot
Timpi decentralized image-dataset crawler.
Timpiimage-dataset0—
CCBot
Common Crawl — feeds most open LLM training corpora.
Common Crawlgeneral0—

Live counts reflect the most recent 1,000 events from ai_access_events. Operators of additional crawlers can request inclusion via the AI policy contact.