What Does ChatGPT Read on an Agency Website? A 12-Week AI Crawler Report
19,039 crawler requests, 1,366 ChatGPT answer fetches, 53 referral visitors, and 3 visitors who inquired: DataFast data from withmeridian.org shows how AI crawling, visits, and conversions relate.
Updated

Key findings
Over approximately 12 weeks, from August 24 to November 17, 2026, AI and search engine crawlers made 19,039 requests to withmeridian.org. DataFast classified 1,402 requests, or 7.4%, as answer fetches.
- ChatGPT accounted for 97.4% of answer fetches. ChatGPT-User made 1,366 fetches; all other answer-fetch crawlers combined made 36.
- 44.2% of all answer fetches went to the homepage: 620 requests. The most frequently read content explained who the company is.
- Approximately 26 fetches corresponded to 1 referral visitor. The 1,366 ChatGPT answer fetches corresponded to 53 visitors arriving from ChatGPT. This is a ratio of aggregate totals, not a click-through rate for individual requests.
- Of those 53 visitors, 3 submitted an inquiry: a 5.7% conversion rate. Google brought 55 visitors during the same period, with 0 visitors submitting inquiries.
- More crawling does not necessarily bring more visits. PerplexityBot fetched 736 distinct URLs, the widest coverage, but this report recorded no answer fetches or referral visitors from it.
- llms.txt received just 1 request, from a Meta crawler. During the same period, robots.txt received 1,608 requests.
Where the data comes from
This article uses a summary of first-party DataFast account data supplied by the withmeridian.org team, retrieved on November 17, 2026.
- Crawler data: DataFast Bot traffic, covering August 24 to November 17, 2026, approximately 12 weeks. The supplied summary marks all 19,039 requests as IP-verified, rather than identified by User-Agent alone.
- Human visitors: DataFast website analytics, recorded from August 28 through November 17, with 779 visitors in total. ChatGPT referral visitors were filtered using
utm_source=chatgpt.com. - Inquiries: Unique visitors who triggered the website form's
inquiry_submitevent. This article counts visitors who inquired, which is not equivalent to sales or revenue.
Crawler purposes follow DataFast's classification. These categories describe request types; they do not directly prove that a page was indexed, used for training, or included in a final answer.
| Purpose | Meaning in this report | Crawlers observed on this website |
|---|---|---|
| Answer fetch | An AI product retrieves a page for a user interaction | ChatGPT-User, Claude-User, DuckAssistBot, GoogleAgent, Google-Read-Aloud |
| Indexing | Discovering and updating content for search or AI search | OAI-SearchBot, Bingbot, Googlebot, PerplexityBot, Baiduspider |
| Training and other collection | DataFast's training category in this summary | GPTBot, ClaudeBot, GoogleOther, meta-externalagent, Amazonbot, Applebot, Bytespider |
Pages were grouped by path: homepage, About, case studies (/case-studies), services (/services), blog (/blog), contact, machine-readable files (robots.txt, sitemap, llms.txt), legal and payment pages, tools, and other pages. Pages under /zh/ followed the same grouping rules and were also marked as Chinese-language pages.
This dataset has five limitations:
- It covers one B2B services website, with small visitor and inquiry samples. The findings are directional and cannot be generalized as average results for all SaaS or GEO websites.
- A fetch is not a citation. We can see which page a crawler read, but not the user's question or whether the final answer cited that page.
- Fetches and visitors cannot be matched request by request. Recording began on different dates; caching, existing indexes, shared links, and repeat visits can all affect the ratio. The “visitors / fetches” metric in this article is not a click-through rate.
- UTM parameters define attribution here. ChatGPT visits without that parameter may not have been identified. This summary also cannot distinguish traditional Google or Bing search visits from visits through their AI features.
- Crawler categories depend on the analytics tool. GoogleOther, for example, is Google's general-purpose crawler; each visit cannot be assumed to be model training. The number of URLs crawled is also not the number of URLs indexed.
Finding 1: Who is crawling the website?
The records contain 18 crawler types over approximately 12 weeks. Under DataFast's classification for this summary, about 48% of requests fell into the training category; answer fetches accounted for only 7.4%.
OpenAI's indexing crawler, OAI-SearchBot, made the most requests: 3,653. Meta, GoogleOther, and Amazonbot also made substantial numbers of requests. But those background collection requests do not themselves prove that a recommendation occurred or a visitor arrived at that time.
The first step, therefore, is to separate indexing, other background collection, and page retrieval associated with user interactions, rather than adding all AI requests together.
Finding 2: When ChatGPT fetches pages, it most often reads who you are
ChatGPT-User read 158 distinct pages. The report records 620 homepage fetches, making it the most concentrated single entry point: 44.2% of all 1,402 answer fetches, or 45.4% of the 1,366 ChatGPT fetches.
The most frequently fetched individual pages included:
| Page | Fetches recorded in the report |
|---|---|
| Homepage | 620 |
/about | 48 |
/case-studies/volumn | 31 |
/case-studies | 28 |
/case-studies/yes-rentals | 27 |
The homepage, About page, and case studies all help establish who a company is and what it has done. The Volumn case study and Yes Rentals case study provide more specific project context than a service promise alone.
Frequently read blog posts included SaaS SEO pricing, with 17 fetches, and choosing a SaaS SEO agency, with 10. Both address pricing and provider comparisons.
Chinese-language pages under /zh/ received 114 fetches in total, 8.1% of all answer fetches. The most frequently read was /zh/blog/how-to-find-geo-competitor-sites, with 12. The denominator for this percentage is all answer fetches, not ChatGPT alone.
Finding 3: The pages AI reads most are not the pages that bring the most visits
Using this report's aggregate figures, approximately 26 ChatGPT page fetches corresponded to 1 referral visitor. But results varied greatly by page: the homepage was fetched 620 times and brought 8 visitors; a free tool was fetched 11 times and brought 25.
| Page | ChatGPT answer fetches | ChatGPT referral visitors | Visitors / fetches |
|---|---|---|---|
/twitter-monitor (free tool) | 11 | 25 | 227% |
/blog/monid-paper-backlinks-domain-rating-case-study | 7 | 4 | 57% |
/blog/best-link-building-services | 6 | 4 | 67% |
/about | 48 | 1 | 2.1% |
| Homepage | 620 | 8 | 1.3% |
/case-studies/volumn | 31 | 0 | 0% |
/case-studies/yes-rentals | 27 | 0 | 0% |
| Entire website | 1,366 | 53 | 3.9% |
227% does not mean a click-through rate above 100%; it is a ratio of two independent aggregate totals. AI can provide a link using existing content or a cache without fetching the page again for every visitor. This table only helps identify pages that appear more likely to receive referral visits.
One possible explanation is that pages serve different tasks:
- Verification. When a question concerns a company's background, capabilities, or case studies, the homepage, About page, and case studies can supply facts. The user may receive an answer within the conversation. These pages can influence how AI describes a brand without directly generating visits.
- Recommendation. When someone needs a usable tool or specific original material, the link itself is part of the answer. Tools and detailed case studies may give people a stronger reason to click.
This is an interpretation of the page distribution; the logs cannot verify users' actual questions. For GEO measurement, it is more useful to connect “fetches → referral visits → inquiries” while measuring each separately.
Finding 4: ChatGPT brought few visitors, but more inquiries
ChatGPT brought almost as many visitors as Google, with 3 visitors submitting inquiries versus 0.
| Source | Visitors | Visitors submitting inquiries | Conversion rate |
|---|---|---|---|
| ChatGPT | 53 | 3 | 5.7% |
| 55 | 0 | 0% | |
| Bing | 35 | 0 | 0% |
| Entire website | 779 | 8 | 1.0% |
In this sample, ChatGPT's inquiry conversion rate was above the website average. However, the sample includes only 3 visitors who inquired and cannot establish long-term channel returns.
Of the website's 8 visitors who submitted inquiries, 2 had made 68 and 19 visits respectively. These require investigation to determine whether they were internal or test traffic. Visit counts alone cannot establish identity, so the table retains the original counts rather than deducting them.
The paths taken by the 3 ChatGPT visitors who submitted inquiries were as follows. Locations refer to the visit locations recorded by the analytics tool:
- US visit:
/zh/services/reddit-services→/zh/case-studies→/zh/contact→ submission. - Hong Kong visit:
/zh/services/kol-services→/zh/case-studies→ homepage →/zh/contact, with a visit to/zh/aboutalong the way. - US visit:
/services/seo-services/programmatic-seo→/contact→ submission.
Chinese-language pages accounted for 8.1% of all answer fetches but received 2 of the 3 ChatGPT visitors who submitted inquiries. The first two paths also passed through case studies. This supports continuing to maintain Chinese-language service and case study content. However, visit location and page language cannot establish users' nationality, question language, identity, or budget.
Finding 5: AI referrals were clearest for OpenAI in this sample
Several companies crawled the website. OpenAI had the clearest attributable AI referral records in this report, with background indexing, answer fetches, and ChatGPT referral visitors all recorded. Google and Bing sources were not separated into traditional search and AI features, so their complete AI referral paths cannot be compared here. These are parallel observations at the channel level, not a traceable chain of individual requests.
| Company | Indexing and other background requests | URLs fetched by indexing crawlers | Answer fetches | Referral visitors |
|---|---|---|---|---|
| OpenAI | 3,820 (OAI-SearchBot 3,653; GPTBot 167) | 341 | 1,366 | 53 |
| 3,952 (GoogleOther 2,955; Googlebot 973; other 24) | 214 | 7 | 55, source recorded as Google | |
| Microsoft | 2,792 (Bingbot) | 496 | 0 | 35, source recorded as Bing |
| Perplexity | 987 (PerplexityBot) | 736 | 0 | 0 |
| Anthropic | 1,032 (ClaudeBot) | Not provided | 14 | 0 |
| Meta | 3,408 (meta-externalagent) | Not provided | 0 | 0 |
| Amazon | 1,557 (Amazonbot) | Not provided | 0 | 0 |
| Baidu / ByteDance | 3 (Baiduspider) / 1 (Bytespider) | Summary records 1 | 0 | Doubao 1 |
OpenAI's background requests plus answer fetches in this table total 5,186; Google's total 3,959. Other crawlers also appear outside these platforms, so the table should not be treated as a complete breakdown of all 19,039 requests.
Three observations merit continued monitoring:
- Perplexity had the widest crawl coverage, but no recorded referral visits. Fetching 736 URLs does not mean all were indexed, much less cited. An absence of answer-fetch records also does not prove that the product did not use an existing index or cache.
- Googlebot made fewer requests than Bingbot. Its 973 requests were about one-third of Bingbot's 2,792. Crawl volume cannot directly explain search traffic, however; ChatGPT and Google were already close in visitor numbers.
- Identifiable Baidu and ByteDance crawler requests were rare. Logs from this single website cannot reveal all the content sources of Chinese AI platforms. They also cannot establish that those platforms do not read official websites or rely only on a few external platforms.
Finding 6: Fetches increased, but visits did not grow with them
ChatGPT's weekly answer fetches rose from 64 in the week of August 24 to 290 in the week of November 9. Weekly referral visitors stayed between single digits and 13.
The observable increase was in page retrieval, which is not equivalent to an increase in brand mentions or recommendations. Together with finding 3, this suggests a direction worth testing: alongside improving factual brand pages, add tools and original resources that target customers will actually use, giving them a reason to visit the website.
The 290 fetches in the week of November 9 were a single-week peak. More weeks of data are needed to determine whether that becomes a new level. Adding the weekly visitor counts gives 55, which is 2 higher than the 53 deduplicated visitors for the full period; one visitor can be counted in two different weeks.
8 takeaways for SaaS and teams expanding internationally
- Treat the homepage as a brand fact sheet. The homepage accounted for 44.2% of all answer fetches. Founding dates, team details, locations, target customers, scope of work, and supported result metrics should be stated as facts readers can verify.
- Make case studies provide verifiable evidence. The report records 192 case study page fetches, and the first two ChatGPT inquiry paths passed through case studies. Each case study should specify dates, numbers, a baseline, and a measurement method wherever possible.
- Give target customers a reason to visit through useful tools.
/twitter-monitorreceived more referral visitors than the homepage, but the performance of a single tool still needs more time to assess. This ratio cannot be treated as a general rule. - Maintain complete Chinese-language pages. Chinese-language service pages received 2 of the 3 ChatGPT visitors who submitted inquiries. Keeping services, case studies, and the contact journey usable in Chinese is a practical action supported by this sample.
- Check each platform separately. Record indexing fetches, answer fetches, referral visits, and inquiries. Avoid replacing platform-specific results with one total for “AI exposure.”
- Get robots.txt and the sitemap right first. robots.txt received 1,608 requests, sitemap.xml received 548, and llms.txt received 1. These are observations from this website, not evidence that a particular file can guarantee citations.
- Measure fetches, visits, and inquiries separately. Approximately 26 ChatGPT fetches corresponded to 1 referral visitor here, and the visitor inquiry rate was 5.7%. The first metric is an aggregate ratio; the second is a conversion rate under this report's attribution method.
- Exclude internal and test traffic. Verify the identity of frequent visitors and test submissions before aligning channel denominators and goal events. Avoid adjusting data based directly on guesses.
Case study card: Use dates, numbers, and a baseline
When AI or a prospective customer checks a service provider, useful verifiable details usually answer: when, from what level to what level, and measured how? This website's observations can be organized into the following case study card.
| Item | Details |
|---|---|
| Period | Crawlers: 2026-08-24 to 2026-11-17; website visitors: 2026-08-28 to 2026-11-17 |
| Baseline | 64 ChatGPT answer fetches in the week of August 24; 0 ChatGPT referral visitors in the first two weeks recorded from August 28 |
| Results | 290 ChatGPT answer fetches in the week of November 9, approximately 4.5 times the first week; 158 distinct pages fetched over the full period, with 53 referral visitors, of whom 3 submitted an inquiry |
| Measurement | DataFast Bot traffic, combined with utm_source=chatgpt.com and the inquiry_submit event |
| Interpretation limits | These are observational records. The report lacks a complete timeline of page launches and promotional activity, so changes cannot be attributed to any single GEO action |
For another client case study, see Yes Rentals. Tools, question sets, and reporting windows differ between projects, so check measurement definitions before comparing them.
Case study writing template
The fields below form a template for readers to fill in, rather than an unfinished client result. Put the largest result number in the title, and pair claims in the body with dates and a baseline.
Title: Client name | Over a stated period, a metric changed from its baseline to its result
Industry / stage: Product and business stage
Period: YYYY-MM-DD to YYYY-MM-DD
Baseline: Start date, metric, value
Result: End date, metric measured consistently, value, and multiple of change
Measurement: Tool, platform, number of questions, region, language
What was done, by date:
YYYY-MM-DD: Action → change subsequently observed
What did not work:
Action, observed result, and possible reasons
Client quote:
A statement approved for publication, with an approved name and job title
Frequently asked questions
What is the difference between ChatGPT-User and OAI-SearchBot?
OAI-SearchBot discovers content for ChatGPT search. ChatGPT-User visits pages in response to user actions in ChatGPT or custom GPTs. The latter is closer to retrieval during user interaction, but logs alone cannot establish the question or answer citations. This website recorded 3,653 and 1,366 requests respectively.
Does a fetch from GPTBot or ClaudeBot mean I will be recommended?
That cannot be determined from the fetch. GPTBot and ClaudeBot primarily concern model development or training, with different roles from search discovery and user-triggered retrieval. DataFast's training category accounted for approximately 48% here, but that tool classification does not prove how every request was ultimately used.
Should I block AI training crawlers?
You can configure training, search, and user-access crawlers separately according to your website's content policy. OpenAI provides separate controls for GPTBot and OAI-SearchBot; Anthropic also distinguishes ClaudeBot, Claude-SearchBot, and Claude-User. Check the official guidance before making changes to avoid blocking search discovery you want to retain. Whether user-triggered requests follow robots.txt also depends on each product's documentation.
How to reproduce this analysis on your website
After enabling Bot traffic in DataFast, use your website API key to call the relevant endpoints. Keep the key in a server-side environment variable, rather than frontend code or a public article.
The crawler queries below use the same August 24 to November 17, 2026 window as this report. Website visit queries begin on August 28, when recording actually started. The crawler API includes all accepted requests by default; the examples use verification=ip_verified to maintain the IP-verified scope. Page results require pagination. The pages endpoint returns pages visited, which are not automatically landing pages. For goal queries, completions counts events and visitors counts unique visitors; comparisons require consistent definitions.
# Crawler totals, individual crawlers, and weekly trends
curl 'https://datafa.st/api/v1/analytics/bot-traffic?startAt=2026-08-24&endAt=2026-11-17&interval=week&verification=ip_verified' \
-H "Authorization: Bearer $DATAFAST_KEY"
# Pages read by ChatGPT answer fetches; paginate with offset
curl 'https://datafa.st/api/v1/analytics/bot-traffic/pages?category=answer_fetch&verification=ip_verified&crawlerId=OpenAI%3A%3AChatGPT-User%3A%3Aanswer_fetch&limit=100&offset=0&startAt=2026-08-24&endAt=2026-11-17' \
-H "Authorization: Bearer $DATAFAST_KEY"
# Pages visited and goals completed by ChatGPT referral visitors
curl 'https://datafa.st/api/v1/analytics/pages?filter_utm_source=is:chatgpt.com&startAt=2026-08-28&endAt=2026-11-17' \
-H "Authorization: Bearer $DATAFAST_KEY"
curl 'https://datafa.st/api/v1/analytics/goals?filter_utm_source=is:chatgpt.com&startAt=2026-08-28&endAt=2026-11-17' \
-H "Authorization: Bearer $DATAFAST_KEY"
To turn these observations into page, content, and conversion-tracking tasks, see Meridian's GEO services, or contact us with your website and current data.
Sources
- A summary of DataFast account data supplied by the withmeridian.org team, retrieved on 2026-11-17. The underlying account data is not public; this article's charts were produced from that summary.
- DataFast Bot traffic API and page-level parameters: crawler purposes, filters, and pagination.
- DataFast verification definitions, page analytics, and goal analytics: request verification, visits, and event counts.
- OpenAI's official crawler documentation: the roles of OAI-SearchBot, ChatGPT-User, and GPTBot.
- Anthropic's official crawler documentation: Claude crawler purposes and controls.
- GoogleOther's official documentation: the purpose of the general-purpose crawler.







