Back to Insights

How 22 Dealer Website Platforms Handle AI Bot Access

AI Visibility
How 22 Dealer Website Platforms Handle AI Bot Access
November 2, 2026
•
Corey Lydstone

Your website platform may let one AI assistant through and turn another away.

That is what stood out when CarGenius compared 22 dealer website platforms. On sampled Dealer.com sites, test requests identifying themselves as ChatGPT-User, Claude-User and Perplexity-User were refused, while requests using the Google-Agent name were not. Sampled DealerOn sites refused none of those four requests.

For dealers, the question is straightforward: if an AI assistant comes looking for your inventory, service hours or dealership details, how does your website platform respond?

A telling contrast
Same four bot names. Very different results.

Observed responses to simulated user-requested AI agents across each platform's sampled sites.

Dealer.com
Three bot names blocked in our simulated tests
ChatGPT-UserBlocked
Claude-UserBlocked
Perplexity-UserBlocked
Google-AgentAllowed
DealerOn
No blocks observed for these four bot names
ChatGPT-UserAllowed
Claude-UserAllowed
Perplexity-UserAllowed
Google-AgentAllowed

The differences did not stop there. Some platforms behaved differently from one site to another, and published bot instructions did not always match the response to a request. The tables below separate actual test responses from written robots.txt rules and llms.txt availability.

Why bot access matters to a dealership

AI assistants can use information from several places, including search indexes, third-party websites and direct visits to a dealer's own pages. When an assistant retrieves your site, it may be looking for a vehicle's availability, service hours, location details or policies. If that request fails, the assistant loses one route to your current information. It may still find your dealership through other sources.

There is also more than one kind of bot. User-requested agents retrieve pages to answer a person's question. Separate crawlers, such as OAI-SearchBot, Claude-SearchBot, PerplexityBot and Googlebot, help their services discover content for search. Training bots serve another purpose, collecting material for possible use in AI model development. Dealerships should make their public inventory, service, and dealership information accessible to legitimate AI search crawlers and shopper-requested agents. Whether to allow AI companies to use that information for model training is a separate decision.

What we checked

1. Direct agent requests

Does a site serve its homepage to four user-requested agents and four search crawlers? This checks the response to the bot name, not access from a verified provider. A security system may correctly reject a request that claims to be an AI bot but does not originate from that provider's verified network. These results should therefore be treated as potential access issues worth investigating, not definitive evidence that the platforms block legitimate AI traffic.

2. Published robots.txt rules

What does the site's instruction file say about user-requested agents, search crawlers and training bots? These instructions are separate from actual security enforcement.

3. llms.txt availability

Does the site offer an optional text file intended to guide AI tools toward useful pages? The file's presence does not establish that an AI assistant uses it.

Which AI bots did dealership websites refuse?

We tested two types of bot requests: agents that retrieve a page for a user, and crawlers that help AI services index pages for search. Both tables below show whether the sampled websites refused a request carrying the bot's name.

Dealer.com showed a consistent pattern: every sampled site refused ChatGPT-User, Claude-User and Perplexity-User requests, but not Google-Agent. Dealer Inspire showed a different pattern, refusing ChatGPT-User and Google-Agent requests while accepting the other two. The Dealer eProcess and Porsche OEM samples refused all four user-requested agents.

These refusals came from a test connection, not from the AI companies' verified networks. They are signs to investigate with a website provider, not proof that the official services cannot access those sites.

User-requested bots

These agents visit a page in response to a user's request, such as looking up a dealership or its current inventory. Google-Agent belongs in this group; it is not the same as Googlebot.

Reading both direct-access tables:

Blocked means every sampled site refused the bot-named request while the ordinary browser test worked. Allowed means none of the sampled sites refused the bot-named request. Partial means some sites refused it and others did not. Refused means all scripted test requests were turned away, including the ordinary browser test, so we could not tell whether the bot name made a difference. These labels describe our simulated tests, not verified access for the real AI services.

Website platformChatGPT-UserClaude-UserPerplexity-UserGoogle-Agent
Dealer.comBlockedBlockedBlockedAllowed
Dealer InspireBlockedAllowedAllowedBlocked
DealerOnAllowedAllowedAllowedAllowed
Dealer eProcessBlockedBlockedBlockedBlocked
Team VelocityPartialPartialPartialPartial
MotiveAllowedAllowedAllowedAllowed
OverfuelAllowedAllowedAllowedAllowed
DealerFireAllowedAllowedAllowedAllowed
Dealer AlchemistAllowedAllowedBlockedBlocked
SokalRefusedRefusedRefusedRefused
Space AutoAllowedAllowedAllowedAllowed
Team MXS / Dealer MastersAllowedAllowedAllowedAllowed
InticeAllowedAllowedAllowedAllowed
JazelPartialPartialPartialPartial
NabThatAllowedAllowedAllowedBlocked
RemoraAllowedAllowedAllowedAllowed
Catalyst IQ / Fox DealerAllowedAllowedAllowedAllowed
Audi OEM Website PlatformAllowedAllowedAllowedAllowed
Porsche OEM Website PlatformBlockedBlockedBlockedBlocked
AutomotiveX / RunMyLeaseAllowedAllowedAllowedAllowed
Dealer Car SearchAllowedAllowedAllowedBlocked
CarsForSale / SiteFLEXAllowedAllowedAllowedAllowed

Search bots

These crawlers help search engines and AI search services find and index website pages. We checked OAI-SearchBot (OpenAI), Claude-SearchBot (Anthropic), PerplexityBot (Perplexity) and Googlebot (Google). Googlebot supports regular Google Search, including Google's AI Overviews and AI Mode. It is different from Google-Agent, which retrieves pages in response to a user's request.

Website platformOAI-SearchBotClaude-SearchBotPerplexityBotGooglebot
Dealer.comBlockedBlockedBlockedBlocked
Dealer InspireBlockedBlockedBlockedBlocked
DealerOnAllowedAllowedAllowedAllowed
Dealer eProcessBlockedBlockedBlockedBlocked
Team VelocityPartialPartialPartialPartial
MotiveAllowedAllowedAllowedAllowed
OverfuelAllowedAllowedAllowedAllowed
DealerFireAllowedAllowedAllowedAllowed
Dealer AlchemistAllowedAllowedBlockedBlocked
SokalRefusedRefusedRefusedRefused
Space AutoAllowedAllowedAllowedAllowed
Team MXS / Dealer MastersAllowedAllowedAllowedAllowed
InticeAllowedAllowedAllowedAllowed
JazelPartialPartialPartialPartial
NabThatAllowedAllowedAllowedAllowed
RemoraAllowedAllowedAllowedAllowed
Catalyst IQ / Fox DealerAllowedAllowedAllowedAllowed
Audi OEM Website PlatformAllowedAllowedAllowedAllowed
Porsche OEM Website PlatformBlockedBlockedBlockedBlocked
AutomotiveX / RunMyLeaseAllowedAllowedAllowedAllowed
Dealer Car SearchAllowedAllowedAllowedAllowed
CarsForSale / SiteFLEXAllowedAllowedAllowedAllowed

Note: a refusal from our independent test connection does not establish whether a crawler arriving from its operator's verified network would be refused. Googlebot is the search crawler; Google-Agent is the separate user-requested agent shown above. Sokal is marked Refused because its security system turned away all scripted requests from our test connection, including the ordinary browser-identifying request. This does not prove that verified AI company crawlers are blocked. See the methodology for sample sizes.

Platform spotlight | Sokal + DataDome

Why Sokal refused every test request

All three Sokal-managed sites in our sample refused every scripted request, including one identifying itself as an ordinary browser. That is why Sokal appears as Refused in both direct-access tables. It does not tell us whether verified ChatGPT, Google, or other AI services can access those sites.

There is a reason for the strict screening. In a March 2025 customer case study, Sokal describes adopting DataDome to combat fraudulent ad traffic, website scraping, and denial-of-service attacks that strained its resources. Sokal reported a sharp reduction in unwanted traffic and less manual security work.

DataDome's documentation says unauthenticated automated requests are blocked by default, while recognized bots and AI agents can be authorized. Even legitimate shoppers may be asked to complete a bot-verification challenge before they can browse inventory or contact a dealership. That extra step adds friction. The Sokal-branded screen shown here illustrates the potential experience, but our study did not measure how often it happens.

Sokal-branded DataDome verification screen reading Verification Required with a slider labeled Slide right to secure your access.
Even human visitors may see a bot challenge. This Sokal-branded verification step can interrupt a shopping visit.
What dealers should ask: Can verified AI search and shopper-requested agents access our public inventory and service pages? How often do real customers encounter verification challenges? Strong security and useful AI access should be managed together.

What the robots.txt files tell different AI bots

A robots.txt file publishes instructions about which parts of a website automated visitors should access. That is different from a security system that actually allows, blocks or challenges a request. A website can publish an allowance for an agent and still refuse it at the security layer.

In our sample, Dealer.com, DealerOn and Team Velocity commonly published path-specific restrictions for user-requested and search bots. These were limits on selected parts of a site, not blanket blocks. Many other sites had no rule specifically naming an agent. That does not guarantee access, because a general rule may still apply.

Google-Agent-specific rules were uncommon across the platforms. Team Velocity published path-specific restrictions, while the Audi OEM sample returned an error when the survey tried to retrieve robots.txt. Google says its user-triggered fetchers generally ignore robots.txt. These published rules are therefore not a definitive test of Google-Agent access.

All means the same rule status appeared at every sampled site. Partial means statuses varied by site. Red means path restrictions worth reviewing, yellow means mixed results, green means an explicit allowance and gray means no clear agent-specific instruction.

User-requested bots

AllowedExplicit permissionRestrictedSelected paths limitedNot namedNo specific rule foundNo fileNo robots.txt foundUnavailableServer error retrieving robots.txtPartialResults differed by site
Website platformChatGPT-UserClaude-UserPerplexity-UserGoogle-Agent
Dealer.comRestrictedAllRestrictedAllRestrictedAllNot namedAll
Dealer InspireNot namedAllNot namedAllNot namedAllNot namedAll
DealerOnRestrictedAllRestrictedAllRestrictedAllNot namedAll
Dealer eProcessNot namedAllNot namedAllNot namedAllNot namedAll
Team VelocityRestrictedAllRestrictedAllRestrictedAllRestrictedAll
MotiveRestrictedAllNot namedAllNot namedAllNot namedAll
OverfuelNot namedAllNot namedAllNot namedAllNot namedAll
DealerFirePartialAllowed / no filePartialNot named / no filePartialNot named / no filePartialNot named / no file
Dealer AlchemistPartialAllowed / not namedPartialAllowed / not namedPartialAllowed / not namedNot namedAll
SokalNot namedAllNot namedAllNot namedAllNot namedAll
Space AutoAllowedAllAllowedAllPartialAllowed / not namedNot namedAll
Team MXS / Dealer MastersNot namedAllNot namedAllNot namedAllNot namedAll
InticeNot namedAllNot namedAllNot namedAllNot namedAll
JazelNot namedAllNot namedAllNot namedAllNot namedAll
NabThatNot namedAllAllowedAllNot namedAllNot namedAll
RemoraPartialNot named / allowedNot namedAllNot namedAllNot namedAll
Catalyst IQ / Fox DealerNot namedAllNot namedAllNot namedAllNot namedAll
Audi OEM Website PlatformUnavailableAllUnavailableAllUnavailableAllUnavailableAll
Porsche OEM Website PlatformNot namedAllNot namedAllNot namedAllNot namedAll
AutomotiveX / RunMyLeaseNot namedAllNot namedAllNot namedAllNot namedAll
Dealer Car SearchAllowedAllAllowedAllAllowedAllNot namedAll
CarsForSale / SiteFLEXNot namedAllNot namedAllNot namedAllNot namedAll

Search bots

These are the published instructions for the same four search crawlers tested above. A rule restricting selected URLs does not mean the whole website is blocked, and a missing bot-specific rule is not proof of access.

Website platformOAI-SearchBotClaude-SearchBotPerplexityBotGooglebot
Dealer.comRestrictedAllRestrictedAllRestrictedAllNot namedAll
Dealer InspireNot namedAllNot namedAllNot namedAllNot namedAll
DealerOnRestrictedAllRestrictedAllRestrictedAllNot namedAll
Dealer eProcessNot namedAllNot namedAllNot namedAllNot namedAll
Team VelocityRestrictedAllRestrictedAllRestrictedAllRestrictedAll
MotiveRestrictedAllNot namedAllRestrictedAllNot namedAll
OverfuelNot namedAllNot namedAllNot namedAllNot namedAll
DealerFirePartialNot named / No filePartialNot named / No filePartialAllowed / No filePartialNot named / No file
Dealer AlchemistPartialAllowed / Not namedPartialAllowed / Not namedPartialAllowed / Not namedNot namedAll
SokalNot namedAllNot namedAllNot namedAllNot namedAll
Space AutoAllowedAllAllowedAllAllowedAllAllowedAll
Team MXS / Dealer MastersNot namedAllNot namedAllNot namedAllNot namedAll
InticeNot namedAllNot namedAllNot namedAllNot namedAll
JazelNot namedAllNot namedAllNot namedAllNot namedAll
NabThatAllowedAllNot namedAllAllowedAllAllowedAll
RemoraNot namedAllNot namedAllPartialNot named / AllowedNot namedAll
Catalyst IQ / Fox DealerAllowedAllNot namedAllAllowedAllNot namedAll
Audi OEM Website PlatformUnavailableAllUnavailableAllUnavailableAllUnavailableAll
Porsche OEM Website PlatformNot namedAllNot namedAllNot namedAllNot namedAll
AutomotiveX / RunMyLeaseNot namedAllNot namedAllNot namedAllNot namedAll
Dealer Car SearchAllowedAllAllowedAllAllowedAllAllowedAll
CarsForSale / SiteFLEXNot namedAllNot namedAllNot namedAllNot namedAll

The eight user-requested and search-crawler robots.txt columns use the October 28 full survey. “Restricted” means only selected paths were limited. “Not named” and “No file” do not prove permission, and “Unavailable” means the survey could not retrieve the file because of a server error. In the Audi sample, that distinction matters: a server error is not evidence that robots.txt does not exist. Google describes how robots.txt server errors can temporarily disrupt crawling.

Training bots

Some AI companies use separate crawlers to collect material that may help train their models. These crawlers serve a different purpose from an assistant retrieving your inventory or service hours for a shopper. Your dealership can choose to allow AI-powered discovery while limiting the use of its website content for model training.

We checked the published robots.txt rules for GPTBot (OpenAI), ClaudeBot (Anthropic), Google-Extended (Google) and Meta-ExternalAgent (Meta). Google-Extended is a policy setting, not a separate crawler that visits the site. It helps control some Google AI training and grounding uses without determining whether a site appears in Google Search.

Colors indicate the site's stated access rule, not whether the platform is good or bad for AI visibility. A dealership may intentionally block training crawlers while allowing search and user-requested agents. Blocked means a full-site disallow rule was found; Restricted means selected paths were limited. The labels describe the published instructions for each named bot, not confirmed access by the real service.

AllowedExplicit permissionRestrictedSelected paths limitedBlockedEntire site disallowedPartialSites differedNot namedNo agent-specific ruleUnavailableServer error retrieving robots.txt
Website platformGPTBotClaudeBotGoogle-ExtendedMeta-ExternalAgent
Dealer.comRestrictedAllNot namedAllNot namedAllNot namedAll
Dealer InspireNot namedAllNot namedAllNot namedAllNot namedAll
DealerOnNot namedAllNot namedAllRestrictedAllNot namedAll
Dealer eProcessNot namedAllNot namedAllNot namedAllNot namedAll
Team VelocityRestrictedAllRestrictedAllRestrictedAllRestrictedAll
MotiveNot namedAllNot namedAllNot namedAllNot namedAll
OverfuelNot namedAllAllowedAllNot namedAllNot namedAll
DealerFirePartialAllowed / No filePartialNot named / No filePartialAllowed / No filePartialNot named / No file
Dealer AlchemistPartialAllowed / Not namedPartialAllowed / Not namedPartialAllowed / Not namedPartialAllowed / Not named
SokalNot namedAllNot namedAllNot namedAllNot namedAll
Space AutoBlockedAllBlockedAllPartialBlocked / AllowedBlockedAll
Team MXS / Dealer MastersNot namedAllNot namedAllNot namedAllNot namedAll
InticeNot namedAllNot namedAllNot namedAllNot namedAll
JazelNot namedAllNot namedAllNot namedAllNot namedAll
NabThatAllowedAllAllowedAllAllowedAllAllowedAll
RemoraPartialNot named / AllowedNot namedAllPartialNot named / AllowedNot namedAll
Catalyst IQ / Fox DealerAllowedAllAllowedAllAllowedAllAllowedAll
Audi OEM Website PlatformUnavailableAllUnavailableAllUnavailableAllUnavailableAll
Porsche OEM Website PlatformNot namedAllNot namedAllNot namedAllNot namedAll
AutomotiveX / RunMyLeaseNot namedAllNot namedAllNot namedAllNot namedAll
Dealer Car SearchAllowedAllAllowedAllAllowedAllAllowedAll
CarsForSale / SiteFLEXNot namedAllNot namedAllNot namedAllNot namedAll

Training-bot rules were assessed from the published robots.txt files on October 28, 2026. The four bot-specific rule columns reflect the survey review of those files. “Unavailable” means the survey encountered a server error; “Not named” is neutral because general rules may still apply. No requests from verified training crawlers were tested. These rules describe instructions, not proof of actual crawler access.

Why the distinction matters:

Blocking a training crawler such as GPTBot does not necessarily block ChatGPT search. OpenAI offers separate rules for GPTBot and OAI-SearchBot. Similarly, Fox Dealer can allow the four training-related tokens shown in this table while restricting other crawlers such as CCBot and Bytespider, which are not included here. Access policies are specific to each bot. Review these choices with your website provider before changing them.

Which platforms publish llms.txt?

An llms.txt file is a proposed way to provide AI tools with a concise guide to a website and links to useful information. It is not a replacement for accessible pages, and there is no established evidence that simply adding the file makes a dealership appear more often in AI recommendations.

Of the 22 platforms, 15 consistently published llms.txt across their sampled websites. One showed mixed results, and six had no file in the sample. Dealer eProcess illustrates why the file alone is not enough: the platform published llms.txt throughout its sample, yet those sites refused our simulated requests from all four user-requested agents. A text file cannot override the site's access controls.

All means every sampled site had an llms.txt file, Partial means some did, and None means none did. The table measures presence, not the quality of the information inside.

Website platformllms.txt present
Dealer.comAll
Dealer InspireAll
DealerOnAll
Dealer eProcessAll
Team VelocityAll
MotiveAll
OverfuelAll
DealerFireNone
Dealer AlchemistAll
SokalAll
Space AutoAll
Team MXS / Dealer MastersNone
InticeAll
JazelPartial
NabThatAll
RemoraAll
Catalyst IQ / Fox DealerAll
Audi OEM Website PlatformNone
Porsche OEM Website PlatformNone
AutomotiveX / RunMyLeaseNone
Dealer Car SearchAll
CarsForSale / SiteFLEXNone

For context, see the llms.txt proposal. File availability was checked in the October 28, 2026 full survey.

What happens when ChatGPT actually tries to access dealership inventory?

To understand how these technical limitations affect real-world AI shopping experiences, we ran a separate test using ChatGPT to explore inventory at Capital Ford Charlotte on October 23, 2026.

We asked ChatGPT to retrieve used vehicle inventory and navigate beyond the first page of results.

What we found was concerning.

  • Page 1: ChatGPT retrieved inventory information showing 2,533 vehicles from a relatively recent crawl.
  • Page 2: ChatGPT returned information from a version of the page crawled approximately two weeks earlier, when the site showed 2,503 vehicles.
  • Filtered inventory: When we requested used Acura vehicles using a filtered inventory URL, the retrieved content showed an unfiltered inventory listing instead, with no Acura matches.

In other words, ChatGPT could appear to be successfully accessing dealership inventory while actually working with older or incorrectly filtered information.

This illustrates an important distinction: An AI assistant finding your website doesn't necessarily mean it has access to your current inventory.

For shoppers asking about specific vehicles, pricing, or availability, that distinction matters. Inventory changes daily, and even a few days of delay can make information unreliable.

These observations don't establish that the dealership's website platform blocked ChatGPT from retrieving live pages. AI retrieval systems can rely on previously crawled content for several reasons, including retrieval limitations and website configuration.

But they highlight a broader challenge for automotive retail: making information discoverable is only part of the problem. AI agents also need reliable access to accurate, up-to-date inventory data.

As AI moves beyond recommending dealerships toward helping shoppers find and purchase specific vehicles, direct access to structured, current dealership data will become increasingly important.

Your website's settings matter more than the platform name

The survey found differences within some platforms. The Team Velocity sample included both sites that refused simulated agent requests and sites that accepted them. Jazel showed a similar split. That pattern is a reason to check your dealership's actual configuration, even if another store on the same platform appears accessible.

These checks also answer a narrower question than whether ChatGPT or another AI service will recommend your store. A website can be reachable and still have outdated inventory, thin service information or inconsistent dealership details. AI can also name a dealership using third-party information without retrieving its website directly. Technical access is one part of making your dealership's information available, not an AI visibility score.

For most dealers, the website provider controls hosting, security rules and bot settings. Your marketing team can identify the concern, but the provider is usually best placed to confirm whether legitimate AI traffic is actually getting through.

Four questions to ask your website provider

If your dealership appears in one of these platform groups, use the results to start a specific conversation with the team that manages your website.

Can verified AI assistants access our public pages?

Ask the provider to check the homepage, inventory search pages, individual vehicle details, service pages and dealership information, not only whether the homepage loads.

Are security tools challenging legitimate AI requests?

Ask for a review of recent security logs for ChatGPT-User, Claude-User, Perplexity-User and Google-Agent. The provider should distinguish verified traffic from a request that merely claims to be one of those agents.

Do our robots.txt rules match what we want AI systems to access?

Review user-directed agents, search crawlers and training crawlers separately. You may want AI services to find your inventory without allowing your pages to be used for model training.

Is our llms.txt file useful and current?

If your site publishes one, make sure it points to real inventory, service and dealership information. If you have an MCP server, consider documenting how AI agents can discover and connect to it.

If your provider uses Cloudflare:

CarGenius has a separate guide to its Search, Agent, Training and Bot Preference Sync controls. Read how to keep Cloudflare from blocking Google and AI crawlers. Check with your provider before changing these settings.

The first useful step is to confirm whether the AI services you want to reach can retrieve your dealership's public information from their verified networks.

See what AI says about your dealership

CarGenius Dealer AI Visibility Reports measure which dealerships AI assistants actually name when shoppers ask non-branded Find, Buy, and Service questions, which competitors win those recommendations, and which sources are shaping the answers. See where your store stands.

View the Visibility ReportsGet a Custom ReportOr book a demo →

About the data

Data used and how we calculated the results

CarGenius surveyed 64 dealer websites across 22 website platforms. The published comparisons come from a complete round of testing of the sampled websites on October 28, 2026, including all eight tested user-requested and search bots, their robots.txt rules, and llms.txt availability. Earlier October 23 testing and manual browser follow-up informed the study. The training-related robots.txt rules were also assessed on October 28. The sample used three sites per platform except AutomotiveX / RunMyLease, which had one. Fifty-two sites came from a selected dealer roster and 12 were additional platform-confirmed examples. At the site level, 15 of 61 evaluable websites refused the simulated ChatGPT-User request, 18 of 61 refused the simulated Googlebot request, and 46 of 64 published an llms.txt file. The sample was selected to compare observed behavior, not to estimate how common a setting is across every site on a platform.

  • Direct requests: A normal browser-identifying request and requests carrying four user-requested agent names and four search-crawler names were sent through a comparable test path. A refusal was recorded when the agent-named request received a denial or challenge while the normal request worked. The requests were not sent from the AI companies' verified networks, so verified bots may be handled differently.
  • robots.txt: Published rules were checked for four user-requested agents and four search crawlers, along with a review of four training-related bot tokens on October 28. “Restricted” means selected paths were limited; “Blocked” means an entire-site disallow was found for a training token. “Not named” means no agent-specific instruction was found; general wildcard rules could still apply. “Unavailable” means robots.txt could not be retrieved in the survey because of a server error, not that the file was absent. Google-Extended is a policy control token, not a distinct crawler. Published instructions do not show whether a security system actually admits a bot.
  • llms.txt: The check records whether a file was available, not the quality of its content or whether AI systems use it.
  • Table labels: In the direct-request tables, “Blocked” means every sampled site refused the bot-named request while the ordinary browser test worked; “Allowed” means no sampled site refused it; “Partial” means results differed; and “Refused” means even the ordinary browser test was rejected, so no bot-specific conclusion is possible. In the llms.txt table, “All” means every sampled site had the file, “Partial” means some did, and “None” means none did. In the robots.txt tables, “All” means the same rule status appeared at every sampled site and “Partial” means results differed. An “Unavailable” result refers to a failed robots.txt request during the survey. All three Sokal sites refused every scripted client in the direct-access test, including the normal-browser control request. They are marked “Refused” to describe that observed response, not to claim that the platform blocks verified AI crawlers.

The survey measured observed technical responses. It did not measure verified crawler access, AI recommendations, dealer search rankings or any change in leads. Robots.txt could not be retrieved for the three Audi OEM sample sites on October 28, so their rule results, including the training-bot rules, are shown as Unavailable. Website settings can change after the testing dates.

Documentation: OpenAI crawlers · Anthropic crawlers · Perplexity crawlers · Google user-triggered fetchers · Google-Extended · Meta crawlers · llms.txt proposal.

More from AI Strategy

CarGenius AI CEO Corey Lydstone Joins The AutoGuide Show to Discuss AI Car Shopping
Company News
October 28, 2026
•
CarGenius AI

CarGenius AI CEO Corey Lydstone Joins The AutoGuide Show to Discuss AI Car Shopping

CarGenius AI CEO Corey Lydstone joins AutoGuide host Colum Wood to discuss AI car-shopping recommendations, agentic commerce, and the future of automotive retail.

How Google AI Overviews, AI Mode and Gemini Differ
Industry Analysis
October 26, 2026
•
Corey Lydstone

How Google AI Overviews, AI Mode and Gemini Differ

Google AI Overviews, AI Mode and Gemini are related but not the same. A plain-English guide to how they differ, how large each has become, and why it matters for car dealers

75%

of car shoppers now start with AI

3.2x

higher conversion with AI search

24/7

instant answers for every shopper

our latest updates

Stay ahead of the curve

Get weekly insights on AI, automotive commerce, and dealer tech delivered to your inbox.

No spam. Unsubscribe anytime.
Thank you!
Something went wrong. Please try again.