You master it in five stages over about 24 weeks, and Rank Dough built this roadmap from 17 sources we opened and checked on 3 November 2026. It starts with how an agent reads a page, because the best GPT-4 agent in the WebArena benchmark finished only 14.41% of web tasks while humans finished 78.24%.

What is the agentic web?

The agentic web is the web used by an AI agent on a person’s behalf. You tell the agent “find me a studio near the university under a set budget and send an enquiry”, and the agent does the browsing, the comparing and the form.

That changes what success means for a website. A person reads the design and the copy. An agent reads the code: labels, buttons, structured data and the form itself.

SEO has always been about getting found. On the agentic web, getting found isn’t enough. The agent also has to finish the job, and I see that as the part most sites haven’t tested yet.

How is the roadmap laid out?

The roadmap runs in five stages, and each one needs the one before it. It assumes 3 to 5 hours a week and an SEO who already knows the basics, so it skips beginner material on purpose.

Source: Rank Dough agentic web roadmap, 3 November 2026.

We built it in a fixed order: Rank Dough’s own notes and tests first, then the live web for primary sources, then every source opened before it went in. Anything we couldn’t open and date was cut.

Where should you start? The skip test

Answer five questions without looking anything up. Every one you get right lets you skip ahead, so you don’t spend weeks on what you already know.

The starting point is Rank Dough’s judgement, not a scored test. Without JavaScript, the rule is:

1. Four or five answered: start at Stage 3.
2. Two or three answered: start at Stage 2.
3. One or none: start at Stage 1.

Stage 1: how does an agent read, decide and act?

You’ll be able to: explain to a client, from the agent’s side, why their booking flow fails an agent when a person gets through it. Time box: weeks 1 to 4.

World class techniques

1. Read the page as the agent does. Agents work from a mix of the page code, the accessibility tree and screenshots. OpenAI says its Atlas browser uses ARIA roles and labels to read page structure. It fails where the agent works from screenshots only. Established.
2. Map the task, not the page. Score a site on whole tasks (find, compare, enquire, pay), the way WebArena and WebMall do, not page by page. It fails on sites with one simple action. Established.
3. Separate the three traffic types. Training crawlers, search crawlers and agents acting for a user behave differently and need different rules. Established.

Sources: arXiv papers above; W3C WAI; OpenAI Publishers and Developers FAQ, checked 3 November 2026.

Unconventional approach: learn from accessibility first. Run a screen reader through a client’s enquiry form before any agent. Where the screen reader stalls, the agent usually stalls too. That’s a hypothesis, and the drill tests it.

Drill: pick three tasks on a live site you work on: find an item under a budget, compare two items, start an enquiry. Run each with an agent and with a screen reader, and log where each one fails.

Proof of mastery: a one page “why the agent failed” teardown per task, each failure tied to a cause: a missing label, a CAPTCHA, a choice lost from the web address, or a control that only works in JavaScript.

Stage 2: how do you see agents on a live site?

You’ll be able to: show a client how many agent visits they get, which tasks agents attempt and how many finish. Time box: weeks 5 to 9.

World class techniques

1. Verify, don’t guess from user agents. Signed agents sign each request with HTTP Message Signatures, and the site finds the key through a Signature-Agent header. It fails for agents driving the user’s own browser, which look human. Established.
2. Split the logs by purpose. Tag every AI hit as training, search, user action or undeclared, using Cloudflare’s four groups. Cloudflare measured crawl to refer ratios of about 887 to 1 for OpenAI and 50,000 to 1 for Anthropic in early August 2025. Established.
3. Measure task success, not visits. An agent visit that ends at a CAPTCHA is a lost lead. Build an agent funnel: landed, found the item, reached the form, submitted. Hypothesis, because there’s no public benchmark for live sites yet.

Sources: IETF datatracker; Cloudflare blog, checked 3 November 2026. The IETF draft has since been replaced by a newer version.

Unconventional approach: build the tool. A small script that reads raw logs, labels each AI hit by purpose and signature, and pushes a weekly table to a dashboard. Building it teaches you what can and can’t be seen.

Drill: run an agent test on a live site before and after a change to its bot settings. Compare verified agent hits, form reaches and submissions.

Proof of mastery: an agent traffic panel on one live site with four weeks of data, labelled by purpose, with the agent funnel next to it.

Stage 3: how do you make a site agent ready?

You’ll be able to: tell a client which standard fits their business in one line, and ship the first piece. Time box: weeks 10 to 15.

Sources: the specification pages linked in the table, all checked 3 November 2026. Chrome status from Daniel Vaughan’s October 2026 write up.

World class techniques

1. Schema first, standard second. NLWeb reads Schema.org, so clean structured data is the cheapest agent work and pays off in search too. Established for NLWeb, a hypothesis for other agents.
2. Fix the form before adding a standard. Labelled fields, ARIA roles, choices kept in the web address, and no tick box CAPTCHA for verified agents. In our Cloudflare agent test the agent failed on a reCAPTCHA tick box, not on a missing standard. Practitioner evidence, one test.
3. Pilot WebMCP on one form. Wrap a single enquiry form as a declarative WebMCP tool. It’s cheap, you can undo it, and few sites have tried it. It does nothing today in a browser without the preview switched on. Hypothesis.
4. Shops: pick UCP or ACP by platform. Shopify merchants get UCP through Shopify. Stripe merchants have the ACP route. Don’t build both. Practitioner evidence, vendor documents only.

Unconventional approach: break it on purpose. On a test page you own, ship a WebMCP form tool, then remove the labels, then add a CAPTCHA. Run the same agent each time and watch what it does.

Drill: write an agent readiness checklist (crawl access, a rule for verified agents, ARIA, schema, the form, the /.well-known files) and run it on two real sites.

Proof of mastery: one live WebMCP or NLWeb pilot on a site you own, and the checklist scored on two real sites.

Stage 4: where do the experts disagree?

You’ll be able to: hold a view on each open fight and say where it breaks. Time box: weeks 16 to 19.

1. Who owns it. John Mueller asked the question on LinkedIn and Joe Hall answered it: SEO owns discovery and understanding, developers own login, payments and APIs. Natzir Turrado’s AX idea, agent experience next to UX, pushes it towards a new role. Your view should name which half you sell.
2. ARIA for agents. OpenAI asks sites to add ARIA so Atlas can act. Adrian Roselli argues this invites ARIA keyword stuffing that hurts people who use screen readers. Read both.
3. Block or let in. Training crawlers give little back, with crawl to refer ratios in the hundreds to tens of thousands. Agents acting for a user bring buyers. One blanket rule is wrong for most businesses. That’s my reading of Cloudflare’s data.
4. Three checkout standards. UCP, ACP and AP2 overlap and nobody knows which wins. Treat any client spend here as a small bet you can undo.
5. WebMCP adoption. It’s still a draft in an early Chrome preview, and the API name has already changed once. Early pilots may need rewriting.

Unconventional approach: read against yourself. For each fight, write your view in three lines, then find the strongest source against it and rewrite.

Drill: one LinkedIn post per fight for business owners, one a fortnight. Track which ones draw objections and from whom.

Proof of mastery: a one page position paper, “What we do and don’t do on the agentic web”, with each claim labelled established, practitioner evidence or hypothesis.

Stage 5: how do you teach, sell and predict it?

You’ll be able to: sell an agent readiness check, run it with an agent, and make calls on the market that come true more often than not. Time box: weeks 20 to 24.

1. Turn the checklist into a service. An agent readiness check: an agent task test, a rule for verified agents, form fixes, schema and one standard picked. Hypothesis, because demand isn’t tested yet.
2. Publish proof, not opinion. Anonymised agent tests on real sites are what most agencies don’t have. Each one becomes a short case note.
3. Keep a prediction log. Ten dated calls, for example “WebMCP reaches stable Chrome by a set date” or “a top Irish retailer publishes /.well-known/ucp”. Score them every quarter.

Unconventional approach: ask the builder. Send one precise question to a WebMCP author or an A2A steering committee member, for example how agents should handle a step that needs a verified human. Publish the answer with permission.

Drill: run one agent readiness check end to end for a warm prospect, with an agent doing the task test.

Proof of mastery: one paid or signed off check, one published case note, and a prediction log with at least ten scored calls.

What should a prediction log look like?

Five columns are enough. Write the call before you check it, and never edit a row after the date passes.

The point is honesty with yourself. The reality is this: most predictions about the web sound right when you make them and nobody checks them later.

What doesn’t this roadmap prove?

It’s a learning plan, not a measured method. The order of stages and the time boxes are Rank Dough’s judgement, and no one has run all 24 weeks end to end yet.

The standards change fast. Every status in the Stage 3 table is dated 3 November 2026, and WebMCP in particular may look different in a few months. The screen reader link in Stage 1 and the agent funnel in Stage 2 are hypotheses until they’re measured on live sites.

Frequently asked questions about the agentic web roadmap

Sources

• Rank Dough agentic web roadmap research, 17 sources opened and checked on 3 November 2026
• Zhou et al., WebArena: A Realistic Web Environment for Building Autonomous Agents, arXiv 2307.13854v4: arxiv.org/abs/2307.13854v4
• W3C Web Machine Learning community group, WebMCP: github.com/webmachinelearning/webmcp

The post How do you master the agentic web? A 24 week roadmap appeared first on Rank Dough Digital.

It might be, and most owners wouldn’t know. Since 15 September 2026, Cloudflare blocks AI agents by default on pages that show ads for every new domain it takes on, and Rank Dough’s own test on 3 November 2026 found the bigger blocker sat further down the page: the agent got through Cloudflare on 3 of 3 prompts and through the enquiry form on 0 of 3.

What did Cloudflare change for AI agents?

On 1 July 2026 Cloudflare stopped treating AI traffic as one thing. It now sorts AI bots into three groups by what they do on your site: Search, Agent and Training.

Each group gets its own switch. You can block it on every page, block it only on pages that show ads, or leave it alone. Before this, Cloudflare offered one “Block AI bots” toggle that did everything at once.

The default is the part to watch. From 15 September 2026, every new domain added to Cloudflare blocks Agent and Training on pages that show ads, while Search stays allowed. If your site carries ads and moved to Cloudflare this autumn, agents may already be turned away on those pages.

Source: Cloudflare changelog, “New options to manage AI traffic”, 1 July 2026, and the Cloudflare blog post of the same date.

One thing Cloudflare doesn’t publish is how it decides a page “displays ads”. Its blog says an ad is a sign the page was built for a person to land on. It doesn’t say which ad scripts count, so I wouldn’t assume a page without banner ads is safe.

Why would a business want AI agents on its site?

Because the agent is the customer’s hands. Cloudflare’s own definition says there is often a human waiting on the other end. A student asks an AI to find a studio near their university and send an enquiry, and the agent goes off and does it.

If your site turns that agent away, the student doesn’t see an error page. The agent tries the next site on its list, and the enquiry lands with a competitor. Nothing shows in your analytics, because the visit never happened.

That’s why I treat this as a sales question, not a technical one. Blocking Training is a choice about your content. Blocking Agent is a choice about whether people who shop through AI can buy from you.

How do you check if your site is on Cloudflare?

Ask whoever runs your domain, or check it yourself in under a minute. Open your site in Chrome, press F12, go to the Network tab and reload. Click the first request and look at the response headers.

If you see “server: cloudflare” and a “cf-ray” header, your traffic goes through Cloudflare and this setting applies to you. If the server says Apache, nginx or something else, Cloudflare isn’t in front of your site and you can skip to the forms section.

When we checked on 3 November 2026, the student accommodation site in our test returned “server: cloudflare” with a cf-ray ID. Our own site, rankdough.com, returned Apache, so the setting doesn’t apply to us.

Where is the Agent setting in Cloudflare?

Log in to the Cloudflare dashboard and pick your domain. Go to Security, then Settings, filter by Bot traffic and open “AI Crawler and Agent access policies”.

You’ll see Search, Agent and Training listed with a choice for each:

1. Block: blocks that group on every page.
2. Block on pages with ads: blocks it only where Cloudflare detects ads.
3. Allow (do not block): lets it through.

Source: Cloudflare bot solutions docs, “AI Crawler and Agent access policies”, checked 3 November 2026.

Take a screenshot of what’s there before you change anything. Cloudflare’s changelog applies the new Agent default to new domains. On an older domain someone may still have switched on the old “Block AI bots” toggle and forgotten. The screenshot tells you where you started.

The Free plan gets these options too. Cloudflare’s changelog says all customers, including Free, can manage the three groups separately.

What happened when we sent an agent through a live site?

Rank Dough ran three booking prompts through a student accommodation site we work on, using an AI agent driving a browser. The prompts were a studio near King’s College London for September 2027, the cheaper of two properties near the University of Manchester, and the price of one named London property with an enquiry.

We logged each step the agent took and where it stopped, and sent no real enquiries. Every test used the name “TEST Rankdough” so the sales team could ignore it.

Source: Rank Dough agent test log, 3 November 2026.

What the agent saw, step by step

These are the pages the agent worked through on the first prompt, captured on 3 November 2026 with the site’s brand blurred. The red boxes mark where it moved on and where it stopped.

So Cloudflare let the agent in. The site’s own form kept it out. I expected the Cloudflare setting to be the story, and it wasn’t on this site. The tick box was.

If Cloudflare lets agents in, what else stops them?

The tick box CAPTCHA is the first one. It exists to stop robots, and an agent is a robot working for a real person. Every enquiry form on the test site used one: the form on the property page, the Enquire Now page and the Speak with us form.

The fix we gave the developer was to swap the tick box for an invisible check, such as reCAPTCHA v3 or Cloudflare Turnstile in managed mode. Real users stop seeing a puzzle, and the server still scores each submission for spam. We haven’t rerun the test since that change, so treat it as the plan, not a result.

Filters that don’t work in the web address are the second. The studio filter and the distance sort worked as URL parameters, so the agent could jump straight to them. The move in month only worked inside the Filters pop up, so the agent couldn’t filter to September 2027 directly.

Wrong data is the third, and it hurts people as much as agents. The “Premium Studio” Enquire Now page showed “Silver 2 Bed Apartment Plus” in its summary box. A person might spot that. An agent will send the enquiry for the wrong room.

The good news on that site: the forms are plain HTML that post with a security token, not built in JavaScript. That makes them easy for agents to read and easy to label later with WebMCP, a proposed web standard that tells agents what a form does.

Buttons and fields with no labels are the fourth. Agents read the page structure as well as the words, and OpenAI says its Atlas browser uses ARIA roles and labels to work out what a button or form field does. A button that is only an icon, or a drop down built from plain boxes, gives the agent nothing to go on.

The fix is ordinary accessibility work: a real label on every field and a clear name on every button. Don’t stuff those labels with keywords. Adrian Roselli, an accessibility consultant, warns that SEO tricks in ARIA make sites worse for people who use screen readers, and the same labels serve both readers.

Facts the agent can’t lift are the fifth. Price, availability and room type should sit in plain text and in structured data on the page, not only inside an image or a pop up. That’s also the cheapest agent work, because it already helps you in search.

Sources: OpenAI Publishers and Developers FAQ, checked 3 Nov 2026; Adrian Roselli, OpenAI, ARIA, and SEO, Oct 2025, updated Jun 2026.

Who was the agent, and did it reach checkout?

The agent was Claude, the AI assistant, driving the browser built into the Claude desktop app on my computer. It wasn’t a separate shopping bot. It searched, sorted, opened pages and read the forms the way an agent does for a person.

I’ll be plain about one thing. On this first run the agent read the page code and web addresses more than it clicked like a shopper. That makes it a kinder test than a browsing agent that clicks every button, which tends to get stuck earlier on pop ups and cookie banners.

It didn’t reach a checkout, because there isn’t one. A student accommodation site doesn’t take card payments online, it takes enquiries. There are two routes: Enquire Now, which stopped at the tick box, and Book Now, which opens a six step application with education, address, emergency contact and guarantor steps.

The application asks for a date of birth, a guarantor and an emergency contact before anyone replies. An agent acting for a student in London or Mumbai also has to notice the American defaults and change them. That’s a lot of steps to get right without a person checking.

What I took from it: the agent did the research part well. Price, distance and room type were found in seconds. It’s the hand over to the business, the form, where it broke, and that’s the part most SEO work never touches.

What happens on a big retail site like Currys?

We ran the same kind of test on Currys UK on 3 November 2026, because it sits on Cloudflare and has a real online checkout. The prompt was simple: find wireless earbuds under £100 and get them to checkout. We ran it twice, once from a cloud server and once from a normal desktop browser.

From the cloud server the agent never saw a product. The home page and the search page both returned a Cloudflare block, “Sorry, you have been blocked”, with a 403 status. Several agents, ChatGPT’s agent among them, browse from remote servers, so this is the first thing many of them meet.

From the desktop browser it got further. The home page passed a Cloudflare check after a few seconds. The search results page then showed a “Verify you are human” box, and the agent stopped there, because it doesn’t solve CAPTCHAs.

The way round was the search box itself. Typing “earbuds” showed product suggestions with prices, and the agent picked Sony WF-C510 earbuds at £39.99 straight from the list. It’s the same lesson as the student site: the more a page shows in plain text, the further an agent gets.

Adding them to the basket opened a pop up offering a three year Instant Replacement plan for £13. The agent had to read it and choose “Continue without Instant Replacement”, or it would buy something nobody asked for. A second panel then suggested a smartwatch and a cable.

The basket and checkout loaded with no more checks. Guest checkout asked for an email address, and the box to join Currys Perks for marketing was already ticked. We stopped there, because the next step needs real personal details.

An agent acting for a real person has to untick that box, or the customer lands on a mailing list they never chose. That’s nothing for a person and an easy miss for an agent.

Source: Rank Dough agent test log, Currys UK, 3 November 2026. No order placed and no personal details entered.

So on a big retailer, Cloudflare was the story after all. On the student accommodation site it let the agent in and the form stopped it. On Currys the same service decided which agents got in at all, depending on where they came from and which page they asked for.

How far did each agent get?

Rank Dough scores every agent test on four steps, because a visit is not the result. An agent that finds the page and can’t send the enquiry is a lost lead that never shows in your analytics.

Source: Rank Dough agent test logs, 3 November 2026. One run per site. The cloud agent on Currys was blocked at step 1 with a 403.

Which setting should you choose?

This is my view, not a tested rule. If you sell, take bookings or collect enquiries, allow Agent on the pages where people buy or enquire. Those are the pages an agent visits for a customer who has already decided to act.

Training is a separate call. Blocking it stops your content going into model training and doesn’t stop a customer’s agent from booking. Plenty of owners will block Training and allow Agent, and that’s a fair split.

Publishers who live on ad revenue have a real reason to block agents on ad pages. My reading is that an agent reads the page without looking at the ad, so the visit earns nothing. For them the default makes sense. For a business that sells something, it usually doesn’t.

There is a middle way. Some agents now sign every request so a site can check who they are. Cloudflare calls them signed agents, and the method is an open draft standard written by engineers at Cloudflare and Google. Letting verified agents through while you keep challenging unknown bots is a fairer rule than one switch for everything.

It has a gap. An agent driving a person’s own browser, like the one in our tests, looks like that person and carries no signature. So the Cloudflare setting is only half the check. The form test is the other half.

Sources: Cloudflare, The age of agents, 28 Aug 2025; IETF draft, HTTP Message Signatures for automated traffic, 2 Mar 2026.

How fast is AI agent traffic growing?

Fast, from a small base. HUMAN Security, which measures bot traffic across its customers’ sites, says traffic from AI agents grew 7,851% year over year in 2025. Its monthly reports for 2026 show the curve getting steeper, not flatter.

Chain the monthly changes together and agent traffic in September 2026 was more than four times its March level. Most of that came in one month: September rose 209.7% on August, with Meta’s new Muse agent going straight to number one.

The index is our calculation. We chained HUMAN’s reported month on month changes from March 2026, so it covers HUMAN’s customers only and moves if HUMAN revises a month. Ecommerce took about 60% of agent traffic in September and travel 17.1%, and booking a room sits close to both.

Sites are pushing back at the same time. The share of agent traffic HUMAN saw blocked went from 4.3% in March to 21.9% in August 2026. That’s about one agent visit in five turned away, before anyone checks whether the agent was working for a customer.

The money side is moving too. Adobe says AI-referred visits to US retail sites converted 38% worse than other traffic in March 2025. By March 2026 they converted 42% better, and by July 2026, 60% better.

Those are shoppers arriving from AI assistants, not agents buying on their own. It’s still the same shopper getting used to letting AI do the legwork, and the next step for that shopper is letting it send the enquiry too.

McKinsey’s forecast is the big number: up to $1 trillion in US retail revenue run through agents by 2030, and $3 trillion to $5 trillion worldwide. It’s a forecast, not a measurement, so I’d read it as a direction rather than a promise.

Sources: HUMAN Security, 2026 State of AI Traffic report (26 March 2026) and State of Agentic Traffic reports (May to October 2026); Adobe blog, 16 April 2026; Digital Commerce 360, 19 August 2026; McKinsey, The agentic commerce opportunity, 17 October 2025.

Who owns the agentic web on your team?

John Mueller at Google asked this on LinkedIn, and Joe Hall answered it in a longer piece. As I read his argument, SEOs shouldn’t own the agentic web. SEO owns whether an agent can find you and understand you. Developers own the parts where it acts: logins, payments, forms and APIs.

Our test fits that split almost exactly. Everything SEO touches worked: robots.txt, crawlable pages, prices in plain text, filters in the web address. Everything that belongs to the developer stopped the agent: the CAPTCHA, the room data behind the form and the six step application.

So I’d change the question. It isn’t who owns it, it’s who checks the hand over. Somebody has to run an agent from the first search to the submitted form, and right now that job sits between two teams and belongs to neither.

The developer side is where the new standards are landing. MCP lets an AI assistant connect to a business’s own tools. A2A lets agents talk to other agents. WebMCP, an early preview in Chrome and still a draft, lets a page tell an agent what a form does and which fields it needs, with a few attributes on the form rather than a rebuild.

In the comments under that debate, Natzir Turrado called this AX, agent experience, sitting next to UX. I like the name because it puts the form at the centre, not the ranking.

For a shop there are three more names to know. They all deal with the checkout, and none of them is a clear winner yet.

Sources: WebMCP repository; Google, Under the Hood: UCP, 11 Jan 2026; Stripe, Agentic Commerce Protocol; AP2 core concepts. All checked 3 Nov 2026. The views are mine, not tested.

The order matters more than the standard. Fix the CAPTCHA, the labels and the data first. A new protocol on a form an agent can’t submit changes nothing.

Where is the web heading?

This is my reading, not a forecast. The web is turning into one page with two readers. People still read, compare and decide who to trust. Agents read the same page for facts they can act on: price, dates, room type and the next step.

The finding part looks a lot like SEO today. An agent finds you the way AI search finds you, through pages it can crawl and facts it can lift. That’s why I don’t see agents as a threat to SEO. They’re one more reader we have to write for.

The new part is the last mile. Up to now a website’s job ended when a person landed and filled in the form. With agents the form is the shop window and the till at once, and a tick box that costs you nothing with people can cost you every enquiry an agent tries to send.

The simple truth is that most sites will block agents by accident long before they decide to welcome them. The ones that check the hand over now will get the enquiries the others never see in their analytics.

What I’d do this month

1. Check whether your site is on Cloudflare and screenshot the AI Crawler and Agent access policies page before changing anything.
2. Allow Agent on the pages where people book, buy or enquire. Decide Training separately.
3. Replace tick box CAPTCHAs on enquiry forms with an invisible check.
4. Make the main filters work in the web address, and check the data on each form matches the page it came from.
5. Run one agent from a real customer prompt to a submitted test form every quarter, and log where it stops.

Is Cloudflare blocking agents on your site?

Tick what’s true for your site. The answer tells you where to look first and links to the part of this post that explains it.

The verdicts are Rank Dough’s reading of Cloudflare’s published defaults and our one test, not a measured model. Without JavaScript, the same checks are:

1. Site on Cloudflare and domain added after 15 September 2026, with ads on key pages: Agent is likely blocked on those pages. Open the Agent setting.
2. Site on Cloudflare and nobody knows who set the bot settings: screenshot the AI Crawler and Agent access policies page before changing anything. See where it is.
3. Tick box CAPTCHA on the form: agents stop there even when Cloudflare lets them in. See what else stops agents.
4. Site not on Cloudflare: this setting doesn’t apply, but the form still might block agents. Check the headers.

What doesn’t this post prove?

It’s one site, one agent and one day, 3 November 2026. A different agent, or the same agent on another day, may get stopped earlier or later.

The agent was Claude reading pages more than clicking them on the student site, so a browsing agent that clicks every button may stop earlier. On Currys we tested one product on one day and haven’t seen its Cloudflare settings, so we can’t say which rule blocked which run. The growth figures come from HUMAN Security’s and Adobe’s own customer data, not the whole web, and McKinsey’s 2030 figure is a forecast.

We haven’t seen that site’s Cloudflare dashboard yet. The agent got through, but we can’t tell you whether the Agent setting is on Allow or whether the site’s pages simply don’t count as ad pages. That’s the next thing we’re checking with the site’s developer.

We also haven’t rerun the test after the form fix. When we do, we’ll add the result here with the date.

Frequently asked questions about Cloudflare and AI agents

Sources

• Rank Dough agent test log, student accommodation site on Cloudflare, three prompts, 3 November 2026
• Cloudflare changelog, New options to manage AI traffic, 1 July 2026: developers.cloudflare.com/changelog/post/2026-07-01-ai-traffic-options/
• HUMAN Security, State of Agentic Traffic, September 2026: humansecurity.com/learn/blog/september-2026-agentic-traffic-triples-meta-muse-debuts-at-no-1/

The post Is Cloudflare blocking AI agents from your site? appeared first on Rank Dough Digital.