What Happens When an AI Agent Visits Your Site

A plant hire company in the West Midlands asked one of its long-standing customers why the quote requests had dried up. The customer had not gone elsewhere on purpose. They had simply started handing the job to an assistant — find me a three-tonne excavator available next Tuesday within twenty miles of Wolverhampton, with day rates — and working from whatever came back. Three firms came back with a rate and a contact. The fourth, which has the largest fleet of the four and a depot four miles closer to the site, was not mentioned.

Nothing on that fourth website is broken. A person can open it, browse the fleet, check availability on a calendar, and send an enquiry in under two minutes. But the fleet list is drawn by a script after the page loads, the day rates live in a downloadable rate card, and the enquiry form's fields are labelled only by grey placeholder text that disappears the moment you start typing.

An AI agent does not see a website the way a person does. It works from structure — headings, labels, links, forms, and machine-readable data — rather than from layout, so anything carried only by visual design, or by a script the agent cannot run, is effectively invisible to it. The sites an agent can complete a task on tend to be the ones that state their information plainly and expose their actions as real, labelled endpoints. Everything a human needed was on that hire company's site. Almost nothing an agent needed was.

What an AI agent is, in plain terms

An agent is a piece of software given a goal rather than a click path. Book a table for six on Friday. Get me three quotes for a static caravan tow. It then works through the steps on its own: searching, opening pages, reading them, comparing, and where it can, filling something in.

It inverts the usual assumption. A website is designed for a visitor who arrived with intent and will forgive a bit of hunting. An agent arrives with a task and a budget — a fixed number of page fetches and seconds before it gives up and reports back with whatever it has. If your answer takes four clicks and a PDF download, the agent will use the competitor whose answer took none.

What happens when an AI agent visits your website

The sequence is unglamorous. The agent requests your page over plain HTTP, usually identifying itself in the user agent string, and often checking robots.txt first. It receives the HTML your server sent — not the page a browser eventually assembles.

From that HTML it extracts a rough map: the title, the headings in order, the paragraph text, the links and where they point, any tables, any forms and their fields, and any structured data you have included. It then looks for the facts its task needs — a price, a date, a coverage area — and either answers from them or moves to an action: submitting a form, following a booking link, or calling a documented endpoint.

The whole visit is typically a handful of requests over a few seconds. There is no scrolling, no hovering, and no patience.

How an agent reads a page differently from a person

A person reads a page as a picture. The largest text is the important bit, the boxed thing on the right is probably the price, the orange rectangle is the button. None of that hierarchy exists in the markup unless someone put it there deliberately.

Consider a hire page that shows £185 in 32-point type beside a photograph of a digger. To a person, that is unambiguously the day rate. To an agent, it is a <div> containing the characters "£185", with no indication of what it prices, whether it includes VAT or delivery, or which of the four machines on the page it belongs to. A sentence such as Day rate for the three-tonne excavator: £185 excluding VAT, delivery within 20 miles included is less striking and considerably more useful, because it survives being lifted out of the page.

The same applies to structure. A heading that is bold text at 24 pixels is decoration. A heading that is an <h2> is a landmark. Agents navigate by landmarks.

What quietly locks an agent out: scripts, walls and unlabelled controls

Five things account for most of what we see, and none of them look like faults.

Content assembled by script. If the prices, stock levels, or opening hours only appear after JavaScript runs, many agents will never see them. The served HTML is an empty shell.

Walls. A login, a cookie banner that blocks the page until dismissed, a region interstitial, or a CAPTCHA in front of ordinary information. Each stops the visit outright.

Unlabelled controls. A form field whose only description is placeholder text has no accessible name. An agent sees three anonymous text boxes and cannot tell which one wants the postcode.

Content in the wrong container. Rate cards as PDFs, opening hours drawn into an image, a service area shown only as a map graphic.

Blanket bot blocking. Rules written years ago against scrapers now exclude the agents your customers are sending. Worth reviewing rather than assuming.

Fixing these is mostly the same work as making a site legible to any machine reader, which is why our notes on what agents need from a site overlap heavily with plain accessibility advice.

Agent traffic is not search traffic, and most analytics does not separate the two

Agent visits look strange in analytics, and most dashboards quietly discard them. There is often no referrer, no cookie, and no return visit: six pages read in nine seconds, no button clicked, session over. In a standard report that is a bounce from an unknown source, filtered out as a bot or averaged into nothing.

Server logs are more honest. Filter a week of raw log lines by user agent and you will usually find a few hundred requests you did not know about, concentrated on a handful of pages — typically pricing, contact, and whichever page names your service area. That distribution tells you what agents came looking for, and whether they found a page or a redirect chain.

The practical step is to stop treating this as search traffic. It has different intent, different behaviour, and different failure modes.

Being cited in an answer is not the same as being usable by an agent

These two are often conflated, and they need different work. Being cited means an assistant names you when someone asks a question — a matter of having plain, quotable, corroborated text on the open web. That is the ground covered by generative engine optimisation.

Being usable means an agent can carry out a task with you: check a date, read a price, submit an enquiry, start an order. A business can be cited constantly and still be unusable, which produces the worst outcome of the two — the agent names you, tries to proceed, hits a script-rendered calendar, and quietly substitutes the firm whose booking form it could complete.

The hire company was in exactly that position on two of the five questions we tested. Named, then abandoned.

What "agent ready" means in practice, in order of effort

Roughly in ascending order of work, and descending order of return.

Put the core facts in served HTML as text. Prices or ranges, lead times, coverage area, opening hours, minimum order.
Use real headings, one idea each, worded as a customer would ask.
Label every form control properly, with a visible label tied to the field.
Keep URLs stable and meaningful, one page per thing.
Add structured data for the entities you have — organisation, product, service, opening hours, FAQ.
Expose your actions as endpoints. Availability, quotes, and bookings as documented, callable operations rather than screens only. Most small firms assume this is out of reach; the notes for developers set out what it involves, and our agent-facing documentation covers the descriptive layer.

Websites, applications and the difference an agent notices

There is a real distinction between a site that describes a business and one that lets you do something with it. A brochure site can be made readable. It cannot be made transactable, because there is nothing underneath to transact with.

This is why the container matters. When a rate card, a stock list, or a booking sheet becomes a published application rather than a page of formatted text, the data behind it is structured by definition: rows, fields, and types rather than sentences. A structured back end can answer a question consistently, whether the asker is a person, a browser, or an agent working through a task list. We set out the wider argument in why applications need to be ready rather than retrofitted; the short version is that bolting an interface for machines onto an unstructured site is far more work than starting from structured data.

How to check what an agent can do on your site today

Three checks, about twenty minutes.

Read what your server sends. Run curl -s https://yoursite.com/prices | less, or switch JavaScript off in your browser and reload. Whatever is missing is missing for a large share of agents too.

Give an assistant a real task. Not tell me about this company, but the job a customer would delegate: using this website, find out whether they deliver to LS12 and what the minimum order is. Watch where it stalls. Then run the same task against two competitors.

Look at your logs. A week of requests, grouped by user agent, tells you which agents are already arriving and which pages they ask for.

For the plant hire firm, the first check took four minutes and explained everything: the served HTML contained a navigation bar, a footer, and an empty container where the fleet should be. What changes once you have run these is not your ranking or your traffic. It is that you know which of your answers exist in a form anything other than a human eye can find — and that is a list you can work through.