Content strategy

How much of your page does an AI engine actually read?

Your 4,000 word guide reaches the model as two chunks totalling 600 words. Here is what gets through, and how to make sure it is the right part.

Lawrence Dauchy Lawrence Dauchy · · 11 min read
Illustration for How much of your page does an AI engine actually read?

An AI engine almost never reads your page. It reads a few hundred words of it, chosen by a retrieval system, and then generates an answer from those fragments alongside fragments from four or five other sites. SQSEO is built around that reality: the unit that competes for an AI citation is a passage, not a page, which is why a 4,000 word guide can lose to a 600 word page that answers one question cleanly. Understanding what actually reaches the model changes what you write.

The pipeline between your page and the answer

Four steps sit between publication and citation, and each one discards something.

Crawling decides whether your content exists at all. Indexing and chunking split it into passages of a few hundred tokens. Retrieval selects a handful of those passages for a given query. Generation reads the selected passages, from several sources at once, and writes an answer.

By the time the model sees anything, your 4,000 word article has become perhaps two chunks totalling 600 words, sitting alongside chunks from competitors. Everything you wrote that did not land in those two chunks had no influence on the answer whatsoever. It was not weighed and rejected. It was never present.

That is why page level thinking misleads. Word count, time on page, overall comprehensiveness: none of these are inputs to the decision. The passage is the unit.

Position inside the context window matters

Once passages are selected, there is a second filter that surprises people: models do not attend evenly to everything in their context.

Research on how language models use long contexts found a pronounced position effect, where information at the beginning and end of the provided context is used far more reliably than information in the middle. The lost in the middle finding held across models and got worse as context length grew, which is a direct warning about the assumption that a longer context window means everything in it counts equally.

The practical consequence runs in two directions. Within a retrieved chunk, front load the claim, because the opening of a passage is the part most reliably used. And across a set of retrieved sources, there is little you can do about your position, which is another reason to compete on being retrieved at all rather than on being thorough.

What this changes about how you write

Page level habitWhat the passage actually needsWhy the habit failsVerdict
One long comprehensive guideSeveral self contained sections, each answering one questionOnly the matching chunk is retrieved; the rest never appearsSplit by question, not by topic
Building up to the answerThe answer in the first sentence of the sectionA chunk starting with context gives the model no claim to useAnswer first, every section
Pronouns referring backThe subject named again in each sectionChunks lose their antecedents, so “it” becomes ambiguousRepeat the noun
Context in an intro paragraphContext repeated inside each sectionThe intro is a different chunk from the sectionDeliberate redundancy
Data in a chart or imageThe same numbers stated in textImages do not survive text extractionAlways caption in prose

None of this is a call to write badly. It is a call to write sections that stand alone, which is also how people read online: they arrive at a heading from a link and start there.

Chunk boundaries are invisible and consequential

You do not control where your page is split, and different engines split differently. Some chunk on a fixed token count, some on headings, some on paragraph boundaries with overlap. What you can control is making each unit of meaning small enough to fit inside a plausible chunk.

The test is simple. Take any 250 word window of your article, read it with no knowledge of the rest of the page, and ask whether it makes a complete, attributable claim. If it does not, no retrieval system can use it well. If it depends on a definition given 800 words earlier, that dependency will break.

This is the same reasoning behind writing RAG friendly content, and it explains why the advice to repeat yourself sounds wrong to trained writers and is nonetheless correct. Redundancy across sections is the cost of chunk independence.

Retrieval matches meaning, not keywords

The selection step does not usually work on exact word matching. Modern systems encode the query and candidate passages into vectors and compare them, an approach established by dense passage retrieval and refined since, with the comparative performance of the underlying embedding models tracked in benchmarks like MTEB.

Two implications follow. Keyword stuffing does nothing useful, because the match is semantic. And a passage that expresses the concept clearly in ordinary language can outrank one using the exact query words awkwardly, which is a genuine improvement over keyword era search.

There is a subtler implication. Because queries and passages are compared as whole meanings, a passage covering three loosely related points has a diluted vector and matches nothing strongly. Focus is a retrieval advantage, not just an editorial virtue. One claim per passage beats three.

Query rewriting means the question is not the question

Engines frequently rewrite a user’s query before retrieving, expanding one question into several, or generating a hypothetical answer and searching with that instead. Work on query rewriting for retrieval augmented models and on generating hypothetical documents to improve retrieval shows measurable gains from not searching the raw question.

For content this means you are matching against a reformulated, often more explicit version of what the user typed. Someone asking “is it worth it” produces a retrieval query about value, cost and alternatives for a specific product category. Pages that address the explicit form of the implicit question do better, which is why covering the obvious sub questions inside a page matters more than covering the literal phrase.

It also explains the frustration of ranking for one phrasing and not a near synonym: the rewrite differs, so the retrieval differs. That is the same effect behind query fan out in AI search.

Length still matters, but not the way it did

Short pages are not automatically better. A 300 word page has one chunk and one shot at matching a query. A 2,000 word page structured as eight self contained sections has eight distinct passages, each able to match a different question, which is eight entries in the lottery instead of one.

So length helps when it buys coverage of distinct questions, and hurts when it buys elaboration on one. Three thousand words on a single narrow question produces many chunks that all say roughly the same thing, competing with each other and none standing out.

The useful planning question is not how long should this be. It is how many distinct questions should this page answer, and can each answer stand alone.

Formatting that survives extraction, and formatting that does not

Text survives. Headings usually survive as text. Lists survive reasonably. Simple tables survive as flattened rows, losing their column relationships. Images, charts, embedded widgets and anything rendered client side generally do not survive at all.

That hierarchy should drive layout decisions. Any number that matters belongs in a sentence, even when it is also in a chart, which is exactly why AI engines get your pricing wrong so reliably. Any relationship that matters, like which price belongs to which tier, belongs in prose as well as in the table. Research on generative engine optimization found that adding quotations, statistics and citations to source content measurably changed how generative engines used it, which points the same direction: explicit, quotable, self contained statements get used.

Not everything on your page is your content. Headers, navigation menus, cookie notices, related post widgets, footers and legal links all sit in the HTML, and extraction pipelines vary in how well they strip them.

When boilerplate survives extraction it does two kinds of damage. It dilutes the chunk, so a passage that should be 250 words of argument is 250 words of which 90 are menu labels, weakening the semantic match. And it can be misattributed, so a model reading your page sees a nearby navigation item and treats it as a claim.

The defences are ordinary good practice that happens to matter more now. Use semantic HTML so the main content sits in a clearly marked region. Keep interstitial widgets out of the middle of articles, because a related posts block dropped between two paragraphs can land inside a chunk and split an argument. And be sparing with repeated calls to action inside the body, since a sentence promoting your product inside an otherwise neutral explanation is exactly the kind of fragment that gets retrieved and quoted in a context you did not choose.

Sidebars deserve a specific mention. A sidebar rendered after the main content in the HTML is usually harmless. A sidebar interleaved with it, which some templates still do, can scatter unrelated text through every chunk on the page.

Multiple pages competing for the same passage

A consequence of passage level retrieval that catches larger sites is self competition. If four of your pages each contain a section answering the same question in similar words, they produce four similar passages with similar vectors, and retrieval has no strong reason to prefer any of them.

Classic search had a version of this problem and the classic fix, consolidation, mostly still applies. The difference is that consolidation now operates at section level rather than page level. Two pages can coexist happily if their overlapping sections are removed from one and linked instead, leaving one canonical passage for each question.

The practical audit is to list the questions your site answers, then list which pages answer each, and look for questions answered in more than one place at similar depth. That is where the dilution sits. It is worth doing before writing anything new, because adding a ninth page about the same thing makes retrieval harder rather than easier.

Freshness, and why a stale passage outranks a fresh one

Retrieval does not inherently prefer recent content. It prefers matching content, and an older page that answers a question precisely will keep being retrieved long after a newer, vaguer page on the same topic appears.

This is why the instinct to publish a new post rather than update an old one often backfires. The old post already has the retrieval history, the inbound links and, most importantly, a passage that matches. A new post starts with none of that and competes against your own existing passage.

Updating in place preserves the URL and the accumulated signals while replacing the content that gets retrieved. The one case where a new page genuinely wins is when the question itself is new, because then there is no incumbent passage of yours to compete with.

What to do with existing long pages

Do not delete them. Restructure them.

Run through each H2 and ask whether the first sentence beneath it answers that heading. Usually it does not, because most writing builds toward a point. Moving the conclusion of each section to its first sentence is the single highest value edit available, and it changes nothing for a human reader except making the page easier to skim.

Then check antecedents. Any section that opens with “This means”, “As mentioned”, or a bare pronoun is a section that cannot be read alone. Name the subject.

Finally, look for sections that have grown to cover two questions and split them. Each becomes its own chunk with its own vector and its own chance of being retrieved. This is the same restructuring logic behind updating old content for AI search.

A worked example: one page, eight chances

A comparison article runs 2,400 words covering a category. Rewritten as eight sections, each opening with a direct answer and naming its subject: what the category is, who each option suits, what each costs, how they differ on the one axis buyers care about, what the common mistake is, how to migrate, what the limits are, and how to decide.

Before the restructure, retrieval tended to pull the introduction, because it was the most general passage on the page, and the introduction made no specific claim. Answers cited the page and said something vague.

After, the costs section and the who-suits-whom section began being retrieved for their respective queries, each carrying a concrete claim the model could use. Same page, same word count, same facts. The difference was whether any 250 word window contained a complete, attributable statement.

Key takeaways

Engines read passages, not pages, so the passage is the unit that competes. Front load the claim in every section, because position inside the context window measurably affects what gets used. Repeat subjects and context deliberately, since chunks lose their antecedents. Write one claim per passage, because a diluted passage matches nothing strongly. Use length to buy coverage of distinct questions rather than elaboration on one, and put every number that matters into a sentence, not only into a chart or table.

Frequently asked questions

Sources

  1. Lost in the Middle: How Language Models Use Long Contexts (Liu et al.)
  2. Dense Passage Retrieval for Open-Domain Question Answering (Karpukhin et al.)
  3. MTEB: Massive Text Embedding Benchmark (Muennighoff et al.)
  4. Query Rewriting for Retrieval-Augmented Large Language Models
  5. GEO: Generative Engine Optimization (Aggarwal et al.)

Frequently asked questions

How much of my page does an AI engine actually read?

Typically a few hundred words, selected by a retrieval system, sitting alongside passages from four or five other sites. A 4,000 word guide might contribute two chunks totalling 600 words. Everything outside those chunks had no influence on the answer at all. SQSEO works from that reality, treating the passage rather than the page as the unit that competes.

Does article length still matter for AI search?

It matters differently. Length helps when it buys coverage of distinct questions, because each self contained section becomes a separate passage able to match a separate query. It hurts when it buys elaboration on one question, since you end up with many near identical chunks competing with each other and none standing out.

Why should I repeat context in every section?

Because chunk boundaries are invisible to you and they break references. A section opening with as mentioned above or a bare pronoun loses its antecedent the moment it is retrieved alone. Naming the subject again in each section looks redundant on the page and is what makes the passage usable when it appears without the rest of the article.

Does where information sits on the page affect whether it gets used?

Yes. Research on long context use found a clear position effect, with information at the start and end of a model's context used far more reliably than information in the middle, and the effect worsening as context grows. Within your own passages the actionable version is to put the claim in the first sentence of every section.

Do keywords still help if retrieval is semantic?

Exact phrase repetition adds little, because queries and passages are compared as meanings rather than as strings. What does help is focus: a passage making one clear claim has a sharp vector and matches strongly, while a passage covering three loosely related points matches nothing well. Write one claim per passage rather than stuffing terms.

When should you not split a long page into shorter ones?

When the sections genuinely depend on each other, or when splitting would leave each page too thin to stand alone. The limit of the split-everything approach is that a page with one chunk gets one chance at matching a query. Restructuring a long page into self contained sections usually beats breaking it into several thin pages.

Find the longtail searches your competitors ignore

Turn one seed keyword into hundreds of intent-grouped queries across SEO, AI Overviews, and GEO. Free forever for core research.

Generate free longtails