Boilerplate ratio: how nav and footers shrink your page
Main-content extraction keeps one block of your page and drops the rest. How to measure what survives, by hand and across a site, and the markup that helps.
fundamentals
13 posts
llms.txt: what it is, how it works, and whether to bother
Is llms.txt worth it? The proposal, the file format with a worked example, what is documented about who reads it, and when the afternoon pays off.
robots.txt patterns for AI: allow, block, and the middle
Four positions cover almost every site. The copy-paste file for each, the matching rules that quietly break them, and the maintenance the file needs.
FAQ sections: leverage or filler
The rich result that justified most FAQ blocks is gone. What still earns one, the test for whether a question belongs in the body, and what filler costs.
Tables and lists that survive extraction
Real markup is only half of it. What flattening does to a table lifted out of your page, and the four fixes that keep one readable on its own.
Chunking: why passages have to stand alone
A page is retrieved in pieces of a few hundred words. What splitting does to ordinary prose, and the three edits that make a section survive it.
What do AI crawlers extract from your page?
An element-by-element inventory of what AI crawlers keep, discard, and only conditionally read — plus the parts you can exclude on purpose.
Answer Engine Optimization: a page-level guide
What happens to a page between fetch and answer, the eight signals answer-readiness turns on, and how to check every one of them by hand and at scale.
SEO, AEO, and GEO: what actually changed
Three acronyms, one page. Here is what answer engines and generative search changed about optimization, and what they did not.
GPTBot vs OAI-SearchBot: a field guide to the AI crawlers
Every AI bot worth naming in your robots.txt, what each one actually controls, and which ones decide whether an assistant can find you at all.
AI crawler access: the technical layer of AI search
Which AI bots exist, what each one actually controls, why robots.txt is only half the answer, and how to check whether assistants can reach your pages.
The 40–60 word answer block, and how to write one
The highest-weighted answer-readiness signal is a passage that answers its own heading in one breath. What it is, how to write one, how to check it.