What to report to the board about AI search
Report four things: whether assistants describe you and how accurately, which pages machines are reading, how impressions move against clicks, and the share of new conversations that begin already informed. Show no figure that cannot be traced to where it came from.
The failure mode of reporting on this subject is a slide with a confident composite score on it. The first person to ask how it is calculated gets an unsatisfying answer, and the whole programme becomes the thing marketing could not explain.
The one page
| Section | Shows | Source |
|---|---|---|
| Presence | Of twenty fixed buyer questions, how many name us, and in which assistants | Monthly sampling, answers recorded |
| Accuracy | Of those mentions, how many describe us correctly | Same sample, read by a person |
| Machine reading | Which pages assistant crawlers fetched, and whether fetches succeeded | Server logs, verified agents |
| Search shape | Impressions against clicks for question queries | Search Console, exported |
| Arriving informed | Share of new conversations where the buyer already understood our method or pricing | Sales notes |
Five rows, each traceable. Nothing on the page is modelled, and every figure can be re-derived by somebody who does not trust you.
Show one figure nobody can trace and the traceable ones stop counting too.
How to handle the three questions you will be asked
Not separable, and say so immediately. Give the cost, which is usually small, and the counterfactual: the questions our market asks are being answered by somebody, and at present that is increasingly us.
Ask what the 42 measures. It is a vendor's sample of prompts the vendor chose. Offer the comparison that is real: on these twenty questions, we are named in this many and they are named in that many.
The honest lever is publishing more settled questions, and the constraint is almost never budget. It is that somebody has to decide what may be said about price and about the competition.
What to leave off
- Any composite score you did not define and cannot explain in one sentence.
- Estimated impressions inside assistants. Nobody has them.
- A number of citations, unless you can show where each one was observed.
- Month-on-month movements from a single sample run, which mostly measure model variance.
- Competitor figures you cannot source.
Questions this settles
Quarterly to a board, monthly internally. The series is the reading, and a board looking at it monthly will react to variance.
Only if it publishes its method and you can explain it in one sentence. A tool that automates the sampling saves real work; a tool that produces an unexplained score creates a liability.
Being named on questions where you were not, and being described accurately where you were described wrongly. Traffic is the wrong first-year outcome to promise.