Prompt Cohort Migration Needs a Bridge Table Before Citation Rates Can Trend
AI citation-rate reports should version prompt cohorts, separate retained prompts from additions and removals, and label full-basket rates as noncomparable when the monitored prompt set changes.
AI citation-rate reports can trend across time only when the prompt cohort, answer-run denominator, and observation rules remain comparable. When the monitored prompt basket changes, the report needs a bridge table: retained prompts form the comparable slice, while removed and added prompts explain why old full-basket and new full-basket rates are not the same time series.
The operating rule is narrow: version the cohort before interpreting the rate.
Why prompt cohort migration breaks citation-rate trends
A citation rate is a ratio. The numerator is usually the count of observed answer runs that cited a source, domain, URL, or brand. The denominator is the count of observed answer runs eligible for that metric. If the prompt basket changes, the denominator has changed even when the formula still says cited runs / observed runs.
That is why a report should not say visibility improved just because a new prompt basket has a higher full-basket rate. The new basket may contain prompts where the measured source is naturally easier to cite. It may remove prompts where the source rarely appeared. The arithmetic can rise while source selection did not improve inside the stable portion of the cohort.
Google Search Console makes a related denominator warning visible in search data. Google explains that clicks, impressions, and position are counted inside specific Search result contexts and that aggregating by page, property, query, device, or country can change what the reported number means (Google Search Console Help). Those are not AI citation metrics, but the measurement lesson is the same: the aggregation unit has to be named before the number can be interpreted.
The same discipline applies to AI answer monitoring. A prompt basket is not just a list of questions. It is part of the measurement instrument.
A versioned prompt cohort should separate prompts from answer runs
Prompt counts and run counts answer different questions. A prompt count tells the reader how many monitored question templates exist. A run count tells the reader how many answer observations were actually collected after engines, dates, locales, retries, and failures are applied.
OpenAI's public Evals repository treats eval data as versioned samples and describes examples where a template consumes an input prompt while records live in sample files (OpenAI Evals documentation). AI visibility measurement does not have to copy that implementation, but it should copy the habit: identify the sample set, version it, and keep the observation records separate from the prompt definitions.
A minimal prompt-cohort record needs these fields:
| Field | What it stores | Why it matters |
|---|---|---|
cohort_id |
Stable name such as ai_visibility_geo_v2026_09 |
Prevents old and new baskets from sharing one label |
prompt_id |
Durable identifier for each question template | Allows retained prompts to be matched across versions |
prompt_text_version |
Exact wording or a content hash | Separates a wording edit from a retained question |
segment |
Category, question shape, locale, engine panel, and audience | Stops unlike markets from pooling into one denominator |
status |
retained, added, removed, or rewritten |
Powers the bridge table |
effective_from |
First date the prompt version is eligible | Defines the start of the new denominator |
retired_at |
Last eligible date for removed prompts | Prevents removed prompts from silently disappearing |
This is also a traceability problem. NIST describes metrological traceability as an unbroken chain of calibrations and documentation connecting a measurement result to a reference (NIST metrological traceability FAQ). Citation reporting is not laboratory metrology, but a practical visibility report still needs a documented chain from the published percentage back to its prompt cohort, collection dates, engine panel, and exclusion rules.
The bridge table that preserves comparable citation-rate slices
Use a bridge table whenever a prompt basket changes. The table should show retained prompts, removed prompts, added prompts, observed runs, cited runs, and comparability labels.
The numbers below are explicitly hypothetical. They are included only to show the math.
| Cohort slice | Prompt status | Prompts | Observed answer runs | Cited answer runs | Citation rate | Trend use |
|---|---|---|---|---|---|---|
Old full basket v2026_08 |
Retained + removed | 100 | 500 | 120 | 24.00% | Not comparable to new full basket |
| Retained slice in old window | Retained only | 80 | 400 | 96 | 24.00% | Comparable bridge slice if run rules also match |
| Removed slice in old window | Removed only | 20 | 100 | 24 | 24.00% | Explains what left the denominator |
| Retained slice in new window | Retained only | 80 | 400 | 104 | 26.00% | Comparable bridge slice if run rules also match |
| Added slice in new window | Added only | 30 | 150 | 45 | 30.00% | Explains what entered the denominator |
New full basket v2026_09 |
Retained + added | 110 | 550 | 149 | 27.09% | Not comparable to old full basket |
The full-basket rate appears to rise from 24.00% to 27.09%. That comparison is not a valid trend because the prompt cohort changed from 100 prompts to 110 prompts and because 30 added prompts entered the denominator while 20 removed prompts left it.
The retained-slice comparison is narrower and more defensible: 96 cited runs out of 400 becomes 104 cited runs out of 400, or 24.00% to 26.00%, if engine panel, dates, locale, collection policy, and exclusion rules also match. Even then, the report should call it a retained-cohort movement, not a global visibility improvement.
The bridge table prevents a quiet denominator swap from becoming a causal story.
Failed and unknown observations should not be hidden inside the denominator
An answer run can be planned, attempted, observed, failed, or unknown. Those states should not collapse into one denominator.
A clean report separates at least four counts:
| Run state | Definition | Denominator treatment |
|---|---|---|
| Planned run | A prompt-engine-date observation scheduled for collection | Shows intended coverage, not a citation-rate denominator |
| Observed answer run | A completed answer where citation extraction could be evaluated | Eligible denominator for citation rate |
| Failed run | Provider error, timeout, blocked retrieval, parser failure, or invalid payload | Excluded from citation rate and reported separately |
| Unknown run | Record exists but citation evaluation cannot be trusted | Excluded or quarantined until resolved |
This distinction matters because a failed observation is not evidence of no citation. It is evidence that the system did not collect a valid observation. Treating failed runs as uncited runs lowers a citation rate without measuring source selection. Dropping failed runs without disclosing them hides coverage loss.
The public Machine Relations Index uses the same evidence-floor principle at the source-segment level. Its September 12, 2026 release reports 15,154 answer runs, 863 monitored prompts, six answer engines, and 79 validated source segments, and it publishes citation rates only after a segment clears minimum observation and date thresholds. That public release is the current citation authority to use here. The September 13 collection did not pass the Google AI Mode provider-loss threshold, so partial September 13 rows should not be presented as a fresh release.
A prompt-cohort migration spec for AI citation reports
A practical migration spec can be implemented with five rules.
- Create a new cohort version for every material prompt-basket change. Adding, removing, rewriting, or resegmenting prompts should create a new cohort identifier.
- Keep stable prompt identifiers for retained questions. A retained prompt must be matchable across versions; a rewritten prompt should be marked as rewritten unless the wording change is immaterial under a documented rule.
- Publish a bridge table beside the first new-cohort report. Readers should see retained, removed, and added slices before they see a trend chart.
- Label noncomparable full-basket rates. Old full basket and new full basket can both be true, but they should not be connected by a trend arrow unless the report explicitly models the cohort change.
- Report planned, observed, failed, and unknown runs separately. Citation rate should use observed answer runs as its denominator; collection health should use planned and attempted runs.
The Generative Engine Optimization paper introduced visibility metrics for generative-engine responses and showed why AI search measurement needs its own evaluation frame rather than a rank-only SEO frame (GEO: Generative Engine Optimization). Prompt-cohort versioning is one of the practical controls that makes that evaluation frame auditable over time.
Machine Relations measurement depends on cohort identity
Machine Relations treats AI visibility as a source-system problem: prompts, engines, citations, source identity, and answer evidence all have to be preserved well enough for a reader to reproduce the reported claim. Prompt-cohort migration is the denominator version of the same problem that URL-resolution accounting handles for source identity.
The already-published Paralax definition of AI citation rate explains the base unit: cited answer runs divided by observed answer runs inside a defined measurement set. The migration standard adds one sentence to that definition: when the measurement set changes, the old full-basket rate and new full-basket rate become two different series until a retained-cohort bridge is shown.
That does not mean teams can never update their prompt baskets. They should update them when the market changes. The point is to carry the old instrument forward long enough to explain the break.
FAQ
Can an AI citation rate improve after a prompt cohort migration?
Yes, but the report has to show which part improved. A retained-cohort slice can indicate movement inside stable prompts when the run rules also match. A new full-basket rate cannot, by itself, prove improvement because additions and removals changed the denominator.
Should failed AI answer runs count as uncited runs?
No. A failed run is not an observed uncited answer. It should be reported as collection loss or unknown evidence, then excluded from the citation-rate denominator unless the methodology explicitly defines a different, conservative metric.
What is the safest first chart after a prompt basket changes?
Use three lines or three rows: old full basket, retained cohort, and new full basket. Only the retained cohort should be treated as the comparable bridge. The full-basket rates are useful context, not a clean time trend.
How long should the old prompt cohort stay alive?
Keep the retained bridge alive for at least one reporting cycle long enough to compare old and new windows under the same engine, locale, date, and exclusion rules. High-stakes reports should run the bridge longer, especially when prompts were added because buyer behavior changed.