Citation Retry Accounting Keeps Duplicate URLs Out of AI Visibility Rates
AI citation-rate reports should separate scheduled observations, retry attempts, successful answers and repeated URLs before publishing any denominator.
AI citation monitoring needs one accounting rule before the rate is trusted: count answer observations, not citation links, and record retry attempts outside the numerator.
That sounds small until an engine times out, a collector retries, and the final answer cites the same URL twice. If the report counts every attempt as a denominator row, the rate can fall because collection was noisy. If it counts every repeated link as a numerator event, the rate can rise because one answer listed the same source twice. Neither outcome measures whether the source was selected by the answer engine.
The safer metric is explicit: a scheduled observation is the plan, a retry attempt is collection telemetry, a successful answer is the denominator candidate, a repeated URL is evidence detail, and a domain-cited boolean is the numerator unit for a domain-level citation rate.
The five records that should not collapse into one row
A citation monitor often starts with one prompt, one engine, one locale, and one scheduled collection time. The temptation is to store one row and keep updating it until something works. That is convenient for operations and dangerous for measurement.
The report needs separate identities for at least five things.
| Unit | Plain definition | Metric treatment |
|---|---|---|
| Scheduled observation | The planned prompt-engine-locale-date collection | Coverage plan, not a citation-rate denominator |
| Retry attempt | A collector call made to complete the scheduled observation | Operational telemetry, not a new answer run |
| Successful answer | A completed response whose citations can be evaluated | Eligible denominator for citation rate |
| Repeated URL | The same normalized URL appearing more than once inside one answer | Evidence detail; deduplicate before URL-level incidence unless the metric says otherwise |
| Domain-cited boolean | Whether the target domain appeared at least once in that successful answer | Domain-level numerator for citation rate |
This keeps the formula stable. For a domain citation rate, the numerator is successful answers where the domain was cited at least once. The denominator is successful answers eligible for the metric. Retry attempts and duplicate in-answer links remain visible, but they do not become extra answers.
Google Search Console offers a useful caution from a different measurement system: impressions, clicks, and position depend on the reporting context and aggregation unit (Google Search Console Help). AI citation monitoring is not Search Console, but the same principle applies. A number changes meaning when the unit of aggregation changes.
A fixed prompt and URL identity ledger
The ledger below is illustrative. It is not a live Machine Relations Index defect report and it does not claim that a current collection duplicated answers. It shows how the accounting should work when prompt wording and URL identity are fixed.
Assume the monitored question is fixed as prompt_id=geo_source_014, the engine is fixed as engine=example_ai, the locale is fixed as en-US, and the source identity rule normalizes https://www.example.com/guide?utm=chat and https://www.example.com/guide to the same URL key after tracking-parameter removal. Google's canonicalization documentation is about Google Search indexing rather than AI monitoring, but it illustrates why duplicate or alternate URLs need a declared identity rule before reports are compared (Google Search Central).
Before accounting is fixed
| Scheduled observation | Collector attempts | Successful answers | Citation rows for example.com | Reported treatment | Resulting mistake |
|---|---|---|---|---|---|
| Sept. 1 | 1 | 1 | 1 | Count one answer, one cited domain | None |
| Sept. 2 | 2 | 1 | 1 | Count two attempts as two denominator rows | Retry makes visibility look lower |
| Sept. 3 | 1 | 1 | 2 repeated URLs | Count two citation rows as two numerator hits | Duplicate URL makes visibility look higher |
| Sept. 4 | 1 | 0 | 0 | Count failed answer as uncited | Collection loss becomes source absence |
A naive report might say there were five denominator rows and four citation rows, producing an apparent 80.00% rate. That is not a valid domain citation rate. It mixed attempts, successful answers, duplicate URL rows, and failure states.
After accounting is fixed
| Scheduled observation | Attempts logged | Successful answer eligible? | Repeated URL count | Domain-cited boolean | Citation-rate row |
|---|---|---|---|---|---|
| Sept. 1 | 1 | Yes | 1 | true | 1 cited answer / 1 eligible answer |
| Sept. 2 | 2 | Yes | 1 | true | 1 cited answer / 1 eligible answer |
| Sept. 3 | 1 | Yes | 2 | true | 1 cited answer / 1 eligible answer |
| Sept. 4 | 1 | No | 0 | not evaluated | Excluded and reported as collection loss |
The corrected domain citation rate is three cited successful answers out of three eligible successful answers, or 100.00%, with one failed scheduled observation disclosed separately. A URL-incidence report could still show that Sept. 3 contained two repeated URL rows, but it would label that as in-answer multiplicity rather than a second successful answer.
The before/after does not prove performance improved. It proves the metric stopped counting unlike things as if they were the same thing.
Retry attempts are collection health, not source selection
Retries matter. A high retry rate can reveal provider instability, blocking, latency, parser fragility, or bad scheduling. The mistake is treating those retries as additional answers.
If a scheduled observation requires three collector calls and only the third call produces a valid answer, the citation-rate denominator should receive one successful answer, not three attempts. The report can still publish attempt health beside it:
| Health field | Example value | Why readers need it |
|---|---|---|
| scheduled_observations | 100 | The planned coverage |
| total_attempts | 118 | Operational load and retry pressure |
| successful_answers | 96 | Citation-rate denominator candidate |
| failed_scheduled_observations | 4 | Collection loss that should not become uncited evidence |
| retry_rate | 18.00% | Reliability context, not visibility performance |
This prevents a monitoring dashboard from punishing a source because the collection system had to retry. It also prevents teams from hiding weak collection coverage by only showing the successful rows.
Repeated URLs are evidence detail, not always numerator events
Repeated links are different from retries. A retry is outside the answer. A repeated URL is inside one successful answer.
For a domain-level citation rate, repeated URL rows should usually collapse to a single domain-cited boolean for that answer. The question is: did the answer cite the domain at least once? If yes, the numerator gets one cited answer.
For a URL-frequency report, repeated URLs may be worth retaining. They can show answer formatting, citation-list behavior, or source emphasis. But that is a different metric. The report should label it as URL citation events, not answer-run incidence.
The public Machine Relations Index shows why this separation matters. Its September 14, 2026 public release describes 15,396 answer runs and 121,750 citation events across the window from May 10 to September 14. Those units are not interchangeable: one answer run can contain several citation events, and a source-domain rate is calculated against answer runs, not against raw link rows.
That released example supports the accounting rule. It does not prove a live duplication defect, a specific buyer problem, or a vendor failure.
A practical schema for citation retry accounting
A minimal implementation can keep the metric auditable with four tables or event streams.
| Record | Required identity fields | Required status fields |
|---|---|---|
| Observation schedule | observation_id, prompt_id, prompt text hash, engine, locale, scheduled date |
planned, canceled, superseded |
| Collection attempt | attempt_id, observation_id, collector version, attempt timestamp |
success, timeout, provider error, parser error, blocked |
| Answer observation | answer_id, observation_id, winning attempt_id, answer timestamp |
valid, quarantined, failed, unknown |
| Citation evidence | citation_id, answer_id, raw URL, normalized URL key, root domain |
extracted, normalized, duplicate-in-answer, excluded |
The report then computes:
- Coverage rate: successful or valid answer observations divided by scheduled observations.
- Retry rate: attempts beyond the first divided by scheduled observations or total attempts, with the chosen formula stated.
- Domain citation rate: successful answer observations with
domain_cited=truedivided by successful answer observations eligible for the metric. - URL event count: normalized citation rows, optionally including duplicate-in-answer flags.
- Exclusion ledger: failed, quarantined, unknown, and duplicate evidence rows disclosed outside the main rate.
NIST's traceability guidance is written for measurement standards, not AI citation dashboards, but the documentation lesson transfers: a reported result should be connected to the evidence and rules that produced it (NIST metrological traceability). In citation monitoring, that chain runs from a published percentage back to prompt identity, engine, locale, scheduled observation, successful answer, URL normalization rule, and citation-evidence rows.
What the dashboard should say out loud
A trustworthy citation-rate dashboard should make these labels visible rather than burying them in pipeline code.
Use language like this:
- "Domain citation rate: 42 successful answers cited the domain at least once out of 120 successful answer observations."
- "Collection coverage: 126 observations were scheduled; 120 produced successful answers; 6 failed or remain unknown."
- "Retry health: 19 additional attempts were required; retries did not add denominator rows."
- "URL evidence: 57 normalized URL citation events were extracted; 8 were repeated within the same answer."
- "Trend status: prompt text, engine panel, locale, URL identity rule, and denominator policy were unchanged."
That is less dramatic than a single visibility percentage. It is also much harder to misread.
Machine Relations measurement depends on unit discipline
Machine Relations treats AI visibility as a relationship between prompts, engines, sources, answer evidence, and the public web. The measurement only works when each unit keeps its identity.
The existing Paralax definition of AI citation rate explains the base ratio: cited answer runs divided by observed answer runs inside a defined measurement set. The September 13 bridge article explains what happens when the prompt cohort changes. This specification covers a different boundary: what happens when the prompt and URL identity stay fixed, but collection retries and repeated in-answer URLs could distort the accounting.
The answer is to keep three ledgers side by side: observation health, answer-run incidence, and citation-evidence detail. Once those ledgers are separate, retries can be investigated, duplicate URLs can be audited, and the citation rate can keep meaning what it claims to mean.
FAQ
Should a retry count as a new AI answer run?
No. A retry attempt is collection telemetry. If several attempts serve one scheduled observation and one valid answer is accepted, the citation-rate denominator receives one successful answer observation.
Should the same URL twice in one answer count twice?
Not for a domain-level citation rate. It should usually count as one domain-cited answer, with the repeated URL retained in the citation-evidence ledger.
Should a failed collection count as an uncited answer?
No. A failed collection is not a valid uncited answer. Report it as failed or unknown coverage, then exclude it from the main citation-rate denominator unless a separately labeled conservative metric intentionally does otherwise.
Does this mean the Machine Relations Index has a duplicate-answer defect?
No. The examples here are illustrative accounting cases. The September 14 public MRI release is used only to show why answer runs and citation events are different units.