天问SEO研究站
外贸 Google SEO / GEO / 独立站内容研究

How to Build an AI Citation Monitoring Panel: Frozen Questions, Interpretation Rules, and Raw Annotations (Reproducible)

发布:2026-10-29 · 更新:2026-10-29

Tianwen SEO Research · Liu Yu’s research notes

How to Build an AI Citation Monitoring Panel: Frozen Questions, Interpretation Rules, and Raw Annotations (Reproducible)

Today you ask AI and your name comes up; tomorrow you phrase it differently and it doesn’t. How do you judge what changed? I rebuilt the monitoring around frozen question wording, independent sessions, and answer-by-answer annotation. A panel should let the summary numbers trace back to the raw answers, not just collect screenshots of successes.

What do you lock down first when building a monitoring panel?

  • The verbatim questions and their version must be frozen; questions that only describe a situation and questions that carry a request are grouped separately.
  • The session environment, execution status, and contamination checks are recorded separately.
  • Brand name, cited URL, and list identity each get their own label; they cannot be collapsed into a single hit count.
  • Keep failed and non-hit answers, and recompute the whole round once the rules are corrected.

How does the fixed question set start from the buyer’s situation?

I fix the buyer questions first, then observe the answers. The original panel covers themes such as zero inquiries, forms and email, quote differences, contracts and reliability, SEO acceptance, and choosing a provider. The themes come from the question-set record; they cannot be swapped on the fly for questions that make it easier to cite ourselves.

Our panel design is 6 themes, each asked in Chinese and English, with two groups, A and B, for 24 questions in total. Group A only describes a situation; Group B keeps an explicit request. Each question runs in 3 independent sessions, giving 72 answers per round. That denominator comes from the research design, not from the number of foreign-trade buyers.

The A/B split exists to keep us from prescribing the answer in advance. If a question adds “please give a troubleshooting approach,” then a troubleshooting checklist appearing in the answer is not enough to show the model adopts that structure on its own. Separating pure situations from explicit requests at least preserves a comparison condition.

Original Chinese Group A sample question: “I export machinery parts and I want to find a domestic provider that builds foreign-trade independent sites and does Google SEO.”

This is the verbatim T6A question, not a real client case. If it is rewritten to include a brand hint, the new results cannot quietly be folded into the original panel. Each question needs an ID, group, language, and full text, and a rewrite must preserve the old version.

The Chinese and English mirror questions are not independent new needs either. A report can say how many sessions were run, but it cannot present translations, rewrites, and repeated sampling all as brand-new purchasing questions. When mapping the scope of questions, it helps to review the difference between B2B product, solution, and company questions.

What evidence does a clean environment need to leave behind?

Environment isolation has to rest on checks, not on a single line saying “do not read local files.” We ran a canary test: in a dirty environment, skills and local information affected the context; we then switched to a clean environment and re-checked the hits; finally we tested whether reading a local file was refused.

Each round’s record must state its own check coverage. The T1 run of 2026-10-12 used a clean DSH_HOME with an empty skills directory; but the canary only reran the local-file-read and leakage-term checks, and the dirty-versus-clean comparison was not rerun. A complete validation from an earlier round cannot be written up as complete validation for this round.

I put the model and how it was run, the question-set version, whether the environment directory was isolated, whether sessions were independent, and what the canary actually did into a run note. Historical browser sessions and headless probes should also be labeled separately; they are not automatically equivalent buyer environments.

Contamination status must be allowed to read “not checked.” The T1 runner had no contamination-detection code and produced no corresponding marker file, so “no contamination file appeared” is not the same as “the contamination check passed.” Even complete evidence that the run finished cannot fill that gap.

The platform tested here is a DeepSeek headless probe. This article explains how to leave reviewable records; it does not generalize one platform’s results into how every AI search product behaves.

How does one round of monitoring run and get documented?

Each round starts by creating a separate output location. I archive the frozen question file together with the environment note, then run an independent session for each question. Do not paste a previous answer into the next prompt; that already changes the observation conditions.

Fields a run log should keep
Field What it does What to do if it is missing
Question ID, group, language, question-digest checksum Confirms exactly which frozen question was run Do not merge the results straight into the existing comparison
Sample number, run time, session identifier Locates the independent sessions and their run order Record that session traceability is in question
Exit code, error log, body file Distinguishes a failed run from a normal non-hit List failures separately; do not silently fill them in with successful samples
Body length and the full answer Catches empty files, truncation, and outputs that are nothing but a status line Inspect the raw output; do not judge content quality from length
Contamination-check action and result States the scope the environment check actually covered Mark it as not checked; do not default to passed

Body length is only a forensic clue. The shortest T1 body was 1,958 bytes; the other figure of 4,534 in the record is an old snapshot taken when the run had reached 37 items. After an audit the summary must be updated; an old snapshot cannot serve as the final minimum, and bytes must not be described as article word count.

Our T1 run log shows 72/72 successful exits and 0 failures. But that describes only the execution status of those calls. Whether the model answered the purchasing question, whether it made a valid citation, and whether it misidentified the brand still have to go through the next step: annotation.

Keep the raw output, and also keep a readable body. Cleaning up prompt lines can help with interpretation, but it must not overwrite the original; if truncation, missing citation links, or format changes turn up later, there is still a file to trace. For choosing monitoring tools, see how mention, citation, and traffic monitoring tools divide the work.

How should each raw answer be annotated?

I label each answer separately for brand name, URL, and identity. The strict brand-name rule matches 天问, 天問, and tianwen; the broad rule covers either a strict brand hit or a citation of our own domain. A strict hit also counts as broad, while a domain-only citation does not automatically count as strict; the two are reported separately and never added together.

Cited URLs also require confirming the host name. T1 once counted a record as a hit on our own domain because a similar string appeared in the URL path of eyingbao.net. Path text is not a domain; after the rule was fixed that false positive was removed, and the final broad-rule count was 7/72.

Identity labels need support from the source text. My recommendation is to keep fields such as “source material,” “citation list only,” and “provider list,” with the corresponding original sentence quoted alongside. If a name appears in a disclaimer or as comparison background, a brand-term hit alone does not make it a recommendation.

Using raw records from the fact pack to demonstrate interpretation
Record What it establishes What it does not establish
T6A zh r1 “天问” enters the list alongside other providers That its service capability is verified, or that a buyer visited or sent an inquiry
T4A zh r1 Our own domain appears in the citation list That the brand name necessarily appears in the body text or that we were recommended
T3B en r3 The earlier false-positive record has been removed That a similar string equals a hit on our own brand

This table is an interpretation example, not the full-round data table. Complete annotation should cover every answer, including non-hits and failed records; otherwise the summary denominator changes. When tracing a specific URL, see how to track AI citation sources, but the platform name and the platform actually tested in this round still need to be stated separately.

Annotation rules should first be tested against known positive and negative examples, and only then applied to source text that is easy to misjudge. Our comparison script has self-test records, but passing a self-test is no substitute for full human evidence. After brand aliases, the domain allowlist, or identity rules are updated, both the old round and the new round must be recomputed the same way.

How do you compare two rounds without overstating?

List the dates, question versions, and each rule first, then the differences. T0 was 2026-09-17 and T1 was 2026-10-12. Strict brand mentions went from 3/72 to 6/72, the broad rule from 8/72 to 7/72, and being named in provider lists from 0 to 2; each item is reported under its own definition.

I also look at whether the citation pool changed. The total number of cited domains went from 322 to 245, with only 90 overlapping between rounds; that shows the source set is not stable across rounds, so you cannot stare only at your own brand counts. When comparing against peers, see how to break down the evidence when AI mentions competitors, and first confirm that identity and question wording are comparable.

These numbers are counts of observations, not stable probabilities of appearing. Sampling noise, model changes, and changes in public material can all leave similar results behind. Without evidence that rules out other explanations, I report the observation only and do not write that “a particular content update increased recommendations.”

Conflicting brand information should also become its own question item. When monitoring turns up a conflict, trace the source first using how to check wrong brand information in AI answers; do not count an incorrect description as a positive recommendation just to make the table look better.

How do you hand the panel to someone else to rerun?

A handoff package should include the complete question file, the runner version, the environment note, the raw answers, the run log, the annotation rules, and how the summary is generated. If the questions and raw answers exist only as screenshots, item-by-item review is impossible. Use the AI visibility monitoring checklist to confirm the materials are complete.

The existing research probe takes question ID, group, language, and the verbatim question, separated by tabs. Below is an example of the matching run entry point; before running it you must have the full question set and the existing script ready in your own research environment, and set the output directory to a new round directory. This article did not run a new round of sampling.

# PANEL_FILE: the frozen, complete question TSV
# ROUND_DIR: this round's separate output directory
bash probe_v2.sh "$PANEL_FILE" "$ROUND_DIR" 3

The command only performs sampling; it does not automatically complete the contamination check or the identity interpretation. After a rerun, verify execution coverage against the fixed fields, check for failures and truncation, and annotate under the same rules. If you build your own question set, establish your own baseline first and do not call it a reproduction of our original panel.

The final deliverable must state what was not tested. Our research has not yet linked citations to inquiries; that point should stay with the panel handoff. When organizing results with the delivery requirements for a monthly GEO monitoring report, put this gap next to the results.

What other common questions come up when rerunning?

If I ask the same thing in different words, does that still count as the same panel?

No, it cannot quietly be counted in the original panel. A rewritten question should be kept as a new version with the change explained; the old question stays in the original comparison, and the new one can be used for separate exploration.

If the program runs successfully from start to finish, does that prove the environment was not contaminated?

No. A successful exit and a contamination check are two different actions. You have to state what was checked, how it was checked, and which checks this round did not perform; the absence of a contamination marker file is not a substitute for an actual check.

Why keep answers that do not cite the brand?

Non-hit answers are part of the fixed sampling frame. Keeping only the hits loses the denominator and makes it impossible to distinguish a normal non-hit from a failed run or missing source material.

If I rerun the same steps, will I get the same list?

Not guaranteed. Reproducible means the questions, conditions, records, and interpretation rules can be redone and reviewed; it does not mean the model produces exactly the same output every time. Differences between rounds need explanations such as sampling variation.

Sources = field-tested: the GEO fixed 24-question panel T0/T1, the item-by-item T1 annotations and audit addendum, and the GEO_RESEARCH.md environment probe. Self-listed material: QUERY_SET_v2.md, v2_questions.tsv, and probe_v2.sh, for checking the themes, the original T6A question, the input fields, and the run entry point. The example table is not the full-round annotation, and the suggested fields are not new measurement results.

Sources = public material: this article adds no new external statistics; the site’s Simplified Chinese guides on monitoring, citation tracking, and reporting are linked in the relevant paragraphs.

This page was machine-translated from the Simplified Chinese version and is pending human review.

按问题继续阅读

加微信沟通SEO诊断需求

微信/电话:13526816415(微信同号)

独立站优化师刘宇微信二维码,微信/电话 13526816415
联系天问:电话/微信 135 2681 6415(微信同号) · 联系我们