Does ChatGPT know you exist? AI visibility as an ops problem.

Something quietly moved in the funnel: people who used to search “best tool for X” now ask the assistant, and the assistant names three brands and links two. If yours is not among them, the pipeline will tell you eventually — usually after the quarter closes.

The industry calls this AI visibility, or GEO, or AEO, depending on the invoice. The useful reframe is duller and more actionable: it is a monitoring problem, same shape as uptime.

The ops framing

• A fixed prompt set is your synthetic checks: thirty buyer-intent questions your customers actually ask, versioned in git.
• Scheduled sampling is the cron: run the set weekly against a couple of models, record answers and citations.
• Assertions are the scorecard: is the brand mentioned, which competitors appear alongside, is the sentiment warm, is anything linked.
• Alerting is the point: share of voice drops week over week → a human looks.

“Crude but honest” is the right ambition, because the answers are stochastic — same question, different day, different list. You sample; you do not assert. The sample size is the honesty.

The DIY v1

A cron job, an LLM API and a JSON file will get you a working instrument in an afternoon:

prompts.txt    # 30 buyer-intent questions, versioned
run.sh         # for each prompt: query 2 models, save answer + citations
score.py       # count mentions: you / competitors, per answer
weekly.sh      # cron: append to history.jsonl, diff vs last week,
               # notify on regression (ntfy / webhook)

If you would rather buy it: Profound, Peec AI and Otterly are among the many now selling exactly this — prompt sets, answer tracking, share-of-voice dashboards, the citation sources behind each answer. Same trade as any monitoring product: you pay to not own the sampling rig.

What moves the number

• Docs. For dev tools, AI answers cite documentation more than anything else. Thin docs make you invisible regardless of blog output.
• The places answers read: Reddit threads, GitHub, comparison pages, Wikipedia. Being discussed beats being advertised; the models mostly repeat what the discussable web already says.
• llms.txt exists and the major engines mostly ignore it so far. Nice to have; not a lever.
• Blocking the crawlers (GPTBot, ClaudeBot, PerplexityBot) in robots.txt is a visibility decision. Some teams make it on principle. Make it with eyes open.

The honest trade-offs

• Your prompt set is your bias. You wrote the questions, so you can accidentally write the answers; keep the set in the customers’ words, not yours.
• Vendor definitions of “share of voice” are not standardized. Compare a tool’s numbers to its own history, not to another tool’s.
• Attribution is mush. Visibility → traffic → pipeline is a chain of estimates; treat the number as a trend, not a truth.
• Gaming it is a treadmill. Manufactured Reddit praise gets detected — by users first, models second.

Where I landed

The v1 that stuck: twenty-five prompts, two models, weekly cron, and a number on the dashboard next to uptime. It turned “does the AI know us” from vibes into a trend line — and the first time the line dipped, the cause was a docs page that had 404’d for a month. That one find is the whole argument for treating this as ops.