Branded vs Unbranded AI Prompts: What Each One Measures

Branded And Unbranded Prompts Ask Different Questions

A branded prompt includes the brand name: “What is Acme Payroll best suited for?” An unbranded prompt does not: “Which payroll tools suit a 30-person UK agency?” The branded version measures recognition, factual understanding, positioning, and reputation after a direct cue. The unbranded version measures category discovery under the constraints in the question.

A brand that appears in every branded answer has not achieved a 100 percent discovery rate; the prompt handed the model the name. Conversely, absence from a broad unbranded list does not prove the system has no knowledge of the brand. It may reflect the prompt's geography, use case, available sources, answer length, or ordinary generation variation.

Use Five Prompt Families, Not One Mixed Bucket

The branded-versus-unbranded split is a starting point. A useful research set separates five prompt families because each reveals a different failure mode.

• Branded factual: “What does Acme Payroll do, and which countries does it support?” Tests accuracy and entity understanding.
• Branded fit: “Is Acme Payroll suitable for an agency with UK employees and EU contractors?” Tests how the brand is matched to a buyer context.
• Named comparison: “Compare Acme Payroll and Northwind Pay for a 30-person agency.” Tests distinctions, tradeoffs, and comparative accuracy after both brands are supplied.
• Unbranded category: “Which payroll platforms should a 30-person UK agency evaluate?” Tests spontaneous shortlist inclusion.
• Unbranded problem: “How should a small agency manage payroll for UK staff and EU contractors?” Tests whether the answer recommends a product category, a service, a process, or no vendor at all.

What Each Family Can And Cannot Tell You

Branded factual prompts are the best place to find outdated descriptions, invented features, and positioning drift. They are weak evidence of category prominence because the brand was supplied. Branded fit prompts reveal whether the system connects the product to the customers you intend to serve, but they can overstate salience because they force the assistant to evaluate that product.

Unbranded prompts are better for observing shortlist inclusion, but they are highly sensitive to constraints. A generic “best CRM” prompt mixes enterprises, freelancers, regulated industries, and local businesses into one artificial market. Adding a real role, company size, location, job, and one or two decisive requirements produces a more interpretable test. Do not add a stack of brand-exclusive features solely to engineer an appearance.

Build The Set From Real Buyer Decisions

Start with evidence about how buyers frame the problem: sales-call notes, support tickets, on-site search, win-loss interviews, community questions, and search-query data. Turn recurring decisions into prompt cells rather than generating dozens of keyword synonyms. The objective is coverage of meaningful situations, not the largest possible prompt count.

A compact matrix can cross four dimensions: task, buyer, context, and constraint. For example, the task may be learn, compare, shortlist, or choose; the buyer may be an owner, practitioner, or procurement lead; context may include company size and country; and the constraint may be budget, integration, regulation, or speed. Select the combinations that occur in the real buying journey.

• Keep a balanced mix of factual, fit, comparison, category, and problem prompts.
• Include important exclusion cases, such as a region or customer type the product does not serve.
• Write prompts in natural buyer language instead of keyword fragments.
• Assign each prompt a purpose and an expected evaluation field before running it.
• Version the set; adding or deleting prompts creates a new baseline.

An Illustrative Eight-Prompt Starter Set

For a fictional payroll platform called Acme Payroll, a small research set might contain the following prompts. The exact balance is illustrative, not an industry benchmark; your evidence should determine the final mix.

• Branded factual: “What does Acme Payroll do?”
• Branded accuracy: “Which countries and worker types does Acme Payroll support?”
• Branded fit: “What kinds of companies are a poor fit for Acme Payroll?”
• Named comparison: “Acme Payroll versus Northwind Pay for a UK creative agency with 25 employees.”
• Unbranded category: “Payroll software options for a UK creative agency with 25 employees.”
• Unbranded constrained: “Which payroll tools handle UK employees and payments to EU contractors?”
• Unbranded problem: “How can an agency reduce manual payroll work without outsourcing the whole process?”
• Decision prompt: “For a UK creative agency with 25 employees that needs payroll for UK employees and payments to EU contractors, create a shortlist of payroll tools and state what I would need to verify before choosing.”

Run The Prompts Like An Evaluation

Do not ask the unbranded question after the branded one in the same conversation. The earlier name becomes context and contaminates the discovery test. Use a new chat for every prompt, document memory or personalization state, and keep search mode, locale, language, and visible product version stable. OpenAI documents that memory can inform responses and search rewrites; Anthropic documents account and project preferences that can guide Claude.

Repeat runs because generated outputs can vary. Grade them with a rubric written before seeing the answers. Anthropic's evaluation guidance emphasizes specific, measurable, task-relevant success criteria; the same discipline applies here even though the subject is brand research rather than an application test.

• For branded factual prompts: score each material claim as correct, incorrect, outdated, unsupported, or unclear.
• For branded fit prompts: record matched use cases, stated exclusions, and whether the rationale reflects the real product.
• For unbranded prompts: record inclusion, answer position only when ordered, and the reason the brand was selected or omitted when stated.
• For all prompts: capture caveats, sentiment, visible citations, full answer text, timestamp, and settings.

Do Not Collapse The Results Into One Visibility Score

Prompted accuracy, comparison presence, and spontaneous inclusion have different denominators. Report each family separately. A useful summary might say that the brand appeared in two of ten unbranded shortlist runs, was described accurately in eight of ten branded factual runs, and was positioned for the wrong customer segment in half of the fit runs. That is more actionable than a blended score of 43.

Prompt tracking is also not demand research. It does not tell you how many people ask these questions, whether real users phrase them this way, or whether an answer changes buying behavior. Validate the prompt set with customer evidence and interpret movement only within a stable protocol. The result is a repeatable observational study, not a census of AI users.