AI Product vocabulary
Lead AI products. 28 hand-written terms with real workplace examples.
Learn ai product terms every day.
One hand-written term. A real workplace example. Thirty seconds before your day gets loud.
Capability
What a model can do reliably enough to build on, measured on your own tasks rather than a demo. Capabilities are uneven, so a model can excel at one task and stumble on a neighboring one.
Tool Use
A model asking your code to run a function, such as an order lookup or a refund, by emitting a structured call and then reading the result. The model chooses when to call; your code decides what each tool may do.
Computer Use
A model operating software the way a person does, by reading screenshots and sending clicks and keystrokes. It reaches apps that have no API, but it is slower and more brittle than a direct tool call.
Model Generation
One big step in a family of models, usually marked by a new major version number. Each one tends to shift what is possible, so work built around today's limits can expire within a generation or two.
Golden Set
A fixed, labeled set of real examples that every prompt, model or harness change is scored against. Version it and add cases on purpose, so this week's score compares honestly with last month's.
Rubric
The written criteria for a good output, broken into specific points a reviewer can check, each with a clear pass bar. A tight one lets two reviewers, or a model, grade the same answer the same way.
LLM-as-Judge
Using a model to grade outputs against a rubric so evals can score thousands of cases. Trust it only once its grades agree with human labels on a sample, and recheck whenever the judge model or the rubric changes.
Error Analysis
Reading failed outputs one by one and sorting them into named failure types before fixing anything. The counts, not the loudest anecdote, then decide which fix comes first.
Regression
A drop in quality on something that used to work, caused by a model, prompt or harness change. In AI products it can hide inside an average that went up, so compare scores by slice, not just overall.
Post-Training
Training after pretraining, such as tuning on examples and on human preferences, that shapes how a model follows instructions and behaves. At a model lab, product teams negotiate changes to it with researchers; it is not a setting to toggle.
Fine-Tuning
Further training of an existing model on your own examples so it adopts a format, style or narrow task. It ties you to that model version, and a newer base model may match it with a prompt, so test that first.
Preference Data
Pairs or rankings of model outputs where people marked the one they preferred. Post-training uses it to teach a model which response is better, so a skewed group of raters teaches a skewed taste.
Harness
The product code that runs a model on a task: the loop, tools, context, permissions and memory that turn single replies into finished work. Agentic loops work when a capable model meets a good harness; neither is enough alone.
Scaffolding
Structure added around a model to cover its current weaknesses, like forced step-by-step chains, extra parsers or rigid templates. It helps today's model and can hold back the next one, so plan its removal from day one.
Capability Overhang
The gap between what models can already do and what products expose to users. Closing it is often product work, such as a new interface, workflow or permission, rather than waiting for a better model.
Bitter Lesson
Rich Sutton's 2019 essay: general methods that scale with computation ultimately beat methods built on hand-coded human knowledge. For products, it warns that workarounds for today's model limits tend to get thrown away.
Key Assumption
A belief your plan depends on that would change the roadmap if it proved wrong, such as how capable the next model will be. Name a few for the next three to six months and recheck them every month.
Bet
A deliberate investment in an outcome you cannot prove yet, with a thesis, a cost and a signal that would make you stop. Keep a few, and stay stubborn about them while the plans around them change month to month.
Planning Horizon
How far ahead a plan commits in detail. Fast-moving AI products suit a short one, such as a month of committed work under three-to-six-month assumptions, since one model release can reset what is worth building.
Prototype
A quick working version built to answer one question, such as whether users trust an agent's edits, before anyone writes a spec. For AI features, several cheap ones often teach more than one polished document.
Trace
The step-by-step record of an agent's run: every model call, tool call, input and output, in order. Reading one shows where a run went wrong, which a final score alone cannot.
Vibe Check
Informal, hands-on trying of a model or a change with prompts you know well; daily dogfooding gives you one for free. It catches surprises fast, but it is no substitute for evals.
Sycophancy
A model telling users what they want to hear: agreeing with a wrong claim, praising weak work or dropping a correct answer under pushback. It can be learned from preference data that rewards agreeable answers.
Model Migration
Moving a product or its customers from one model version to a newer one. It means rerunning evals and retuning prompts, and regressions or the old model's retirement date can block or force the move.
Inference Cost
What each model call costs to run, driven mostly by the model chosen and the tokens in and out. Agents multiply it, since one task can trigger dozens of calls.
Metering
Measuring each customer's usage, such as tokens, minutes or completed actions, so you can bill for it, cap it or warn before a limit. The unit you meter shapes behavior, so pick one customers can predict.
Entitlement
The record of what a plan or customer may use: which features, which models and how much of each. The product checks it before a request runs, which is how tiers and limits are enforced.
Gross Margin
Revenue minus the direct cost of delivering it, usually shown as a share of revenue. For AI features, inference is part of that cost, so heavy use on a flat plan can push it toward zero or below.
Building toward a role? Browse career pathways that map terms to jobs.