Somebody in a meeting says “RAG pipeline” and you nod like you know what that means. You don’t. Nobody judges you for it, because half the people saying these terms don’t fully understand them either.
AI has a jargon problem. Vendors throw around technical terms because it makes their product sound more sophisticated. Articles use acronyms without explaining them. And somewhere between “machine learning” and “agentic orchestration layer,” you lost the thread and stopped trying to follow along.
This glossary is for you. We’ve split it into the 30 terms that actually show up in business conversations, vendor pitches, and buying decisions, and the 20 that you can safely ignore unless you’re building AI systems yourself.
The 30 terms you need to know
The basics
Artificial intelligence (AI) is software that can perform tasks that used to require human thinking: reading text, recognizing patterns, making decisions, generating content. In practice, when someone says “AI” in 2026, they almost always mean one specific type: large language models. More on that next.
Large language model (LLM) is the technology behind ChatGPT, Claude, Gemini, and most AI tools you’ve encountered. It’s software trained on enormous amounts of text that can understand and generate human language. Think of it as a very sophisticated autocomplete that has read most of the internet. It predicts what words should come next, and it’s gotten remarkably good at it.
Machine learning (ML) is the broader category that includes LLMs. It’s any software that improves at a task by learning from data rather than being explicitly programmed with rules. Your email spam filter uses machine learning. So does Netflix’s recommendation engine. LLMs are machine learning, but not all machine learning is an LLM.
Generative AI (GenAI) is AI that creates new content: text, images, code, audio, video. ChatGPT generating an email is generative AI. A spam filter categorizing your inbox is not. When people say “AI” in a business context, they usually mean generative AI specifically.
Model is the trained software that does the actual AI work. GPT-4o is a model. Claude Sonnet is a model. When vendors talk about which “model” they use, they’re talking about this. Different models have different strengths: some are faster, some are smarter, some are cheaper to run.
How you interact with AI
Prompt is what you type (or say) to an AI tool. “Summarize this contract” is a prompt. “Write a follow-up email to a customer who complained about shipping” is a prompt. The quality of what you get back depends heavily on how you write the prompt. This matters more than most people realize.
Prompt engineering is the skill of writing prompts that get consistently good results. It’s less like programming and more like writing a really good brief for a freelancer. The clearer and more specific your instructions, the better the output. It’s a learnable skill, not a dark art.
Context window is how much text an AI model can “see” at once. It’s measured in tokens (roughly words). A small context window means the model can only work with short texts. A large one (Claude’s 200,000 tokens, for instance) means you can feed it a 300-page document. This matters when you want to ask questions about your own data.
Token is the unit AI models use to measure text. Roughly, one token equals about three-quarters of a word. A 1,000-word document is about 1,300 tokens. You’ll see tokens mentioned in pricing (you pay per token processed) and in context window sizes. When a vendor says their model supports “128K tokens,” that’s the maximum amount of text it can consider at once.
System prompt is a hidden set of instructions that shapes how an AI tool behaves. When a company builds a customer service chatbot, the system prompt tells it things like “you are a helpful customer service agent for Acme Corp, you should be polite, and you should never discuss competitor products.” The end user doesn’t see this, but it controls the conversation.
Temperature is a setting that controls how creative or predictable an AI’s responses are. Low temperature: more predictable, more factual, less variation. High temperature: more creative, more varied, more likely to go off-script. For business use, you generally want lower temperature. For brainstorming, higher can be useful.
What businesses are actually building
AI agent is an AI system that can take actions on your behalf, not just generate text. A chatbot answers questions. An agent can read an email, look up the customer in your CRM, draft a response, and send it. The key distinction: agents do things in your business tools. We wrote a full article on this one: when you need an agent and when you don’t.
Chatbot is an AI that has a conversation with users, typically through text. Customer service bots, website chat widgets, internal Q&A tools. Chatbots answer questions. They don’t (usually) take actions in other systems. Many things called “AI agents” are actually chatbots with a fancier name.
RAG (retrieval-augmented generation) is a method for making AI work with your company’s specific information. Instead of relying only on what the model learned during training, RAG lets it search through your documents, knowledge base, or data and include relevant information in its responses. This is how you get an AI that can answer questions about your products, policies, or internal processes.
Think of it this way: the AI’s training is like a general education. RAG is like giving it access to your company’s filing cabinet before it answers a question.
Fine-tuning is modifying an AI model with your own data so it behaves differently by default. Unlike RAG (where the model searches your data at question time), fine-tuning actually changes the model itself. It’s more expensive, more complex, and usually unnecessary for most business use cases. For most SMBs, RAG is what you actually want. Fine-tuning is rarely worth the cost.
Knowledge base is a collection of documents, FAQs, policies, and information that an AI system can search through. If you want an AI to answer questions about your business, it needs a knowledge base to pull from. The quality of that knowledge base determines the quality of the answers.
Workflow automation is using AI to handle multi-step business processes. “When a new lead comes in, research their company, score them, draft a personalized email, and add them to the right pipeline.” That’s workflow automation. It’s where AI starts saving serious time.
Integration / API is how AI tools connect to your existing software. An API (application programming interface) is the technical mechanism. When someone says “we’ll integrate AI with your CRM,” they mean they’ll connect the AI tool to your CRM through its API so they can share data. Most modern business software has APIs, which is what makes this possible.
The risks and limitations you should understand
Hallucination is when an AI confidently generates information that is completely wrong. It doesn’t “know” it’s wrong. It’s not lying. It’s doing what it always does (predicting likely next words) and sometimes that produces plausible-sounding nonsense. This is the single biggest risk of using AI in business. Always verify important AI-generated information against a reliable source.
Bias in AI means the model reflects patterns and prejudices present in its training data. If the training data contains biased language about certain groups, the model may reproduce those biases. This matters for hiring tools, customer communications, and any AI that makes decisions about people.
Guardrails are rules and restrictions built into AI systems to prevent bad behavior. Content filters that stop the AI from generating harmful content. Rules that force it to escalate certain questions to a human. Output checks that flag potential hallucinations. Good guardrails are the difference between a useful business tool and a liability.
Human in the loop means a human reviews and approves AI outputs before they go anywhere that matters. An AI drafts a customer email; a human reviews it before sending. An AI suggests a contract clause; a lawyer approves it. For anything with real consequences, this is non-negotiable. At least for now.
Data privacy / data handling is what happens to your data when you use an AI tool. Does the vendor store your inputs? Use them to train their models? Keep them after you cancel? These questions matter especially if you handle customer data, health information, financial records, or anything regulated. Read the terms of service. Ask the vendor directly.
The business and buying terms
SaaS AI is AI delivered as a cloud subscription service. ChatGPT, Claude, most tools on the market. You pay monthly, you access it through a browser or app, your data goes to their servers. Convenient, but raises data privacy questions for some businesses.
On-premise / self-hosted AI means running AI models on your own servers instead of a vendor’s cloud. More control over your data, but more expensive and more complicated to maintain. Most SMBs don’t need this. It starts making sense when you have strict data regulations or very specific security requirements.
API pricing / per-token pricing is how you pay when building custom AI tools. Instead of a flat monthly fee, you pay based on usage: how many tokens (roughly, how many words) you process. A few cents per interaction doesn’t sound like much, but it adds up. Always estimate your monthly volume before committing to an API-based approach.
AI readiness is how prepared your business is to actually benefit from AI. It covers your data (is it organized?), your team (do they know how to use these tools?), your processes (are they documented?), and your goals (do you know what problem you’re solving?). Most businesses that struggle with AI aren’t struggling with the technology. They’re struggling with readiness.
ROI (return on investment) for AI usually means time saved, errors reduced, or revenue gained versus the total cost of the tool, implementation, and training. The honest version: AI ROI is often real but takes 2-3 months to materialize, and it’s harder to measure than vendors suggest.
Pilot project is a small, controlled test of an AI tool on one specific workflow before committing to a larger rollout. This is the approach we recommend to every business. Pick one thing, test it for two to four weeks, measure the results, then decide whether to expand.
The 20 terms you can safely ignore
Unless you’re building AI systems yourself, these terms won’t affect your buying decisions. If a vendor drops them in a pitch, they’re trying to impress you rather than inform you. Smile, nod, and ask them what it means for your Tuesday morning.
Transformer architecture is the technical design behind LLMs. You don’t need to know how the engine works to drive the car.
Embeddings are how AI converts text into numbers for comparison and search. Engineers care about this. You don’t need to.
Vector database is where those embeddings get stored. Plumbing behind RAG systems. Your vendor handles it.
Neural network is the foundational technology behind modern AI. Interesting if you like computer science history, irrelevant to tool selection.
Training data is what the model learned from. Unless you’re fine-tuning (which most businesses shouldn’t be doing), this is background context you can skip.
Parameters (as in “a 70-billion parameter model”) measure model size. Vendors love quoting big numbers here. Bigger isn’t always better, and the number won’t help you evaluate results.
Inference is the technical term for “using the model.” You already understand the concept; you just didn’t know it had a name.
Latency is response time. Matters for engineers, not for buying decisions.
Quantization is a technique for making models run faster by reducing precision. Engineering plumbing.
RLHF (reinforcement learning from human feedback) is how models learn to be helpful and safe. Fascinating rabbit hole if you’re curious, but it won’t show up in your vendor evaluation.
Diffusion models power image generation tools like Midjourney and DALL-E. Unless you need to generate images for your business, move on.
Multimodal means an AI that handles text, images, audio, and video. It sounds impressive in pitches, but most business use cases are still text-based.
Federated learning is a privacy-focused training approach that’s still mostly academic. You won’t encounter it in buying decisions for at least a few years.
Synthetic data is AI-generated data used to train other AI. Turtles all the way down. Matters for researchers, not for buyers.
Edge AI means running AI locally on devices instead of in the cloud. Relevant for manufacturing and IoT. If your team works in an office, skip it.
Attention mechanism is the core math behind transformers. Pure computer science.
Prompt injection is a security attack where someone tricks an AI into ignoring its instructions. You don’t need to understand the technical details, but you should ask your vendor whether they protect against it. A “yes, here’s how” is what you want to hear.
Mixture of experts (MoE) is a model architecture where specialized sub-models handle different tasks. Your vendor picks this, not you.
Agentic orchestration is a fancy way of saying “managing multiple AI agents working together.” If a vendor pitches this to a 50-person company in 2026, they’re almost certainly overselling.
Foundation model is the base model before customization. GPT-4 is a foundation model; a customer service bot built on top of it is a customized version. The term shows up in articles but won’t change how you evaluate a product.
A note on this glossary
AI vocabulary changes fast. Some of these terms will be irrelevant in a year, and new ones will take their place. We’ll update this guide when that happens.
But the useful test stays the same: when someone uses a term you don’t recognize, ask them what it means for your business specifically. Not what it means technically. What it means for your operations, your team, your bottom line. If they can explain that clearly, the term is worth learning. If they can’t, it’s jargon, and you can safely let it go.