Reference

AI & LLM Glossary

50 terms you meet the moment a large language model goes into a real product — written by an engineer who ships these systems, not by a marketing team.

Every entry says what the term means and why it matters in practice, in plain language, without pretending the hard parts are solved. Each term has its own anchor link, so you can point a colleague — or an assistant — straight at a single definition.

Searches all 50 definitions and every page, in this language. Runs entirely in your browser.

LLM fundamentals8 terms

Large language model (LLM): A neural network trained on very large amounts of text to predict the next token. That single mechanism is enough to summarise, translate, classify and write code — but it also means the model produces plausible continuations, not verified facts.
Token: The unit an LLM actually reads and writes: roughly a word piece, about four characters in English and noticeably less efficient in Hungarian or German. Context limits, latency and billing are all counted in tokens, not words.
Context window: The maximum number of tokens a model can consider at once — system prompt, conversation, retrieved documents and its own answer together. Exceeding it truncates the oldest content, which is why long chats quietly start forgetting instructions.
Prompt: Everything sent to the model for one request: instructions, examples, retrieved context and the user's question. In production a prompt is not a clever sentence but a versioned, tested artefact — changing it changes behaviour the same way changing code does.
System prompt: The standing instruction that frames every request in a conversation: role, tone, rules, output format. It carries more weight than the user's message, which makes it the natural place for guardrails — and a frequent target of prompt injection.
Temperature: A sampling parameter controlling how much randomness is allowed when picking the next token. Near 0 the output is repeatable and conservative, which is what you want for extraction and classification; higher values buy variety at the cost of reliability.
Top-p and top-k sampling: Two ways to limit which tokens may be chosen. Top-k keeps the k most likely candidates; top-p (nucleus sampling) keeps the smallest set whose probabilities sum to p. Tightening them is usually a better way to stabilise output than temperature alone.
Hallucination: A fluent, confident answer that is simply wrong — an invented citation, API method or figure. It is not a bug to be patched but a property of next-token prediction, so it is managed with retrieval, grounding, output validation and evaluation rather than eliminated.

RAG and retrieval9 terms

RAG (retrieval-augmented generation): Retrieve the relevant passages from your own data first, then let the model answer using only those. It is how an LLM gets access to private, current or too-large-to-fit knowledge, and it turns answers into something you can trace back to a source.
Embedding: A vector of numbers representing the meaning of a piece of text, so that similar meanings sit close together. Embeddings are what make semantic search, clustering and deduplication possible without matching a single keyword.
Vector database: A store built to find the nearest vectors to a query vector in milliseconds across millions of entries — pgvector, Qdrant, Pinecone. For moderate data volumes a Postgres extension is usually enough; a separate service earns its keep at scale.
Semantic search: Search by meaning instead of by string match, using embeddings. It finds the right document when the user's words differ from the text's words — and it is also why an exact product code can rank worse than with plain keyword search.
Chunking: Cutting documents into retrievable pieces. The single most underrated decision in RAG: chunks that split a table from its header, or an answer from its question, quietly poison retrieval no matter how good the model is.
Reranking: A second pass that reorders the retrieved candidates with a slower, more accurate model before they reach the LLM. Retrieve twenty, rerank, keep five: usually the cheapest single improvement to answer quality in a working RAG system.
Hybrid search: Combining keyword (BM25) and vector search and merging the two result lists. It covers each method's blind spot — exact identifiers and rare words for the keyword side, paraphrases and synonyms for the vector side.
Grounding and citation: Requiring the model to answer only from supplied sources and to name which passage each claim came from. Citations are not decoration: they make an answer auditable, and they make a wrong answer diagnosable.
llms.txt: A plain-text file at the root of a site that gives AI assistants a structured summary and a map of the important pages — robots.txt for meaning rather than permission. The companion llms-full.txt carries the entire content in one fetch.

Training and tuning8 terms

Pre-training: The original, enormously expensive training run on broad text that produces a base model's general ability. Almost nobody outside the large labs does this; everything most teams call 'training' is one of the cheaper adaptations built on top.
Fine-tuning: Continuing training on your own examples so the model adopts a format, tone or narrow task. It teaches behaviour, not facts — if the model needs to know your data, use retrieval; fine-tune when you need it to answer in a specific way, consistently.
LoRA and parameter-efficient tuning: Training a small set of extra weights instead of the whole model, cutting cost and hardware needs by orders of magnitude. Adapters can be swapped per customer or task and stacked on one shared base model.
RLHF (reinforcement learning from human feedback): Aligning a model using human preference judgements between candidate answers. It is what turns a raw text predictor into an assistant that follows instructions and declines the things it should decline.
Instruction tuning: Training on instruction-and-response pairs so a model reliably does what it is asked rather than continuing the text. It is the difference between a completion engine and something you can hand a task to.
Zero-shot and few-shot prompting: Zero-shot means asking with instructions only; few-shot means including a handful of worked examples in the prompt. Three good examples routinely beat a page of prose instructions, and cost far less than fine-tuning.
Quantization: Storing model weights at lower numeric precision so a model fits on smaller hardware and runs faster. Quality loss is modest down to about 4-bit for most tasks, which is what makes self-hosted models practical.
Distillation: Training a small, cheap model on the outputs of a large one. Done on a narrow, well-defined task it can match the big model closely at a fraction of the latency and cost — and it is a sound way to get an expensive pipeline into production economics.

Agents and tool use8 terms

AI agent: An LLM that can take actions — call tools, read results, decide the next step — instead of only producing text. The engineering work is rarely the reasoning; it is the boundaries, permissions and stopping conditions around it.
Agentic workflow: A multi-step process where the model plans, acts and checks its own work in a loop. Powerful for open-ended tasks, but each extra step multiplies cost, latency and failure modes — most production problems are better served by a fixed pipeline.
Tool calling (function calling): Giving the model a typed list of functions it may invoke — a database query, an API call, a calculation — and letting it return a structured call instead of prose. This is how an LLM touches real systems safely and verifiably.
MCP (Model Context Protocol): An open protocol for connecting models to tools and data sources through a common interface, so an integration written once works across assistants. It replaces the bespoke glue every team used to write per vendor.
Multi-agent system: Several specialised agents working on one task, often with a coordinator. It genuinely helps when subtasks need different tools or permissions; it mostly adds cost and non-determinism when a single well-prompted call would do.
Guardrails: The checks around a model: input filtering, output schema validation, allowed-action lists, budget and step limits. Guardrails are ordinary software — deterministic code you can test — and they are what makes a non-deterministic component safe to ship.
Human in the loop: Requiring human approval before an action that is expensive, irreversible or customer-facing. The design question is not whether to include a human, but exactly which steps need one — too many approvals and the automation stops paying for itself.
Orchestration: The layer that decides what runs when: retrieval, model calls, tools, retries, fallbacks and state. Frameworks like LangChain or LangGraph live here — useful, but a plain state machine is often the more maintainable choice.

Evaluation and quality8 terms

Benchmark: A standard test set used to compare models. Useful for a shortlist, misleading as a decision: public benchmarks leak into training data, and none of them measure your documents, your users or your definition of a good answer.
Eval: Your own automated test suite for an LLM feature: fixed inputs, expected properties, a score you can watch across model and prompt changes. Without evals you are not engineering an AI feature, you are adjusting it by feel.
Golden dataset: A curated set of real inputs with verified correct outputs, owned by the domain experts rather than the developers. A hundred well-chosen cases beat ten thousand scraped ones, and it is the single asset that makes every later model change measurable.
LLM as a judge: Using a model to grade another model's output against a rubric. It scales evaluation to volumes humans cannot review, but it inherits the judge's biases — so it needs calibrating against human ratings before you trust the numbers.
Determinism and reproducibility: Whether the same input reliably produces the same output. Full determinism is not achievable with hosted models, but temperature near zero, fixed prompts, pinned model versions and structured outputs get close enough to test against.
Precision and recall: Precision is how many of the results returned were correct; recall is how many of the correct ones were found. They trade off against each other, and choosing which matters more is a business decision — a missed fraud case and a false accusation do not cost the same.
LLM-based classification: Using a model to sort text into categories — tickets, documents, intents — without training a dedicated classifier. Fast to build and strong on messy language; it needs a fixed label set, a confidence threshold and an explicit path for 'none of these'.
Observability and tracing: Recording every prompt, retrieved chunk, tool call, token count and latency for each request. When a user reports a bad answer, this is the difference between reproducing it in minutes and guessing for a day.

Integration and operations9 terms

Inference and inference API: Inference is a single run of the model on your input; the inference API is the hosted endpoint you call for it. Treat it as a third-party dependency with an SLA: it can be slow, rate-limited or down, and your product has to behave sensibly when it is.
Prompt caching: Reusing the processed form of a long, unchanged prompt prefix across requests. With a large system prompt or a fixed document set it cuts both cost and time to first token substantially — provided the stable part comes first.
Streaming: Sending tokens to the client as they are generated instead of waiting for the full answer. It does not make generation faster, but it cuts perceived latency dramatically — and it needs your whole stack, proxies included, to pass chunks through unbuffered.
Rate limit: The provider's cap on requests or tokens per minute. It is the most common cause of an AI feature that works in demo and fails at launch, so queueing, backoff and a fallback model belong in the design, not in the incident review.
Token cost: Billing is per input and output token, at different rates. The practical consequence: retrieved context, long system prompts and chat history are the real cost drivers, and an unbounded conversation is an unbounded invoice.
Latency and time to first token: Total time matters less than when the user sees something happen. Time to first token is the number worth optimising: streaming, prompt caching, smaller models for simple steps, and doing retrieval in parallel rather than in sequence.
Self-hosted vs. hosted models: An open-weights model on your own hardware gives data control, stable pricing and no vendor cut-offs; a hosted frontier model gives higher capability with no ops burden. Most systems end up mixed: local for bulk and sensitive work, hosted for the hard requests.
Data protection in LLM systems: Under the GDPR, sending personal data to a model provider is processing by a third party: it needs a legal basis, a data processing agreement, a known storage region and a retention answer. Whether the provider trains on your data is a contract question, not a technical one.
AEO and GEO: Answer Engine Optimization and Generative Engine Optimization: making a site retrievable and quotable by AI assistants rather than only rankable by search engines. In practice it is structured data, machine-readable summaries and content that survives being cut into chunks.

Building something with an LLM?

I design and ship production AI systems — RAG pipelines, semantic search, evaluation and the engineering around them.

Get in touch Back to homepage