Glossary

Instead of telling the computer every rule, you show it lots of examples and it figures out the pattern itself.

A branch of AI in which a model improves at a task by learning statistical patterns from data, rather than being given explicit instructions for every case.

Methods that fit a parameterized function to data by minimizing a loss over training examples, aiming to generalize to unseen inputs. Paradigms include supervised, unsupervised, and reinforcement learning.

A web of tiny math "neurons" loosely inspired by the brain, stacked in layers, that learns to recognize patterns.

A computing structure made of layers of connected units ("neurons"); each connection has a weight the system tunes during training so the network maps inputs to useful outputs.

A composition of parameterized linear transformations and non-linear activations. Weights are learned via backpropagation and gradient descent to minimize a loss; depth and width govern representational capacity.

A kind of machine learning that stacks many layers of "neurons," letting it learn very complex patterns like faces or speech.

Machine learning based on neural networks with many layers, which learn increasingly abstract features of the data and power most modern AI breakthroughs.

A subfield using deep multi-layer networks trained end-to-end by backpropagation; depth enables hierarchical feature learning and strong performance given large data and compute.

You teach the model with examples that already have the right answers, so it learns to answer new ones.

Training on labeled data — inputs paired with correct outputs — so the model learns to map new inputs to answers. It underlies most practical AI today.

Learning a function from input–label pairs by minimizing prediction error (e.g., cross-entropy, MSE); the basis of classification and regression when labels are plentiful.

A kind of AI — not built yet — that could do almost anything a person can, not just one narrow job.

A hypothetical AI able to match human performance across most cognitive tasks, not just narrow ones. Definitions vary widely between labs.

An informal target denoting broad, human-level competence across domains; it lacks a rigorous, agreed operationalization, which complicates measurement and claims.

A model design that looks at how every word in a sentence relates to every other word, all at once, so it handles language really well.

A neural-network architecture, introduced in 2017, that uses an "attention" mechanism to weigh how much each part of the input relates to every other part — enabling fast, highly parallel training at scale.

An architecture built on self-attention, computing weighted combinations of value vectors from query–key similarities across all positions. Removing recurrence enables parallelization; positional encodings restore order.

An AI that has read enormous amounts of text and learned to predict what comes next — usually a word or part of a word (a "token"), which turns out to be enough to hold a conversation.

A transformer-based model trained on very large text corpora to predict the next token, which gives it broad abilities to write, summarize, translate, and answer questions.

A high-parameter autoregressive model estimating the probability of the next token given prior context. Pre-trained by self-supervised next-token prediction, then typically aligned via instruction tuning and RLHF.

AI that makes new things — a paragraph, a picture, a tune — instead of just sorting or scoring what already exists.

A class of models that produce novel content (text, images, audio, video, code) by learning the patterns of their training data and sampling new outputs from them.

Models that approximate the data distribution and sample from it. Families include autoregressive transformers (text), diffusion models (images/audio), and GANs, differing in how they represent and sample the distribution.

AI that can handle more than one kind of input or output — you can show it a picture and talk to it about it.

A model that accepts and/or produces more than one type of data — text, images, audio, video — within a single system, closer to how people naturally communicate.

A model that maps multiple modalities into a shared representation space, enabling cross-modal reasoning and generation. Often built by aligning modality-specific encoders to a language backbone.

A way to turn words or pictures into lists of numbers, so the computer can tell which things are similar in meaning.

A representation that maps data such as words or images into numeric vectors where similar items sit close together, enabling meaning-based search and matching.

A learned mapping into a dense vector space where geometric proximity approximates semantic similarity; foundational to retrieval, clustering, and nearest-neighbor search.

An AI image-maker that starts with random static and cleans it up step by step until a picture appears.

A generative model that creates images by starting from random noise and gradually denoising it into a coherent result; it's behind most high-quality AI image generators.

A model trained to reverse a gradual noising process, denoising samples step-by-step from a prior toward the data distribution; enables high-fidelity image and audio synthesis.

AI models that work through a problem step by step before giving an answer, instead of replying instantly — better for tricky maths or logic.

Language models trained or prompted to produce intermediate reasoning steps before a final answer, trading extra compute at answer-time for higher accuracy on complex tasks.

Models optimized to allocate additional inference-time computation to multi-step reasoning (e.g., extended chains of thought), often reinforced toward verifiable correct solutions on math, code, and logic benchmarks.

An AI that can actually do things for you — search, click, send, book — step by step, not just chat.

A system that pursues a goal by planning and taking actions — calling tools, browsing, or using other software — with limited human supervision, and adjusting based on results.

An LLM-driven control loop that interleaves reasoning with tool/API calls, observes results, and iterates toward a goal. Capability and risk scale with autonomy, tool access, and memory.

Before answering, the AI searches a set of documents and uses what it finds — so replies are more accurate and up to date.

A method that retrieves relevant text from an external source and feeds it to a language model before it answers, grounding responses in real, current, or private data.

A pipeline that embeds a query, retrieves nearest-neighbor passages from a vector store, and conditions generation on them — decoupling knowledge from model weights and improving factual grounding and freshness.

A shared "plug" standard that lets AI apps connect to your tools and data without custom wiring for each one.

An open protocol that standardizes how AI applications connect to external tools, data sources, and services — so any compliant client can talk to any compliant server.

A client–server protocol that exposes tools, resources, and prompts to AI hosts over a defined message interface, decoupling model applications from bespoke, per-integration connectors.

The skill of asking an AI the right way — clear instructions, examples, and the format you want — to get a better answer.

The practice of designing inputs — instructions, examples, constraints, and structure — to steer a model toward more accurate and useful outputs.

Conditioning model behavior through the input context: task framing, few-shot exemplars, output schemas, and decoding constraints, exploiting in-context learning without changing weights.

The little pieces — usually a word or part of a word — that an AI breaks text into. Costs and limits are counted in these.

The unit of text a language model processes — often a word or sub-word fragment. Context limits and pricing are typically measured in tokens.

A discrete symbol from a fixed vocabulary produced by a tokenizer (e.g., byte-pair encoding). Sequences of token IDs are the model's actual input and output; token count drives context and cost.

How much text an AI can "hold in mind" at one time. Go past it and it forgets the earliest parts.

The maximum span of text, measured in tokens, a model can attend to in a single request — setting how much it can "remember" in one conversation or document.

The fixed maximum token length the model conditions on per forward pass; exceeding it forces truncation, chunking, or retrieval, and attention cost scales with it.

Taking a ready-made AI and giving it extra training on your specific topic so it does that job better.

Taking an already-trained model and training it further on specialized data so it performs better for a specific task or style — without training from scratch.

Continued training of pretrained weights on a narrower dataset, typically at a small learning rate; variants include full fine-tuning and parameter-efficient methods (e.g., LoRA, adapters).

The pile of examples an AI studies to learn. Good examples in, good AI out — and the reverse.

The examples a model learns from; their range and quality largely decide how capable — and how biased — the model becomes.

The dataset optimized over during training; distribution, coverage, labeling quality, and contamination directly shape generalization, fairness, and failure modes.

When an AI confidently makes something up — a fake fact, quote, or source — that sounds right but isn't.

An output that is fluent and confident but factually wrong or fabricated, arising because the model generates plausible text rather than retrieving verified facts.

A failure mode where a generative model produces content unsupported by its inputs or ground truth — a byproduct of probabilistic next-token generation optimized for plausibility, not veracity.

Being able to see why an AI made a decision, instead of it being a black box.

How well humans can understand why an AI reached a particular decision — crucial in high-stakes areas like medicine, credit, and law.

The degree to which a model's internals or decisions can be interpreted by humans; spans intrinsically interpretable models and post-hoc methods (e.g., feature attribution), with fidelity trade-offs.

When an AI treats some groups unfairly — usually because the data it learned from was skewed.

Systematic, unfair skew in a model's behavior, typically inherited from unrepresentative training data or design choices, that can disadvantage particular groups.

Statistical and societal disparities in model outputs across attributes, originating in data sampling, labeling, objective design, or deployment context; measured via fairness metrics that can mutually conflict.

The work of making sure AI does what we want and doesn't cause harm — especially as it gets more capable.

The field focused on ensuring AI systems behave reliably and as intended, and don't cause harm, even when powerful, widely deployed, or in unexpected situations.

Research and practice spanning robustness, alignment, evaluation, interpretability, and control — aiming to keep system behavior within intended bounds under distribution shift and adversarial pressure.