What is an LLM?

LLM stands for Large Language Model. It is a type of artificial neural network trained on massive amounts of text (billions of web pages, books, articles, source code) to learn to predict the next word in a sentence. ChatGPT, Claude, Gemini, Llama, Mistral, DeepSeek: all those names you have been seeing everywhere since 2023 are LLMs or products built on LLMs.

The principle may seem ridiculously simple for a technology that generates code, writes essays, summarises 500-page documents and passes medical exams: predict the next word. But when you train a neural network that is large enough on enough text, this prediction task gives rise to capabilities nobody truly anticipated: reasoning, translation, problem solving, code generation, and a form of language understanding that, while not human, produces results often indistinguishable from those of a human.

How an LLM works

An LLM is a very complex mathematical function. It takes a sequence of words (or more precisely "tokens", word fragments) as input, and produces a probability distribution over all possible tokens as output. The token with the highest probability is selected as the next word, then added to the sequence, and the process starts again. This is how the model generates text, word by word (or token by token).

The underlying architecture is called the Transformer, introduced by Google researchers in 2017 in a paper titled "Attention Is All You Need". The key innovation of the Transformer is the attention mechanism: instead of processing words one by one from left to right (like the recurrent networks that existed before), the Transformer can consider all words in the sequence simultaneously and calculate the relationships between every pair of words. This is what allows it to understand context: when you write "the bank of the river", the model knows that "bank" refers to the river's edge and not a financial institution, because it has calculated the relationship between "bank" and "river".

In 2026, the dominant architecture is Mixture of Experts (MoE): the model contains trillions of total parameters, but only a fraction (a few tens of billions) is activated for each token. It is like a building with hundreds of specialised offices: for each question, you are directed to only 2 or 3 relevant offices, not all 200. This allows frontier-quality output at a much more reasonable computational cost.

The three stages of creating an LLM

An LLM is not born capable of holding a conversation. Its creation goes through three distinct stages, each giving it different capabilities.

Pre-training. The model is trained on a massive text corpus (billions of web pages, books, source code, scientific papers, forum discussions). The objective is simple: predict the next word. By doing this billions of times across billions of texts, the model learns grammar, facts, logical reasoning, style, and a form of "world knowledge" encoded in the statistical relationships between words. Pre-training is the most expensive phase: training GPT-3 consumed 1,287 MWh of electricity, equivalent to powering 330 American homes for a year. The 2026 models consume even more.

Supervised fine-tuning (SFT). After pre-training, the model can complete text, but it does not know how to answer questions, follow instructions or hold a structured conversation. Supervised fine-tuning involves training it on high-quality conversation examples: question-answer pairs written by humans, instruction-following examples, reasoning tasks. This step transforms a "text completer" into a conversational assistant.

Alignment (RLHF, DPO, GRPO). The third stage aligns the model's behaviour with human preferences. The historical method is RLHF (Reinforcement Learning from Human Feedback): humans compare two of the model's responses and indicate which is better. A reward model is trained on these preferences, then used to optimise the LLM via reinforcement learning. More recent methods like DPO (Direct Preference Optimization) and GRPO (Group Relative Policy Optimization) simplify this process. Alignment is what prevents the model from producing toxic content, following dangerous instructions or responding in an unnecessarily verbose way.

Tokens: the basic unit

LLMs do not work with words but with tokens. A token is a text fragment: a common word is often a single token, a rare word may be split into several tokens, and a special character or punctuation mark is usually a token on its own. On average, for English or French text, one token corresponds to roughly 3/4 of a word. A 1,000-word text is approximately 1,300 tokens.

This distinction matters because LLMs charge and limit by tokens, not words. The "context window" (the amount of text the model can process at once) is measured in tokens. In 2026, frontier models offer context windows ranging from 128,000 to 1 million tokens (Gemini, Claude), and Meta Llama 4 Scout reaches 10 million tokens. But "context window" and "effective context" are not the same thing: most models lose accuracy on information located in the middle of a long context.

What LLMs can do

The list of LLM capabilities in 2026 is long, and it grows with each new generation of models.

Text generation. Writing emails, reports, articles, scripts, social media posts. This is the most common and visible use case.

Summarisation and analysis. Condensing a 200-page document into a few paragraphs, extracting key points, identifying main arguments, comparing two texts.

Translation. LLMs translate between dozens of languages with quality that often rivals human translators for everyday texts (less reliable for highly specialised or literary texts).

Code generation and analysis. Writing code in virtually any programming language, debugging existing code, explaining what a programme does, converting code from one language to another. In 2026, roughly 85% of developers regularly use LLM-assisted coding tools.

Reasoning. Reasoning models (GPT-5 with configurable effort, Claude with "extended thinking", Gemini 3, DeepSeek R1) can solve complex mathematical problems, logic puzzles and tasks requiring multi-step planning. They "think" by producing an internal chain of thought before giving their final answer.

Multimodality. Modern LLMs are no longer limited to text. They can analyse images (describe a photo, read text in an image, interpret a chart), process audio files (transcription, voice translation), and in some cases generate images or video.

What LLMs cannot do

Despite their impressive capabilities, LLMs have fundamental limitations that you need to understand to use them correctly.

Hallucinations. LLMs sometimes invent false information with complete confidence. They can cite studies that do not exist, attribute quotes to people who never said them, or give completely made-up figures. The problem stems from their very nature: they generate the most probable text, not the most truthful text. For an LLM, a false statement that "sounds right" can have a higher probability than a true statement phrased in an unusual way. The hallucination rate has decreased with each generation of models, but it is not zero, and it probably never will be.

Reliable reasoning over long chains. LLMs can chain 3 to 5 reasoning steps reliably. Beyond that, errors compound. A problem requiring 15 steps of logical reasoning will be solved inconsistently, even by the best models. Reasoning models (which "think" for longer) raise this ceiling but do not eliminate it.

Up-to-date knowledge. An LLM is frozen in time. It knows the information contained in its training data but knows nothing about what happened after its training cutoff date. To access recent information, it needs tools (web search, database access), which amounts to building an AI agent around the LLM.

Mathematics and counting. Paradoxically, LLMs are poor at basic arithmetic when they have to do it "in their head" (counting letters in a word, multiplying large numbers). They work well on mathematical reasoning (understanding a problem and formulating the approach) but not on raw computation. This is why the best systems give them access to a code interpreter to run calculations rather than doing them themselves.

Memory between conversations. By default, an LLM has no memory between two conversations. Each exchange starts from scratch. Memory systems (summaries of previous conversations, context databases) are layers added by applications, not a native capability of the model.

The main LLMs in 2026

The LLM ecosystem has structured itself around two categories: proprietary models (accessible only via API or web interface) and open-source models (downloadable and runnable locally).

GPT-5 (OpenAI) is OpenAI's flagship model, the successor to GPT-4. It includes a configurable reasoning mode that lets it "think" for longer or shorter depending on the task's complexity. ChatGPT, built on GPT-5, reaches 900 million weekly active users in 2026.

Claude (Anthropic) is Anthropic's family of models, with an architecture focused on safety and alignment. Claude is available in several sizes (Haiku, Sonnet, Opus) suited to different quality/cost/speed trade-offs. Anthropic serves over 300,000 business customers.

Gemini (Google) is Google's family of models, integrated into Google Search, Gmail, Google Docs and Android. The Gemini app surpasses 900 million monthly active users in 2026. Gemini stands out for its very large context windows (up to 1 million tokens).

Llama (Meta) is the leading open-source model family, freely downloadable and modifiable. Llama 4 Scout reaches 10 million tokens of context. Llama models are widely used by companies that want to host their own LLM without depending on a cloud provider.

DeepSeek (DeepSeek) is a Chinese open-source model that sent shockwaves through the industry in early 2025 by demonstrating GPT-4-comparable performance at a far lower training cost. DeepSeek proved that frontier performance does not necessarily require frontier budgets.

Mistral (Mistral AI) is the leading European model, developed by a French startup. Mistral focuses on efficiency: smaller models that punch well above their weight, particularly suited to use cases where latency or cost is critical.

Parameters: what do they mean

When people talk about a "70 billion parameter" or "1 trillion parameter" model, parameters are the numerical values adjusted during training that define the model's behaviour. Each parameter is a floating-point number stored in the neural network. Roughly speaking, more parameters means the model can encode more knowledge and more complex relationships between words.

But size is not everything. An 8-billion-parameter model well trained on high-quality data can outperform a 70-billion-parameter model poorly trained on mediocre data. The quality of training data, the model architecture and post-training techniques matter as much as, if not more than, the raw parameter count. This is the lesson DeepSeek taught the industry: performance does not depend solely on budget.

Proprietary vs open-source models

The debate between proprietary and open-source models is one of the most active in the industry in 2026.

Proprietary models (GPT-5, Claude, Gemini) are accessible only via API or web interface. You cannot see their source code, download them or modify them. The advantage: they are generally the highest performing, constantly updated, and you do not need to manage infrastructure. The disadvantage: you depend on the provider (pricing, terms of use, availability, data privacy policy).

Open-source models (Llama, DeepSeek, Mistral, Qwen) are downloadable, modifiable and runnable on your own infrastructure. The advantage: total control over data (nothing leaves your servers), the ability to fine-tune the model for your specific use case, no vendor dependency. The disadvantage: you must manage infrastructure (expensive GPUs), performance is generally slightly below the best proprietary models, and updates are less frequent.

In 2026, the performance gap between the two categories is narrowing. DeepSeek V3.2, Qwen3 and Llama 4 rival proprietary models on many benchmarks. For many companies, the choice is less about raw performance than about privacy requirements, cost and control.

The cost of using LLMs

Proprietary LLMs charge per token (input and output). Prices vary considerably depending on the model and provider. A frontier model typically costs between $2 and $15 per million input tokens, and between $8 and $60 per million output tokens. Smaller or older models cost a fraction of these prices.

For self-hosted open-source models, the main cost is hardware: a professional GPU (NVIDIA A100 or H100) costs between $10,000 and $30,000, and a large model requires multiple GPUs. The per-token cost is lower once the infrastructure is in place, but the upfront investment and maintenance are significant.

The global LLM market is estimated at $7.81 billion in 2025 and is projected to reach $130 billion by 2034. OpenAI has surpassed $20 billion in annualised revenue, Anthropic has reached $14 billion.

Risks and limitations to keep in mind

Data privacy. When you send text to an LLM via a cloud API, that text is transmitted to the provider's servers. Always check the privacy policy: some providers may use your data to train their future models. If privacy is critical, consider a self-hosted open-source model or an enterprise contract with explicit guarantees.

Bias. LLMs reflect the biases present in their training data. If the data over-represents certain cultural, geographic or demographic perspectives, the model will reproduce those biases. Providers work to reduce these biases through alignment and filtering, but the problem is not solved.

Energy consumption. Training and running LLMs consumes considerable amounts of energy. The AI sector could consume between 85 and 134 TWh per year by 2027, almost as much as the Netherlands. This is a growing environmental concern.

Dependency. The concentration of the market around a few providers (OpenAI, Google, Anthropic, Meta) creates dependency risks. A price change, a policy update or an outage can affect millions of users and applications.

An LLM is a neural network trained to predict the next word. This simple task, applied at massive scale, gives rise to reasoning, writing, translation and programming capabilities that have transformed computing in three years. In 2026, LLMs are used by hundreds of millions of people, but their limitations (hallucinations, fragile reasoning over long sequences, lack of native memory) remain real. Understanding what an LLM is and what it is not is the condition for using it effectively.