A large language model and a small language model do the same basic job, predicting text one piece at a time, but they're built for different jobs entirely. What separates them is measured in parameters: the internal values a model adjusts during training to learn patterns in language, and roughly, the more of them a model has, the more it can hold and reason about. Large models chase broad capability: more knowledge, better reasoning, stronger performance across almost any task. Small models chase efficiency: something that fits on a phone, answers fast, and costs close to nothing to run.

There's no official cutoff between the two categories, but the difference in what each one can do, and where each one runs, is real.

The quick answer

  • Small language models (SLMs): roughly half a billion to 14 billion parameters. Built to run on a phone, laptop, or modest server, often for a single narrow task.
  • Large language models (LLMs): tens of billions of parameters up into the trillions. Built for broad reasoning and general-purpose use, almost always served from the cloud.
  • The real dividing line is where the model runs and what it's tuned to do, not size on its own. A 9B model fine-tuned (retrained further on data specific to one job) can beat an older 70B general-purpose model at that job.
  • Cost and speed favor small models. Depth and range favor large ones.

What counts as "small" and what counts as "large"

Parameter count is the standard way to size a model, and both categories above are defined by it.

Small models built for phones and laptops publish exact numbers: Meta's Llama 3.2 ships 1B and 3B text-only versions built for edge use, meaning the model runs right on the device instead of in a data center. Microsoft's Phi-4 Mini runs at 3.8B. Google's Gemma line spans 2B up through 27B, and its 9B version was reported to outperform several 70B models that came before it. Alibaba's Qwen3 and Hugging Face's SmolLM2 both ship sub-2B versions built to run entirely on a phone's CPU, the chip that handles general processing rather than the specialized graphics chip larger models depend on.

On the large end, the numbers get harder to pin down for closed models. OpenAI and Anthropic don't publish exact parameter counts for GPT or Claude, so any specific figure floating around for those is an estimate, not a confirmed spec. Open-weight labs do publish theirs: DeepSeek's V3 and V4 models run at 671 billion total parameters, and Mistral Large 3 is a 675-billion-parameter model. Meta's largest Llama 3.1 release hit 405 billion. Those are the real, checkable numbers behind "large."

Where each one runs

This is the more useful dividing line than parameter count alone. Small models are built to run locally: on a phone, a laptop, or a single modest GPU (the chip that handles the heavy math behind AI models), without a live connection to anyone's servers. In 2026, flagship phones from Samsung, Google, and Motorola run on-device inference, meaning the phone generates the response itself instead of sending the request to a server, for models up to roughly 4 billion parameters, covering Phi-4 Mini, Gemma's smallest variants, SmolLM2, Qwen3 1.7B, and Llama 3.2 3B.

Large models need real infrastructure. A 70-billion-parameter-plus model wants dozens of gigabytes of GPU memory at minimum, and the frontier closed models run on data-center hardware most people will never own, which is exactly why they're only available through an app or an API, a way other software can send the model requests directly, rather than a download. For a closer look at what it takes to run an open model yourself, see what is a local LLM.

What each size is good for

Small models are built for one job done well, close to the user: a voice assistant that works without signal, a phone keyboard that predicts text, a customer-support bot answering a fixed set of questions, or any task running at a volume where cloud API costs would add up fast. Fine-tuned on the right data, a small model can match or beat a much larger general-purpose model at that one job.

Large models are built for range: writing that needs real nuance, multi-step reasoning, coding across an unfamiliar codebase, or any task where the question itself is unpredictable. Nobody's fine-tuning a 3B model to handle "whatever a user throws at it today," because that's precisely the job large, general-purpose models are trained for.

Which one to reach for

If the task is narrow, repeated often, needs to run offline, or needs to stay cheap at high volume, a small model is usually the better fit, and it's worth checking whether an open-weight model you can self-host covers it before paying per token. If the task is broad, unpredictable, or needs real depth of reasoning, a large model like one of Claude's models is still the more capable choice.

Most real setups end up using both: a large model for the hard, unpredictable work, and a small one handling the repeatable jobs behind the scenes. If you're deciding how a team or organization should split that work, that's worth a conversation.

Sources:
ibm.com: small language models · machinelearningmastery.com: introduction to small language models · datacamp.com: top small language models · turingpost.com: small language models list