What is a local LLM?
Definition
A local LLM is a large language model whose files (the model weights) you download and run on your own computer, instead of sending your prompts to a cloud service. Once the model files are downloaded, it can run without an internet connection. LM Studio's docs say the app "can operate entirely offline" once you have the model files.
Checked: 2026-10-29 (Beijing time) · Sources
Examples
- Ollama in a terminal: after installing Ollama,
ollama run gemma4:e2bdownloads the Gemma 4 E2B model and starts a chat on your computer. Ollama's quickstart says the download is about 7.2 GB and recommends 8 GB of available VRAM (or unified memory on a Mac). With less VRAM it can use system RAM, but responses may be slower. Local requests need no API key and go to a server atlocalhost:11434. - LM Studio as a desktop app: LM Studio downloads models from Hugging Face and runs them on Mac, Windows and Linux with
llama.cpp(GGUF models), and with Apple's MLX on Apple Silicon Macs. It can also serve a local model on OpenAI-compatible endpoints so other apps can use it.
What you need to know first
Memory is the main limit: Ollama's quickstart notes that larger context windows need more memory, and that responses may be slower when a model does not fit in VRAM. Open WebUI, a self-hosted chat interface that supports Ollama, can sit on top of a local model, and the terminal coding tool Aider can connect to local Ollama models (Aider's docs warn that Ollama's default 2k context window is small, so Aider raises it). Ollama's quickstart also shows how to launch coding agents (see AI agent) with its models. Our best AI to run locally page compares the full local stack.
Related tools on this site
- Ollama: Run open AI models locally (and in the cloud) on macOS, Windows and Linux.
- LM Studio: Desktop app to run local models; Bionic is its agent for documents, slides, PDFs and code.
- Open WebUI: Self-hosted, user-friendly AI chat interface supporting Ollama and OpenAI-compatible APIs.
- Aider: AI pair programming in your terminal; automatically commits changes with sensible commit messages.
- ComfyUI: Modular node-graph GUI, API and backend for running diffusion models locally.
Related pages
Sources
- Ollama docs: Quickstart. Checked: 2026-10-29.
- LM Studio docs: Welcome to LM Studio Docs. Checked: 2026-10-29.
- Aider docs: Ollama. Checked: 2026-10-29.
- Open WebUI docs: Home. Checked: 2026-10-29.
We only state facts we checked on the linked sources on the dates shown. Found a mistake? Email jazz.fanpeng@mail.grokbot.com.