The easiest way to run open-source LLMs locally
Introduction
Ollama distills running open LLMs locally into a single command, handling model downloads, quantization and serving across Windows, Mac and Linux. Modelfiles customize prompts and parameters, and built-in OpenAI-compatible APIs make it the first stop for local-AI enthusiasts.
Key Features
- One-command model pull and run
- Automatic quantization for local VRAM
- OpenAI-compatible local API
- Modelfile for custom prompts and params
- Cross-platform with Apple-silicon optimization
Pros
- Frictionless local model experience
- Extremely rich ecosystem integrations
Cons
- Average concurrency and production-grade throughput
Pricing
Completely free and open-source.（See official site for current pricing）
FAQ
What GPU do I need?
8GB VRAM for 7B models; small models also run CPU-only.
Does data leave my machine?
Inference is fully local; data never leaves the machine.
Ollama vs LM Studio?
CLI folks pick Ollama; GUI folks pick LM Studio.
Related Sites
Collected on lucifer.city · Reviewed by the founder