APL 3 · Dev Setup
Ollama
Run LLMs locally on your machine. Full privacy, no API costs, complete control.
Quick Facts
- Level: L3 (Dev Setup)
- Type: Local LLM runtime
- Maker: Ollama (open source)
- Runs on: Your machine (macOS, Linux, Windows)
- Access: Fully local - nothing leaves your machine
What It Solves
Ollama lets you run open-source LLMs on your own hardware. No API keys, no cloud, no data leaving your machine. It's the privacy-first approach to AI coding assistance.
Where It Belongs
APL 3 with a unique advantage: AAL is inherently limited because nothing leaves your machine. This makes it ideal for sensitive codebases where data privacy is non-negotiable.
What It's Good At
- Privacy-sensitive environments (no data leaves your machine)
- Offline development (works without internet)
- Cost-free inference after initial setup
- Experimentation with different models (CodeLlama, Mistral, Phi, etc.)
- Integration with other L3 tools as a backend
- Learning how LLMs work by running them directly
What It's Not For
- Users without decent hardware (8GB+ RAM recommended, 16GB+ ideal)
- Tasks requiring the latest frontier model performance
- Complex agentic workflows out of the box (it's a runtime, not an agent)
- People who need managed infrastructure
Risk Profile
Ollama has a unique risk profile - it's both L3 in capability and inherently private. The main risks are:
- Model quality: Local models are less capable than frontier cloud models
- Hardware requirements: Running large models needs significant RAM and GPU
- Over-trust: Just because it's local doesn't mean the output is correct
Getting Started
Install Ollama, pull a model, and start generating. The setup is straightforward:
- Install from ollama.com
- Pull a code model:
ollama pull codellama - Start using:
ollama run codellama