The moment you paste a contract, a customer list, or source code into a cloud chatbot, it's on someone else's server. Local AI tools run models entirely on your machine — your data never leaves. This is the fastest-growing category in privacy, and it's all open source.
The open-source tools below have no affiliate programs — they're open source, so nobody pays us to recommend them. We list them on merit because they're genuinely the right answer for a specific, growing problem: how to use AI without handing your data to a third party. If that's your threat model, this page is the setup you need. (One exception below — Proton's Lumo — earns us a commission, and it's flagged as such.) Hardware requirements: a modern laptop can run small models (7B) acceptably; for serious work you want a desktop with 16GB+ RAM and a decent GPU.
Three pieces: a runtime to run models, an interface to talk to them, and a document tool when you need retrieval over your own files.
One command installs and runs Llama, Mistral, Qwen, and hundreds of open models on macOS, Linux, or Windows. Built-in OpenAI-compatible API means every AI tool you already use can point at it. It's how we run our own private assistant stack — zero cloud, zero account. If you try one local AI tool, make it this one.
Visit Ollama ↗If Ollama is the terminal-first option, LM Studio is the GUI-first one. Browse, download, and chat with models without touching a command line. Excellent for testing different models side by side to find what fits your hardware. Same local-only privacy story — everything runs on your machine.
Visit LM Studio ↗Point it at a folder of PDFs, contracts, or notes and ask questions with full retrieval — all local, all private. The reference implementation for private RAG (retrieval-augmented generation). Setup is more involved than Ollama, so it's for the technically comfortable — but it's the pattern every "chat with your documents" product is built on, done without the cloud middleman.
View on GitHub ↗A drop-in, OpenAI-compatible API server that runs fully self-hosted — text, audio, image, and embedding models. If you're building an internal tool and want to swap the OpenAI endpoint for a local one without rewriting code, this is the cleanest path. Docker-friendly, works on CPU-only boxes (slower, but works), and the most complete self-hosted API surface of the group.
Visit LocalAI ↗Sometimes local hardware isn't enough — you need a frontier model, or you're on a laptop that can't run one. That's where Lumo fits: Proton's AI assistant with the same zero-access architecture as Proton Mail. Every conversation is end-to-end encrypted, never logged, never used for training, and hosted on European infrastructure. It can't read your data any more than it can read your inbox — the privacy property is identical to the local tools above, just without the hardware requirement. Also includes an image model and web search with citations.
With cloud AI, your prompts are the product input — logged, retained, and often eligible for training use. With local AI, the model weights live on your disk, inference runs on your CPU/GPU, and the only copy of your data is the one you control. For anyone handling PII, PHI, or proprietary material, that's not a trade-off — it's the whole point. This is the same reasoning that drives our broader work on local-first data handling.
New finds, updated reviews, and the occasional take on what the security industry gets wrong.
Subscribe Free