local-ai-demo · JARS workshop

Local AI.
Zero External APIs.
Fully Offline Deployment.

Build a full-stack AI notes assistant with Next.js, FastAPI, PostgreSQL and a 523 MB model on Ollama. Then package every image and the model weights into one verified bundle and deploy it to a Linux server with no internet connection.

Start the workshop View on GitHub

Next.js
FastAPI
PostgreSQL
Ollama
qwen3:0.6b
Docker Compose

POST /api/ask · recorded on a laptop CPU (Docker Desktop)no network

QuestionWhere is the backup NAS?

Retrieved by PostgreSQL full-text search#1 Server rack layout (rank 0.8)

Answer from qwen3:0.6bThe backup NAS is in rack 4, slot 12. [#1]

retrieval 2 ms · inference 488 ms (model already loaded) · 122 tokens in · 18 tokens outA real response from this stack, kept as a static example.

Three promises, kept apart

They are often blurred together. They are different claims, enforced in different places, and each one is tested separately.

No external AI APIs

Every answer comes from a model running in your own container. No API keys, no cloud inference.

How: The backend has exactly one model client, for Ollama at http://ollama:11434. It has no pull method and no fallback provider. Ollama runs with OLLAMA_NO_CLOUD=1.

tests/test_llm.py, grep the backend for any other host

No internet at runtime

Once the bundle is on the server, nothing is downloaded: not images, not models, not fonts.

How: The installer loads images with docker load, restores the model from the bundle and starts Compose with --pull never. The site uses system fonts.

./scripts/airgap-test.sh --require-offline-host on a disconnected server

Strict network isolation

Even on a connected host, the backend, database and model cannot reach the internet.

How: They sit on a Docker network with internal: true. Only nginx has a published port. The model volume is mounted read-only.

./scripts/airgap-test.sh checks egress from inside each container

Architecture

Four containers, two networks, two volumes, one port.

Browser

You

any device on your network

edge network

web

nginx: static site + /api proxy

local-ai-demo-web · :8080

private network · internal: true · no route to the internet

backend

FastAPI: notes, retrieval, prompts

local-ai-demo-backend · :8000

postgres

notes + full-text index

volume: postgres-data

ollama

qwen3:0.6b on the CPU

volume: ollama-models (read-only)

One published port. Containers find each other by service name through Docker's DNS, never through localhost. The browser only ever talks to web.

Service	Image	Size	Role	Reachable from
web	local-ai-demo-web	62.7 MB	nginx (unprivileged): the static site, /api reverse proxy	your browser, port 3000
backend	local-ai-demo-backend	158 MB	FastAPI: notes, retrieval, prompts, readiness	web only (private network)
ollama	local-ai-demo-ollama	142 MB	Ollama 0.40.3, CPU-only build of the official image (5.57 GB)	backend only (private network)
postgres	postgres:18.6-alpine3.24	304 MB	PostgreSQL: notes and the full-text index	backend only (private network)

Sizes as reported by `docker images` (unpacked). The model (qwen3:0.6b, 523 MB) is not in any image: it lives in the ollama-models volume.

Connected build → air-gapped deployment

Everything that needs the internet happens on the build machine. The server only ever receives one archive.

online On the connected build machine

01Test. Unit and integration tests, lint, Compose validation./scripts/build-bundle.sh runs them first
02Build. Multi-stage images: web, backend, slim Ollamadocker compose build
03Pull. Third-party image: postgresdocker pull postgres:18.6-alpine3.24
04Download. The model, with Ollama verifying every blob./scripts/pull-model.sh
05Export. docker save | gzip per image; the model blobs as a tarimages/*.tar.gz, models/*.tar
06Describe. manifest.json (versions, architectures, sizes) + SHA256SUMSevery file is checksummed
07Package. One archive with a stable file order and timestampsdist/local-ai-demo-bundle-*.tar

offline On the offline Linux server

01Verify. Archive checksum, then every file inside itsha256sum -c *.tar.sha256
02Check. Linux, architecture, Docker, Compose, CPU, RAM, disk./scripts/check-prereqs.sh
03Load. docker load each image, compare image IDs to the manifestno registry is contacted
04Restore. Model blobs into the model volume, re-hashed on disksha256 of each blob
05Start. docker compose up --pull never --no-build --wait./scripts/offline-install.sh
06Prove. Health, a note in PostgreSQL, a real answer, a restart./scripts/smoke-test.sh
07Diagnose. Every failure prints the command that fixes it./scripts/diagnose.sh

Read the full offline deployment guide →

The workshop

11 modules, about 4 hours. Every command says what it does, why, and what you should see. Every module ends with exercises and troubleshooting.

Part 1 · Core: run and understand the stack

Part 2 · Advanced: ship it to an offline server

You will be able to

Explain every hop: browser → nginx → FastAPI → PostgreSQL and Ollama
Use Docker DNS and an internal network instead of localhost and open ports
Run retrieval before inference with nothing but SQL full-text search
Read Ollama timings, manifests and blobs, and verify a model by checksum
Shrink an image from 5.6 GB to 142 MB with a multi-stage build
Package, move, verify and deploy a stack on a server with no internet

Quick start

Prerequisites, measured on a real run of this stack.

Operating system	Linux x86_64. Tested in Docker Desktop (WSL 2) and an offline Docker-in-Docker simulation; not yet on bare metal. Docker Desktop on Windows or macOS works for the core workshop.
Docker	Engine 24+ with the Compose v2 plugin 2.24+ (tested: 28.2.2 and 28.5.2)
CPU	2 cores minimum, 4+ recommended. AVX2 helps; no GPU needed.
Memory	4 GB minimum, 8 GB recommended. Measured: about 1.2 GB for the whole stack with the model loaded (Ollama 1.11 GiB).
Disk	About 1.2 GB for loaded images and the model, plus the 774 MB bundle while installing. 4 GB free recommended.
Network	Only on the build machine. The server needs none.

Run it locally (connected machine)

git clone https://github.com/jars-demo/local-ai-demo.git
cd local-ai-demo
cp .env.example .env            # then set POSTGRES_PASSWORD
./scripts/pull-model.sh         # downloads qwen3:0.6b (523 MB)
docker compose up -d --build --wait
./scripts/smoke-test.sh         # health, a note, a real answer

Ship it to an offline server

./scripts/build-bundle.sh                        # on the connected machine
# carry dist/local-ai-demo-bundle-*.tar and its .sha256 to the server, then:
sha256sum -c local-ai-demo-bundle-*.tar.sha256
tar -xf local-ai-demo-bundle-*.tar && cd local-ai-demo-bundle-*/
./scripts/offline-install.sh

Then open http://localhost:3000. Windows and macOS work for the core workshop through Docker Desktop; the offline deployment targets Linux.

Where to go next

Source code

github.com/jars-demo/local-ai-demo

Module 01

Architecture and prerequisites

Offline deployment

The operator guide that ships in the bundle

The model

Qwen3 0.6B by the Qwen team, Apache 2.0

An educational project. A 0.6B model is small: it answers simple questions about short notes well and can be wrong. It is not a production platform and makes no availability or accuracy guarantees.

Local AI.Zero External APIs.Fully Offline Deployment.