Air-Gapped LLM Reference Architecture for Regulated Codebases (2026)

Run open-weights models on infrastructure you control, with default-deny egress, so AI coding and agent tooling can read your regulated codebase without a single byte reaching a cloud vendor. That is the whole point of an air-gapped LLM reference architecture. If your engineers and QA teams are already using AI copilots against SOC 2, ISO 27001, financial, healthcare, or GCC data-residency scopes, the model quality is not your problem. The data flow is.

This is a security and compliance decision, not a developer-tooling preference. Below is the threat model, a reference architecture across isolation tiers, and how each control turns into evidence your auditor can actually sample.

Why is a cloud LLM a governance problem for regulated code?

Because the context is the crown jewels. An AI coding assistant does not just see the line you are typing. To be useful it reads surrounding files, imports, config, sometimes whole repositories, and increasingly test data and production-like fixtures. An AI agent goes further: it opens files, runs commands, calls APIs, and reads whatever it needs to complete a task.

When that tooling points at a cloud model, all of that leaves your boundary. For a regulated codebase that creates four concrete exposures:

• Data egress. Source, secrets embedded in code, customer data in test fixtures, and internal architecture all transit to a vendor endpoint. On a data-residency scope, the destination region alone can be a violation.
• Vendor prompt and response logging. Many providers retain prompts and completions for abuse monitoring or model improvement. Even with a no-training commitment, retention windows and subprocessor access are hard to evidence to an auditor.
• Model supply-chain and provenance. You cannot attest to what a closed cloud model is, how it was built, or what changed between releases. For sovereign work that opacity is itself a finding.
• Tool-call blast radius. Once an agent has credentials to your repos, CI, and cloud APIs, a prompt-injection or a bad plan can cause damage without any network exfiltration at all.

The uncomfortable part: most of this happens without a code change and without a ticket. A developer installs an extension, an agent gets a broader token, and regulated context starts flowing. That is why AI Security Posture Management treats “what AI tooling touches regulated data, and where does it send that data” as a first-class inventory question.

What does the air-gapped threat model actually cover?

Air-gapping removes the cloud egress vector, but it does not remove the whole threat model. Be precise about what you are defending against so you do not build a wall around the model while leaving the credentials wide open.

Notice that only the top two threats are actually solved by air-gapping the model. The rest live at the agent and credential layer, which is where regulated deployments most often go wrong. AI Agent Runtime Protection exists precisely for that lower half of the table.

What does the reference architecture look like?

Isolation is a spectrum, not a switch. Pick the tier your data classification and auditor require, and do not over-build for a scope that only needs the middle option.

The building blocks are the same across tiers. Only the boundary moves.

Inference layer. Serve open-weights models with Ollama for single-node and workstation use, or vLLM for higher-throughput private-VPC serving behind an OpenAI-compatible API. Both let you download a model once, verify it, and serve it locally with no external dependency at runtime.

Model registry and provenance. Do not let inference hosts pull weights from the internet. Download open-weights models in a controlled staging step, verify the published checksum, record the source and version, and store the artifact in an internal registry. Production inference hosts pull only from that registry, and only models on a reviewed allowlist. Keep specific model and version choices generic and governed by the allowlist rather than hardcoded, since capable open-weights releases ship often.

Network and egress control. This is the load-bearing control for compliance. The inference host and any agent runtime sit in a segment with default-deny egress. The only permitted destinations are the internal model registry, the audit log sink, and whatever internal systems a given workload legitimately needs. Everything else is blocked and alerted. In a full air-gap there is no external route at all; in a private VPC you enforce this with egress firewall rules and continuous monitoring so a regression cannot silently open a path out.

Secrets and context hygiene. Assume any file the agent reads may end up in a prompt. Keep secrets out of source, inject them at runtime from a scoped secrets manager, and run pre-inference redaction to strip tokens and obvious PII from context before it reaches the model. This limits damage from both prompt injection and accidental logging.

Agent and tool layer. Give each agent least-privilege, short-lived credentials scoped to exactly the repos, CI jobs, or APIs its task requires. Gate state-changing tool calls (writes, merges, deploys, deletes) behind approval. Validate and sanitize model output before it is executed as a command or committed. This is the containment that keeps a prompt-injected agent from turning read access into a breach.

Audit and logging. Log every prompt, every completion, and every tool call to an internal, tamper-evident sink you own. That single decision produces most of your compliance evidence and most of your incident-response signal. When something goes wrong, this trail is the difference between a two-hour investigation and a two-week one.

Where does a QA or Playwright agent fit?

As one workload among many. A Playwright or QA agent that reads test suites, generates test data, and drives a staging environment is a perfectly good tenant of this architecture. It gets an inference endpoint from the private-VPC model, a scoped credential to the test environment only, redaction on any fixtures that contain real-like data, and the same tool-call logging as everything else. It is an example of the pattern, not the reason for it. The architecture has to hold for coding agents, security automation, data pipelines, and QA agents alike.

How do these controls map to compliance evidence?

The reason to build it this way is that every control emits an artifact an auditor can sample. You are converting an opaque vendor data flow into controls you own and can prove.

The mapping is intentionally framework-agnostic here. SOC 2 Trust Services Criteria and ISO 27001 Annex A controls both care about access control, change management, logging, and vendor risk, and this architecture speaks to all four. For data-residency regimes, the egress evidence and the “no external inference path” architecture diagram are usually the two artifacts that close the question fastest. Confirm the exact control identifiers with your auditor rather than quoting clause numbers, since scope and framework versions vary. [verify]

What should you avoid?

A few failure patterns show up repeatedly in regulated deployments:

• Air-gapping the model but not the agent. The most common mistake. You isolate inference, then hand the agent a broad personal-access token to every repo. The blast radius is now credentials, not network. Scope and log everything the agent can touch.
• Pulling weights straight from the internet at deploy time. That reintroduces the supply-chain vector you were trying to close. Stage, verify, register, then serve.
• Trusting a vendor “no-training” checkbox as your control. A contractual promise is not technical evidence. If the data still leaves your boundary, you own the residual risk and the audit burden.
• No output validation. Treating model output as safe to execute or commit turns prompt injection into remote code execution against your own systems.
• Logging to a sink the agent can reach and rewrite. Your audit trail has to be outside the blast radius or it is worthless during an incident.

The takeaway

If AI tooling is touching your regulated codebase, the question is not whether the model is good enough. It is whether you can prove that regulated source, test data, and secrets never left your boundary, and that a misbehaving agent could not have done damage even if they had. An air-gapped LLM reference architecture with open-weights models on Ollama or vLLM, default-deny egress, verified model provenance, scoped agent credentials, and full audit logging answers both questions with evidence, not vendor promises.

Start by inventorying what AI tooling reads and where it sends context. Then pick the isolation tier your data classification actually requires, and build the egress, provenance, and audit controls first. The model choice is the easy part.