Docker & AI Governance: Securing Containerised AI Workloads
Containerised AI workloads introduce governance blind spots that traditional container security doesn't address — from model supply chain risks to prompt injection and data leakage through containers.
The New Attack Surface
AI workloads running inside Docker containers introduce a unique governance challenge. Traditional container security focuses on image integrity, runtime isolation, and network policies — but AI workloads add prompt injection vectors, model exfiltration paths, and data leakage through container logs and volumes that existing policies don't address.
Image Provenance in the Age of AI Models
AI containers often bundle large model files (weights, embeddings, fine-tuned checkpoints) alongside application code. This creates several governance risks:
- Unverified model sources: A container image may pull a model from an untrusted registry or URL at build time, introducing poisoned weights.
- Stale base images: AI frameworks update frequently; running outdated PyTorch or TensorFlow versions means known vulnerabilities persist in production.
- Layer bloat: Model files stored directly in image layers are immutable once baked in, making it impossible to patch a discovered model vulnerability without rebuilding the entire image.
Best practice: Use external model registries (e.g., Hugging Face with verified publishers) and mount models at runtime via signed volumes or CSI drivers. Pin base images by digest, not tag.
Runtime Governance: Beyond Standard Policies
Standard container runtimes enforce security contexts, read-only filesystems, and resource limits. AI workloads need additional controls:
Prompt Injection Guardrails
Containers running inference APIs are vulnerable to prompt injection through user-supplied input. Governance policies must enforce:
- Input sanitisation at the container boundary (sidecar or network policy)
- Output validation against structured schemas
- Rate limiting per tenant/user to prevent abuse
Data Residency and Leakage
AI containers process sensitive data — PII, financial records, health information. Governance requires:
- Encryption of all volumes at rest (no plaintext model inputs on disk)
- Network policies blocking outbound connections except to approved model registries
- Audit logging of every inference request and response, retained per compliance requirements
- Ephemeral containers that discard all state between runs
RBAC for Model Access
Not every service needs access to every model. Implement policy-as-code (OPA, Kyverno) to enforce:
- Which services can pull which models
- Which fine-tuned models are approved for production
- Version pinning for audit trails
Supply Chain Security
The AI container supply chain involves multiple stages:
- Source: Model weights downloaded from Hugging Face, internal training outputs, or third-party providers.
- Build: Dockerfile that assembles the runtime environment, dependencies, and model artifacts.
- Sign: Cosign or Notary signs the image after build.
- Deploy: Admission controllers (OPA Gatekeeper, Kyverno) verify signatures and policy compliance before scheduling.
- Run: Runtime enforcement of resource limits, network policies, and data access controls.
Each stage needs verification. A single compromised model in step one invalidates the entire chain.
Practical Implementation
Example: Signed Image Pipeline
# Build with cosign
export COSIGN_EXPERIMENTAL=1
docker build -t ai-inference:v2.1 .
cosign sign --key cosign.key ai-inference:v2.1
# Deploy with Kyverno policy enforcement
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: require-signed-images
spec:
validationFailureAction: Enforce
rules:
- name: verify-image-signature
match:
any:
- resources:
kinds: ["Pod"]
verifyImages:
- imageReferences: ["ai-inference:*"]
keys: ["cosign.pub"]
Example: OPA Policy for Model Access
package ai.governance
deny[msg] {
input.request.user != "svc-ml-pipeline"
input.request.path == "/models/fine-tuned/*"
msg := "Only the ML pipeline service may access fine-tuned models"
}
Governance Checklist
Before deploying an AI workload in Docker:
- [ ] Image signed with cosign or equivalent
- [ ] Base image pinned by digest, not tag
- [ ] Model loaded from a verified external source at runtime
- [ ] All volumes encrypted at rest
- [ ] Network policies restrict outbound traffic
- [ ] Audit logging enabled for all inference calls
- [ ] Resource limits enforced (CPU, memory, GPU)
- [ ] RBAC policy defines which services access which models
- [ ] Input sanitisation in place against prompt injection
- [ ] Ephemeral containers preferred over persistent state
Conclusion
Dockerised AI workloads don't require a completely new governance framework — they extend existing container security practices with AI-specific controls. The key insight is that the model itself is part of the supply chain, and every inference call is a potential data exposure point. Treat both with the same rigour you'd apply to any production service.
Tags: Docker, AI Governance, Container Security, OPA, Kyverno, Supply Chain, MLOps, Compliance