> For the complete documentation index, see [llms.txt](https://docs.uibakery.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.uibakery.io/on-premise/additional-configurations/self-hosted-ai-chat-setup/kubernetes.md).

# Kubernetes setup

Use the official OpenSandbox Helm chart. Bakery Sandbox API, Chat and PostgreSQL use plain Kubernetes YAML. For model and storage settings, see [Self-hosted AI chat](https://docs.uibakery.io/on-premise/additional-configurations/self-hosted-ai-chat-setup.md).

You need Linux/amd64 nodes, a CNI that enforces NetworkPolicy, permission to install CRDs and RBAC, and a default StorageClass for OpenSandbox’s 8 GiB volume, plus another 8 GiB if you install PostgreSQL in the cluster. Cluster policies must allow OpenSandbox’s init and egress containers. The examples use the `ui-bakery` namespace.

## 1. Prepare OpenSandbox

Use Helm 3 and `kubectl` with access to your cluster. Set the context:

```bash
KUBE_CONTEXT=YOUR_CONTEXT
```

Save the namespaces, storage and security settings as `opensandbox.yaml`:

```yaml
apiVersion: v1
kind: Namespace
metadata:
  name: opensandbox-system
---
apiVersion: v1
kind: Namespace
metadata:
  name: bakery-sandboxes
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: opensandbox-state
  namespace: opensandbox-system
spec:
  accessModes: [ReadWriteOnce]
  resources:
    requests: {storage: 8Gi}
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: bakery-opensandbox-workload
  namespace: opensandbox-system
data:
  batchsandbox-template.yaml: |
    spec:
      replicas: 1
      template:
        spec:
          restartPolicy: Never
          automountServiceAccountToken: false
          enableServiceLinks: false
          nodeSelector: {kubernetes.io/os: linux, kubernetes.io/arch: amd64}
          securityContext:
            seccompProfile: {type: RuntimeDefault}
          containers:
          - name: sandbox
            securityContext:
              runAsNonRoot: true
              runAsUser: 1001
              runAsGroup: 1001
              allowPrivilegeEscalation: false
              privileged: false
              capabilities: {drop: [NET_ADMIN]}
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: sandbox-isolation
  namespace: bakery-sandboxes
spec:
  podSelector: {}
  policyTypes: [Ingress, Egress]
  ingress:
  - from:
    - namespaceSelector:
        matchLabels: {kubernetes.io/metadata.name: opensandbox-system}
      podSelector:
        matchLabels: {app.kubernetes.io/name: opensandbox-server}
    ports:
    - {protocol: TCP, port: 44772}
    - {protocol: TCP, port: 18080}
  egress:
  - to:
    - namespaceSelector:
        matchLabels: {kubernetes.io/metadata.name: kube-system}
      podSelector:
        matchLabels: {k8s-app: kube-dns}
    ports:
    - {protocol: UDP, port: 53}
    - {protocol: TCP, port: 53}
  - to:
    - ipBlock:
        cidr: 0.0.0.0/0
        except: [0.0.0.0/8, 10.0.0.0/8, 100.64.0.0/10, 127.0.0.0/8, 169.254.0.0/16, 172.16.0.0/12, 192.168.0.0/16, 198.18.0.0/15, 224.0.0.0/4, 240.0.0.0/4]
    ports:
    - {protocol: TCP, port: 80}
    - {protocol: TCP, port: 443}
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: opensandbox-server-access
  namespace: opensandbox-system
spec:
  podSelector:
    matchLabels: {app.kubernetes.io/name: opensandbox-server}
  policyTypes: [Ingress]
  ingress:
  - from:
    - namespaceSelector:
        matchLabels: {kubernetes.io/metadata.name: ui-bakery}
      podSelector:
        matchLabels: {app: ui-bakery-sandbox-api}
    ports:
    - {protocol: TCP, port: 80}
```

The policy allows public HTTP/HTTPS; the egress sidecar restricts each sandbox to its own allowlist. Adjust the DNS selector if your cluster does not use `kube-system/kube-dns`. If cluster networks use publicly routable addresses, add those CIDRs to `except` before applying.

```bash
kubectl --context "$KUBE_CONTEXT" apply -f opensandbox.yaml
```

Generate the two sandbox keys once:

```bash
CLIENT_KEY=$(openssl rand -hex 32)
PROVIDER_KEY=$(openssl rand -hex 32)
printf '%s' "$CLIENT_KEY" | \
  kubectl --context "$KUBE_CONTEXT" -n ui-bakery create secret generic bakery-sandbox-client \
    --from-file=api-key=/dev/stdin --dry-run=client -o yaml | \
  kubectl --context "$KUBE_CONTEXT" -n ui-bakery apply -f -
for namespace in ui-bakery opensandbox-system; do
  printf '%s' "$PROVIDER_KEY" | \
    kubectl --context "$KUBE_CONTEXT" -n "$namespace" create secret generic opensandbox-provider \
      --from-file=api-key=/dev/stdin --dry-run=client -o yaml | \
    kubectl --context "$KUBE_CONTEXT" -n "$namespace" apply -f -
done
unset CLIENT_KEY PROVIDER_KEY
```

Keep these keys when updating. If you use a secret manager, use the same Secret names and data keys.

## 2. Install the official Helm chart

Download the pinned [official OpenSandbox chart](https://github.com/opensandbox-group/OpenSandbox/tree/531700ca18a7fa5fab6a2f25d6dc68a76eddab12/manifests/charts/opensandbox):

```bash
git clone --depth 1 --branch release-1.1.1 https://github.com/opensandbox-group/OpenSandbox.git
helm dependency build OpenSandbox/manifests/charts/opensandbox --skip-refresh
```

Save as `opensandbox-values.yaml`. This uses the official server image pinned below. Its sandbox containers run as non-root with privilege escalation disabled; upstream drops `NET_ADMIN`, but does not preserve `drop: [ALL]`.

```yaml
base:
  fastSandbox:
    namespaces: {createSystem: false, create: false}
    rbac: {create: false}
opensandbox-controller:
  controller:
    image:
      repository: opensandbox/controller
    resources:
      requests: {cpu: 50m, memory: 128Mi}
      limits: {cpu: 500m, memory: 512Mi}
opensandbox-server:
  server:
    image:
      repository: opensandbox/server
      tag: release-1.1.1@sha256:bcd5f264101262cebfa7e1ff9cd0424403e42f31a6596e4364cf90d8cb5c936b
    resources:
      requests: {cpu: 100m, memory: 256Mi}
      limits: {cpu: '1', memory: 1Gi}
    podSecurityContext:
      runAsNonRoot: true
      runAsUser: 1000
      runAsGroup: 1000
      fsGroup: 1000
      seccompProfile: {type: RuntimeDefault}
      sysctls:
      - {name: net.ipv4.ip_unprivileged_port_start, value: '80'}
    containerSecurityContext:
      allowPrivilegeEscalation: false
      readOnlyRootFilesystem: true
      capabilities: {drop: [ALL]}
    env:
    - name: OPENSANDBOX_SERVER_API_KEY
      valueFrom:
        secretKeyRef: {name: opensandbox-provider, key: api-key}
    volumes:
    - name: workload-template
      configMap: {name: bakery-opensandbox-workload}
    - name: state
      persistentVolumeClaim: {claimName: opensandbox-state}
    - name: temporary
      emptyDir: {sizeLimit: 128Mi}
    volumeMounts:
    - {name: workload-template, mountPath: /etc/opensandbox/bakery-template.yaml, subPath: batchsandbox-template.yaml, readOnly: true}
    - {name: state, mountPath: /var/lib/opensandbox}
    - {name: temporary, mountPath: /tmp}
  configToml: |
    [server]
    host = "0.0.0.0"
    port = 80
    max_sandbox_timeout_seconds = 86400
    [proxy]
    resolve_internal = true
    [runtime]
    type = "kubernetes"
    execd_image = "opensandbox/execd:v1.1.0"
    [store]
    type = "sqlite"
    path = "/var/lib/opensandbox/opensandbox.db"
    [kubernetes]
    kubeconfig_path = ""
    namespace = "bakery-sandboxes"
    workload_provider = "batchsandbox"
    image_pull_policy = "IfNotPresent"
    batchsandbox_template_file = "/etc/opensandbox/bakery-template.yaml"
    sandbox_create_timeout_seconds = 180
    [kubernetes.execd_init_resources]
    requests = { cpu = "50m", memory = "64Mi" }
    limits = { cpu = "250m", memory = "128Mi" }
    [egress]
    image = "opensandbox/egress:v1.1.7"
    mode = "dns+nft"
    requests = { cpu = "25m", memory = "32Mi" }
    limits = { cpu = "250m", memory = "256Mi" }
```

Bakery Sandbox API uses the server’s internal proxy, so this profile does not need a separate ingress gateway. The Helm chart itself is unchanged.

```bash
helm upgrade --install opensandbox OpenSandbox/manifests/charts/opensandbox \
  --kube-context "$KUBE_CONTEXT" --namespace opensandbox-system \
  --values opensandbox-values.yaml --wait --timeout 5m

kubectl --context "$KUBE_CONTEXT" apply -f - <<'YAML'
apiVersion: v1
kind: ResourceQuota
metadata:
  name: sandbox-quota
  namespace: bakery-sandboxes
spec:
  hard:
    requests.cpu: '4'
    requests.memory: 6Gi
    limits.cpu: '4'
    limits.memory: 6Gi
    pods: '6'
    count/batchsandboxes.sandbox.opensandbox.io: '3'
YAML
kubectl --context "$KUBE_CONTEXT" -n bakery-sandboxes wait \
  --for=jsonpath='{.status.used.count/batchsandboxes\.sandbox\.opensandbox\.io}' \
  resourcequota/sandbox-quota --timeout=60s
```

## 3. Deploy Bakery Sandbox API

Save as `bakery-sandbox-api.yaml`. The examples use `latest`. To pin a release, replace `latest` in the Bakery Sandbox API, runtime and Chat images with the same Bakery version. Adjust the namespace and ConfigMap names if yours differ.

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ui-bakery-sandbox-api
  namespace: ui-bakery
spec:
  replicas: 1
  strategy: {type: Recreate}
  selector:
    matchLabels: {app: ui-bakery-sandbox-api}
  template:
    metadata:
      labels: {app: ui-bakery-sandbox-api}
    spec:
      automountServiceAccountToken: false
      terminationGracePeriodSeconds: 120
      nodeSelector: {kubernetes.io/os: linux, kubernetes.io/arch: amd64}
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
        runAsGroup: 1000
        seccompProfile: {type: RuntimeDefault}
      containers:
      - name: ui-bakery-sandbox-api
        image: cruibakeryonprem.azurecr.io/cloud/bakery-sandbox-api:latest
        securityContext:
          allowPrivilegeEscalation: false
          readOnlyRootFilesystem: true
          capabilities: {drop: [ALL]}
        env:
        - {name: UI_BAKERY_SANDBOX_PROVIDER_ENDPOINT, value: 'http://opensandbox-server.opensandbox-system.svc:80'}
        - {name: UI_BAKERY_SANDBOX_INSTALLATION, value: bakery-byok}
        - {name: UI_BAKERY_SANDBOX_PROVIDER_QUALIFIED, value: 'true'}
        - name: UI_BAKERY_DOCKER_SANDBOX_IMAGE
          value: cruibakeryonprem.azurecr.io/cloud/bakery-sandbox-runtime:latest
        - name: UI_BAKERY_AZURE_CONTAINER_PREFIX
          valueFrom:
            configMapKeyRef:
              name: ui-bakery-configmap
              key: UI_BAKERY_AZURE_CONTAINER_PREFIX
              optional: true
        - name: UI_BAKERY_DOCKER_SANDBOX_API_KEY
          valueFrom:
            secretKeyRef: {name: bakery-sandbox-client, key: api-key}
        - name: UI_BAKERY_SANDBOX_PROVIDER_API_KEY
          valueFrom:
            secretKeyRef: {name: opensandbox-provider, key: api-key}
        resources:
          requests: {cpu: 100m, memory: 256Mi}
          limits: {cpu: '1', memory: 512Mi}
        ports: [{name: http, containerPort: 7071}]
        volumeMounts: [{name: temporary, mountPath: /tmp}]
        lifecycle:
          preStop:
            exec:
              command: ["sleep", "90"]
        startupProbe:
          httpGet: {path: /status, port: http}
          periodSeconds: 5
          timeoutSeconds: 20
          failureThreshold: 120
        readinessProbe:
          httpGet: {path: /status, port: http}
          periodSeconds: 15
          timeoutSeconds: 20
        livenessProbe:
          tcpSocket: {port: http}
          periodSeconds: 30
      volumes:
      - name: temporary
        emptyDir: {sizeLimit: 128Mi}
---
apiVersion: v1
kind: Service
metadata:
  name: bakery-sandbox-api
  namespace: ui-bakery
spec:
  type: ClusterIP
  selector: {app: ui-bakery-sandbox-api}
  ports: [{name: http, port: 7071, targetPort: http}]
```

```bash
kubectl --context "$KUBE_CONTEXT" apply -f bakery-sandbox-api.yaml
kubectl --context "$KUBE_CONTEXT" -n ui-bakery rollout status \
  deployment/ui-bakery-sandbox-api --timeout=180s
```

The defaults keep one agent and one build ready, each requesting 0.25 CPU and 1 GiB RAM, with a 1 CPU limit. If you increase the pools, also adjust `sandbox-quota`.

Agents can access npm and shadcn. For attachments, allow the storage download hostname as described in [Attachments and storage](https://docs.uibakery.io/on-premise/additional-configurations/self-hosted-ai-chat-setup.md#attachments-and-storage). Builds have no external network access.

## 4. Configure Chat

Add these values to your existing `ui-bakery-configmap`:

```yaml
data:
  UI_BAKERY_CHAT_OPERATING_MODE: self_hosted
  UI_BAKERY_CHAT_URL: http://chat:7070
  UI_BAKERY_CHAT_SERVICE_URL: http://chat:7070
  UI_BAKERY_CHAT_API_URL: https://YOUR_BAKERY_HOST/chat
  UI_BAKERY_DOCKER_SANDBOX: http://bakery-sandbox-api:7071
```

Save as `chat-secrets.yaml` and set your Anthropic key. Generate the database password and Chat secrets with `openssl rand -hex 32`. Use the same database password in both entries. For an external database, use its connection details as described in step 5. Keep the file private.

```yaml
apiVersion: v1
kind: Secret
metadata:
  name: ui-bakery-chat-secrets
  namespace: ui-bakery
type: Opaque
stringData:
  UI_BAKERY_JWT_CHAT_SECRET: <jwt-secret>
  UI_BAKERY_CHAT_SERVICE_SECRET: <service-secret>
  UI_BAKERY_CHAT_UID_SALT: <uid-salt>
  CHAT_DATABASE_PASSWORD: <database-password>
  UI_BAKERY_CHAT_DATABASE_URL: postgres://postgres:<database-password>@chat-db:5432/chat_db
  UI_BAKERY_AGENT_ANTHROPIC_KEY: <anthropic-api-key>
```

```bash
kubectl --context "$KUBE_CONTEXT" apply -f chat-secrets.yaml
```

Add these entries to the Bakery backend container’s `env`:

```yaml
- name: UI_BAKERY_JWT_CHAT_SECRET
  valueFrom:
    secretKeyRef: {name: ui-bakery-chat-secrets, key: UI_BAKERY_JWT_CHAT_SECRET}
- name: UI_BAKERY_CHAT_SERVICE_SECRET
  valueFrom:
    secretKeyRef: {name: ui-bakery-chat-secrets, key: UI_BAKERY_CHAT_SERVICE_SECRET}
```

## 5. Set up PostgreSQL

Choose an external PostgreSQL database or install one in the cluster.

### Use an external database

Create a dedicated database and a user with permission to create and update its tables. In `chat-secrets.yaml`, replace the connection URL:

```yaml
UI_BAKERY_CHAT_DATABASE_URL: postgres://USER:PASSWORD@HOST:5432/DATABASE
```

Use your provider’s TLS settings and URL-encode special characters in the credentials. Allow connections from Chat to the database.

Remove `CHAT_DATABASE_PASSWORD` from the Secret; it is only used by the in-cluster PostgreSQL container. Apply the Secret with the updated URL and continue to step 6. Do not apply `chat-db.yaml`.

### Install in the cluster

Save as `chat-db.yaml`. PostgreSQL uses an 8 GiB persistent volume and is accessible only inside the cluster.

```yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: chat-db
  namespace: ui-bakery
spec:
  accessModes: [ReadWriteOnce]
  resources:
    requests: {storage: 8Gi}
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: chat-db
  namespace: ui-bakery
spec:
  replicas: 1
  strategy: {type: Recreate}
  selector:
    matchLabels: {app: chat-db}
  template:
    metadata:
      labels: {app: chat-db}
    spec:
      automountServiceAccountToken: false
      containers:
      - name: postgres
        image: postgres:16.1
        args: ["postgres", "-c", "wal_level=logical"]
        env:
        - {name: POSTGRES_USER, value: postgres}
        - {name: POSTGRES_DB, value: chat_db}
        - {name: PGDATA, value: /var/lib/postgresql/data/pgdata}
        - name: POSTGRES_PASSWORD
          valueFrom:
            secretKeyRef: {name: ui-bakery-chat-secrets, key: CHAT_DATABASE_PASSWORD}
        ports: [{name: postgres, containerPort: 5432}]
        readinessProbe:
          exec:
            command: ["pg_isready", "-h", "127.0.0.1", "-U", "postgres", "-d", "chat_db"]
          periodSeconds: 5
          timeoutSeconds: 5
        resources:
          requests: {cpu: 100m, memory: 256Mi}
        volumeMounts:
        - {name: data, mountPath: /var/lib/postgresql/data}
      volumes:
      - name: data
        persistentVolumeClaim: {claimName: chat-db}
---
apiVersion: v1
kind: Service
metadata:
  name: chat-db
  namespace: ui-bakery
spec:
  selector: {app: chat-db}
  ports: [{name: postgres, port: 5432, targetPort: postgres}]
```

```bash
kubectl --context "$KUBE_CONTEXT" apply -f chat-db.yaml
kubectl --context "$KUBE_CONTEXT" -n ui-bakery rollout status deployment/chat-db --timeout=180s
```

PostgreSQL creates `chat_db` on first startup. Keep the PVC and database password when updating.

## 6. Deploy Chat

Save as `chat.yaml`:

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ui-bakery-chat
  namespace: ui-bakery
spec:
  replicas: 1
  selector:
    matchLabels: {app: ui-bakery-chat}
  template:
    metadata:
      labels: {app: ui-bakery-chat}
    spec:
      automountServiceAccountToken: false
      containers:
      - name: chat
        image: cruibakeryonprem.azurecr.io/cloud/chat:latest
        envFrom:
        - configMapRef: {name: ui-bakery-configmap}
        - secretRef: {name: ui-bakery-chat-secrets}
        env:
        - name: UI_BAKERY_DOCKER_SANDBOX_API_KEY
          valueFrom:
            secretKeyRef: {name: bakery-sandbox-client, key: api-key}
        ports: [{name: http, containerPort: 7070}]
        readinessProbe:
          tcpSocket: {port: http}
          periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
  name: chat
  namespace: ui-bakery
spec:
  selector: {app: ui-bakery-chat}
  ports: [{name: http, port: 7070, targetPort: http}]
```

```bash
kubectl --context "$KUBE_CONTEXT" apply -f chat.yaml
kubectl --context "$KUBE_CONTEXT" -n ui-bakery rollout status deployment/ui-bakery-chat --timeout=180s
```

Restart the Bakery backend, frontend and gateway after applying their settings. The gateway routes `/chat` to Chat; no separate Ingress is needed. Allow Chat to reach PostgreSQL, Anthropic and Bakery Sandbox API.

## 7. Verify

Check the API and sandbox pods:

```bash
kubectl --context "$KUBE_CONTEXT" -n ui-bakery exec deployment/ui-bakery-sandbox-api -- \
  node -e "fetch('http://127.0.0.1:7071/status').then(async r=>{console.log(await r.json());process.exit(r.ok?0:1)})"
kubectl --context "$KUBE_CONTEXT" -n bakery-sandboxes get pods
```

In Bakery, ask the agent to install a package, build an app and use an attachment if storage is configured.

Before using Chat, verify that the main sandbox container runs as UID 1001, drops `NET_ADMIN`, has privilege escalation disabled and has no service-account token. Confirm agent and build commands work, while access to other cluster services and cloud metadata is blocked. `UI_BAKERY_SANDBOX_PROVIDER_QUALIFIED=true` acknowledges this isolation profile; it does not perform these checks automatically.

Run one API replica and one OpenSandbox server replica. Preserve the keys, installation ID and provider volume when updating.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.uibakery.io/on-premise/additional-configurations/self-hosted-ai-chat-setup/kubernetes.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
