Labs on Kubernetes¶

With clabernetes (c9s), a lab runs as pods on a Kubernetes cluster instead of containers on one machine. The apps show it like any other lab: the same topology file, the same canvas, terminals, and captures.

Two ways to reach a cluster¶

Only containerlab talks to clabernetes. Who runs containerlab, and with which Kubernetes identity, depends on how you reach the cluster:

Every app adds both the same way: Add lab host asks Where should labs run? and offers This computer (Docker or Podman), Kubernetes (your kubeconfig; containerlab runs on this computer), and API Server (clab-api-server, which may run Docker or Kubernetes). A choice that cannot work in the app stays in the list, greyed, with the reason and what to do instead.

Either way, Kubernetes RBAC decides what works, and the cluster's audit log names the identity.

Run labs from your machine¶

The app runs the containerlab CLI on your machine with -r c9s for deploy, destroy, inspect, apply, save, node start, stop and restart, logs, events, and capture. It never runs it with sudo, which would drop your kubeconfig. Terminals, browser ports, SSH, and node stats go straight to the cluster through the same kubeconfig and context.

You need:

• Linux, with containerlab 0.83.0 or later. See Platforms.
• A cluster with clabernetes 0.11.0 or later, installed with its lab roles (the ClusterRole clabernetes-lab-editor).
• A kubeconfig context for that cluster with the rights below.

Kubeconfig and context¶

The app and containerlab look for the kubeconfig in this order:

1. the lab host's own kubeconfig file, when you set one (a leading ~ is your home folder),
2. CLAB_KUBECONFIG,
3. KUBECONFIG, which may list several files,
4. ~/.kube/config.

The context is the lab host's kubeContext, else the kubeconfig's current context. The app hands both to containerlab as CLAB_KUBECONFIG and CLAB_KUBE_CONTEXT, so a cluster whose context lives in its own file works without KUBECONFIG in the app's environment.

Namespaces and rights¶

Pick a fixed namespace when you may only use one. Destroy with cleanup deletes a lab's own c9s-<lab> namespace; a fixed namespace stays.

Add a cluster¶

In VS Code and the desktop app:

The cluster shows next to This computer, with Kubernetes · <context> · one namespace per lab (or the namespace) under its name. Edit opens the same form; Remove forgets the cluster and leaves its labs running.

When you deploy a topology from this computer in VS Code and clusters exist, VS Code asks where: this computer or a cluster. It remembers the answer for that topology.

Check connection¶

Check connection, in the form and on every lab host on this computer, runs these checks and lists each with ✓, ✗, or – (not run, because an earlier one failed), what it found, and how to fix a failure:

1. containerlab is 0.83.0 or later. Development builds pass.
2. The kubeconfig loads and has the context.
3. The cluster answers.
4. clabernetes 0.11.0 or later is installed. When your account may not read its manager, the check passes and containerlab checks the version before it changes the cluster.
5. Your account may create Topologies, pods/exec and pods/portforward, read pods/log, and, with a namespace per lab, get, create, and delete namespaces: containerlab creates the lab's namespace on deploy and deletes it on Destroy (cleanup).

A missing right comes with the command a cluster admin runs, for the user the cluster sees you as, for example:

kubectl create rolebinding clab-lab-editor-alice --clusterrole clabernetes-lab-editor --user alice -n team-a

A lab host whose checks fail stays in the list with the failed checks, so you can fix or edit it.

Where it is kept¶

VS Code keeps a cluster in the containerlab.connections setting, next to the API Servers:

"containerlab.connections": [
  {
    "kind": "kubernetes",
    "kubeContext": "lab-prod",
    "kubeconfig": "~/.kube/lab-prod.yaml",
    "namespace": "team-a",
    "exposeType": "None",
    "imagePullSecret": "regcred",
    "label": "Lab prod"
  }
]

Only kind and kubeContext are required; see the settings reference. Entries with "kind": "local", "runtime": "c9s" from earlier versions keep working. containerlab.runtime (docker or podman) is the runtime of this computer only; containerlab.binaryPath names the containerlab to run. The cluster's page in the settings sets its deploy defaults, a default deploy timeout, and its own deploy arguments, capture method and Wireshark VNC image; settings it does not support, such as the container runtime and Edgeshark, are greyed out there with the reason.

The desktop app keeps clusters under clusters in ~/.config/Containerlab/local-backend.json and writes the file itself when you add, edit, or remove one:

{
  "clusters": [
    { "kubeContext": "lab-prod", "kubeconfig": "~/.kube/lab-prod.yaml", "namespace": "team-a", "label": "Lab prod" },
    { "kubeContext": "kind-dev", "persistence": false }
  ]
}

To make the app's own lab host a cluster instead of this computer's Docker, start it with CONTAINERLAB_DESKTOP_RUNTIME=c9s and CONTAINERLAB_DESKTOP_KUBE_CONTEXT; see the desktop settings. That host is set by the environment and cannot be edited in the app.

Web app¶

The browser never receives or uploads a kubeconfig, so Add lab host in the web app shows Kubernetes greyed, with how the server adds one. The web app normally reaches clusters through an API Server. For a personal setup its server can run labs itself: WEB_LOCAL_LABS=true with WEB_LOCAL_RUNTIME=c9s, and WEB_LOCAL_KUBECONFIG and WEB_LOCAL_KUBE_CONTEXT for the cluster (WEB_LOCAL_KUBE_NAMESPACE, WEB_LOCAL_C9S_EXPOSE_TYPE, WEB_LOCAL_C9S_PERSISTENCE and WEB_LOCAL_C9S_IMAGE_PULL_SECRET set the defaults). The cluster then shows in the list with its details and Check connection. In a pod without a kubeconfig, it uses the pod's ServiceAccount. Everyone who can open the app works with that one identity, without signing in. See the web settings.

Run labs through an API Server¶

With CLAB_RUNTIME=c9s, clab-api-server runs the labs of everyone who signs in on one cluster. Add it with Add lab host > API Server and sign in with your Linux account as on any API Server; you never get a kubeconfig, and the browser never talks to Kubernetes.

• The server holds one kubeconfig, for its own ServiceAccount, which may only set up lab namespaces and act as users.
• On a user's first deploy of a lab, the server creates the namespace c9s-<user>-<lab> and binds clabernetes-lab-editor in it to clab:<user>.
• Every lab action then runs as clab:<user> in the group clab:users; members of SUPERUSER_GROUP also get clab:admins. A user can do through the server what kubectl would let that user do.

Once you are signed in, the lab host shows what the server runs: API Server · c9s · cluster <cluster> · namespaces c9s-<user>-{lab}, and the server's deploy settings, Set by the API Server: ports exposed: None · node storage kept · no image pull secret. These are the server's policy: Deploy with options shows them read-only instead of fields the server would ignore, and so does the server's page in the settings, marked Set by the API Server, together with its default timeout, its Wireshark VNC image and pull policy, and Without a Topology resource: Off (the server deploys every lab with one). A server without the lab-host-info feature shows only API Server · c9s; one that does not report its runtime at all is older than c9s support and shows Update it to 0.10.0 or later.

To set up the server, its ServiceAccount, and the roles, see Running on Kubernetes in the clab-api-server repository.

Platforms¶

containerlab runs on Linux only, so a cluster on this machine needs Linux too, even though the lab runs elsewhere:

• Linux: VS Code and the desktop app.
• Windows: VS Code connected to WSL, or the Linux desktop app inside WSL.
• macOS: a Linux VM.

Elsewhere, Add lab host shows This computer and Kubernetes greyed with containerlab runs on Linux only, and how to go on: use an API Server, or open the window in WSL. Kubernetes lab hosts already configured are left out (VS Code says so in its output channel, the desktop app in its log). Reach the cluster through an API Server instead.

What changes on a cluster¶

• Nodes are pods. The node panel shows each node's namespace, pod, container, and exposed ports, and a kubectl exec command to copy; it names the lab host's context and kubeconfig file.
• Run with options offers the c9s deploy options: Namespace, Expose ports, No persistent volumes, Image pull secret, and Without a Topology resource, next to Timeout (containerlab's default on a cluster is 10m; raise it when big images pull slowly) and Max workers (Kubernetes requests sent in parallel). Each shows the cluster's default; a plain deploy uses the defaults. With one namespace per lab and no namespace set, Image pull secret says that the Secret must reach the new c9s-<lab> namespace. On an API Server the c9s options show read-only, as set by the server; Timeout is there when the server lists lab-lifecycle-timeout, with the server's default (10m on a cluster).
• Lab Settings → Deploy keeps these options for one lab on the cluster, so its redeploys, applies, and destroys use the same namespace, timeout, and storage. They live with the lab host, not in the topology file.
• What a cluster cannot do stays in the menus and editors, greyed out with the reason, so you see it exists but not here; what it does differently keeps working with a hint. Docker and Podman lab hosts show everything as before. See Greyed out on a cluster.
• Two things are left out instead of greyed out, since there is nothing of them on a cluster: Manage Images (the cluster's nodes pull images themselves; for a private registry set an image pull secret) and the lab host's CPU, memory, and disk stats. Node stats need metrics.k8s.io in the cluster.
• Packet captures come from clabernetes as one pcapng stream; see Labs on a Kubernetes cluster for where they open in each app. Wireshark VNC of a cluster on this machine runs its container on this machine's Docker, not in the cluster, and the app writes the capture into it: it needs Docker here and a wireshark-vnc-docker image with the capture pipe, but no permission to create pods.

Greyed out on a cluster¶

containerlab's c9s runtime and the clabernetes compiler refuse or ignore these, so the app greys them out on a Kubernetes lab host, used directly or through an API Server:

A field that already holds a value stays editable, so you can remove it. These work with a difference and keep a hint: subnets (the management network of clabernetes), port mappings (through a Kubernetes Service, without the host port), bind mounts (copied read-only from the machine that deploys, up to 950 KB each), labels (Kubernetes label values), network aliases (Service names), and exec commands (run as the container starts; one that fails does not stop the node).