Skip to main content

Concepts

This page explains the core ideas behind KubeNDT. They map directly onto the API Reference, so understanding them makes the endpoints self-explanatory.

Topology​

A topology is a declarative JSON description of a virtual network:

  • Nodes - each node is a container image (a host, router, switch, or any image you like) deployed as a Kubernetes StatefulSet, optionally with multiple replicas.
  • Links - point-to-point connections between node interfaces, realized as L2/L3 overlays via the Meshnet CNI.

An abbreviated topology looks like this:

{
  "nodes": [
    {"name": "pc", "image": "alpine", "type": "host", "replicas": 4},
    {
      "name": "r1", "image": "quay.io/frrouting/frr:10.7.1",
      "type": "router", "driver": "FRRRouterDriver"
    }
  ],
  "links": [
    {
      "node": "pc-0", "localIntf": "eth1",
      "localIp": ["10.0.1.11/24", "fd00:1::11/64"],
      "peerNode": "r1-0", "peerIntf": "eth1",
      "delay": "10ms", "bandwidth": "50mbit"
    }
  ]
}

A link can carry delay, jitter, loss and bandwidth (tc units, delay one-way per direction), which the platform applies on both ends once the link is wired. See the reference topologies for complete, runnable examples.

Addresses are IPv4 or IPv6, and both ends of a link share a family. localIp and peerIp take one address or an array, for dual stack or a second address: the first is wired with the link and the platform sets the rest. NAT stays IPv4 only.

Every link gets the same MTU once it is wired, 1450 by default (KUBENDT_LINK_MTU on the backend), so throughput and OSPF adjacencies don't depend on whether Meshnet built a veth or a VXLAN. A link can declare its own mtu. Cluster setup explains the default and how to go higher.

Topologies are reproducible: you can save and re-import the JSON to recreate the exact same network.

The Meshnet dependency​

Links are wired by the Meshnet CNI, so KubeNDT tracks whether its dataplane is actually running and reports it as ok, degraded, missing or unknown - cluster-wide and per Kubernetes node, both in the dashboard and on the cluster and pods API responses.

Because a missing dataplane means topology objects get created but never wired - a silent failure - deploying or modifying a topology returns 412 Precondition Failed (reason: meshnet_missing) instead of leaving pods unwired. Only missing blocks: degraded and unknown pass through. Clearing a topology and deleting a namespace are always allowed, and ?force=true overrides the check.

A node can pin its pods with nodeSelector (labels a worker must carry) or nodeName (one worker, by name) in the topology JSON. Both are validated against the cluster before anything is created, so a typo fails fast with a 400 instead of pods left Pending. Placement applies to every replica of the node: with nodeSelector the scheduler still spreads them over the matching workers, with nodeName they all land on that worker. To pin instances to different workers, declare them as separate nodes.

Where the two ends of a link land decides how Meshnet realizes it: a veth pair when both pods share a worker, a VXLAN tunnel across workers. The link panel in the dashboard shows it, and GET /network/links/{namespace} lists every link with the worker behind each endpoint and its realization.

Nodes also accept terminationGracePeriodSeconds. The platform default is 2 s instead of the Kubernetes 30 s, since emulated nodes are stateless and a shell or a sleep infinity as the main process ignores SIGTERM. Raise it per node for workloads that need an orderly shutdown.

Namespaces​

Every topology lives in a Kubernetes namespace labelled kubendt/enabled=true. Namespaces provide isolation: you can run multiple independent topologies side by side, each with its own nodes, links, and configuration. The API exposes namespace lifecycle (create / list / delete) and an aggregated operational summary per namespace.

Drivers and capabilities​

KubeNDT abstracts network configuration behind a driver and capability system. Instead of issuing image-specific commands, you declare what you want (assign this IP, add this route, enable OSPF, set up NAT…) and the driver for that node type translates it into the right commands.

  • A driver knows how to operate a particular node type (e.g. a plain Linux host, an FRRouting router, or a VyOS appliance).
  • A capability is a declarative action a driver supports (L2 interface ops, L3 addressing/routing, routing protocols, NAT…).

This is why the same network/configure request works across heterogeneous nodes - the declarative intent is constant; the driver fills in the details. Every action is validated before it runs: a missing or malformed field fails the action with the field named, and nothing reaches the pod. GET /drivers lists each driver's actions with their parameters. Traffic control and custom commands are not tied to a driver and work on every node.

A driver tied to an image can also prepare it: an FRR router on the stock image needs no startup commands, since its driver enables the daemons behind its capabilities (OSPFv2, OSPFv3, RIP) and starts FRR.

To support a new node type, see writing a driver.

Traffic control and throughput​

Any interface of any node can be shaped, whether or not its image ships tc: KubeNDT falls back to a toolbox container injected into the pod. A shaping action takes a netem (delay, jitter, loss with an optional bursty model, duplicate, corrupt, reorder, rate), a tbf, or an htb class tree that splits a link rate into classes with their own matches and leaf queue. A links section in the configure request shapes a link by one of its ends, mirrored on the peer or not, and GET /pods/tc/{namespace}/{pod} reads the state back.

To measure the result, GET /iperf/run/{namespace}/{pod} runs an iperf3 test from a node toward an address in the namespace, TCP or UDP, with the server started on the destination node or one the lab already runs. The dashboard runs the same test from the node menu and plots one bar per second. Both the throughput test and the traceroute take IPv6 targets, except TCP traceroute, which stays IPv4. The all-features example walks through shaping a link and watching the bandwidth drop.

Configuration and persistence​

Operations applied through the configuration API are persisted per pod and replayed whenever KubeNDT recreates that pod: the Restart action, a modify that restarts a neighbour, or the heal pass that repairs interfaces after an operation. Meshnet also recreates the interfaces on the pod's neighbours, so their persisted operations on those interfaces (bridge membership, addresses, routes, qdiscs) are re-applied as well.

The history keeps the last word on each piece of state, so a set, its undo and the same set again all run. An action sent with options.persist_history: false runs once and is never replayed. The link MTU and the extra addresses of a link are not operations at all: they belong to the topology and the platform writes them again after every wiring.

Replay only covers recreations that KubeNDT drives. A pod recreated behind its back (kubectl delete pod, an eviction, a worker failure) comes back with the CRD-declared addresses only: trigger a Restart on it to run the full recovery. The history is dropped when the namespace, the topology or the node is deleted.

The configuration applied in a namespace can be exported as a configure request (GET /network/export-conf/{namespace}, or Export network conf in the dashboard), ready to load into another namespace. It does not carry the MTU or the extra addresses, which travel with the topology export.

note

Manual changes made directly inside a pod (ad-hoc shell commands not sent through the KubeNDT API) are intentionally not tracked for replay. Persist state by applying changes through the API, mounted config files, or startup scripts.

See Operations for what survives a backend or cluster failure and what stays manual.

Files and mounts​

Each namespace has a file manager: upload or write text files there and mount them into nodes at a given path. Files mount read-only as ConfigMaps, or as Secrets when flagged sensitive, and pods need a restart to see edits. Size cap is 1 MiB, UTF-8 text only. Full model, API surface and caveats in the file manager guide.

Authentication​

The dashboard and API are gated by a password login (which opens a browser session) plus revocable API tokens for scripts and CI. Auth can be disabled on trusted networks. See Getting started to set it up.

The dashboard and the API​

Everything you can do in the React dashboard is backed by the REST API. The dashboard is one client; your scripts and CI can be another. The API Reference documents every endpoint, grouped by area:

auth · cluster · namespaces · pods · network · drivers · files · shell · capture · trace · iperf · kubeconfig · system