Set up a Kubernetes cluster

KubeNDT runs on top of your own Kubernetes cluster: the backend reads the kubeconfig mounted from the host, so the active kubectl context is the cluster KubeNDT will manage. This page summarizes how to prepare a suitable cluster. For the full step-by-step procedure see the Cluster Bootstrap Guide in the KubeNDT repository.

Requirements

• Linux host (or Linux VMs for kubeadm).
• Kubernetes 1.25+ (ephemeral containers, which the toolbox needs) with kubectl configured and working. 1.30 or newer is recommended: pod execs go over WebSocket there, and an older API server falls back to SPDY, about a third slower per exec.
• Docker and Docker Compose (to run the KubeNDT services).
• Git (to clone the Meshnet CNI).
• Recommended lab resources: 4 vCPU, 8 GB RAM.

Choose a scenario

The guide covers three setups; pick the one that matches your needs:

Scenario	Tool	Best for
Single-node lab	Minikube	Quick functional validation
Two-node local cluster	kind	Reproducible multi-node testing
Two-node production-style	kubeadm	Closer-to-production deployments

Example: single-node with Minikube

minikube start --driver=docker --cpus=4 --memory=8192 --disk-size=30g \
  --extra-config=kubelet.allowed-unsafe-sysctls="net.ipv4.ip_forward,net.ipv6.conf.all.forwarding"
kubectl config current-context
kubectl get nodes -o wide

Why the allowed-unsafe-sysctls flag?

KubeNDT router and switch pods set net.ipv4.ip_forward and net.ipv6.conf.all.forwarding via pod sysctls, so they forward both families. Without this flag the kubelet rejects those pods with SysctlForbidden. On kubeadm clusters, put the allowedUnsafeSysctls list in the KubeletConfiguration that kubeadm manages rather than editing /var/lib/kubelet/config.yaml by hand, which a minor upgrade rewrites. The guide covers both a fresh install and an existing cluster.

Upgrading from a release before 1.6.0

An allowlist with only net.ipv4.ip_forward was enough until 1.6.0, which added the IPv6 sysctl. On such a cluster every router and switch pod stays in SysctlForbidden and the deploy times out: add net.ipv6.conf.all.forwarding to the allowlist on every node before upgrading.

Install the Meshnet CNI (required)

KubeNDT uses the Meshnet CNI for the L2/L3 overlays between nodes. The same installation works for every scenario:

git clone https://github.com/networkop/meshnet-cni.git
cd meshnet-cni/
kubectl apply -k manifests/base

Meshnet really is required

KubeNDT checks the Meshnet DaemonSet and shows its health in the dashboard. If the dataplane is missing, deploying or modifying a topology fails with 412 Precondition Failed rather than creating pods that never get wired. See the Meshnet dependency.

Validate the cluster

Run these checks after creating the cluster and installing Meshnet:

kubectl get nodes -o wide
kubectl get pods -A
kubectl get ds -A | grep -i meshnet
kubectl get crd | grep -E 'topologies.networkop.co.uk|network-attachment-definitions.k8s.cni.cncf.io'
kubectl api-resources | grep -i topology

You are ready when:

• All nodes are Ready.
• The Meshnet workloads are running.
• The topology CRD is registered and discoverable.

Link MTU

KubeNDT sets one MTU on every link of a namespace, 1450 by default. Meshnet builds a veth pair (1500) when both pods share a worker and a VXLAN tunnel (the worker NIC's MTU minus 50) across workers, and mixed values fail silently: TCP stalls on full-size segments and OSPF neighbours stay in ExStart. 1450 is the largest value that works on every placement of a 1500-byte underlay.

If your network carries more, set KUBENDT_LINK_MTU on the backend, for example 8950 on a 9000-byte jumbo underlay. Check the path between workers first with a DF ping (ping -M do -s 8972 <other worker>): a value a worker cannot carry fails the operation naming the link and the worker. The cluster guide lists every hop that has to allow it.

Once the cluster is ready, continue with Getting started to run KubeNDT and deploy your first topology.