Kubernetes/Setup/Multi-node cluster with kubeadm on Ubuntu

ProcedureUpdated 3 weeks ago

Multi-node cluster with kubeadm on Ubuntu

Bootstrap a real multi-node cluster on Ubuntu VMs with kubeadm and containerd: prepare nodes, init the control plane, install a CNI, join workers.

#kubernetes #setup #kubeadm #linux

Steps

1 control plane + N workers. Minimum 2 CPU / 2 GB RAM per node, unique hostnames, ports 6443 and 10250 open.

STEP 1 OF 7

Every node: disable swap, load overlay + br_netfilter, set sysctl

1. Every node: prerequisites

swap, modules, sysctl

SHELL

1sudo swapoff -a && sudo sed -i '/ swap / s/^/#/' /etc/fstab

2printf 'overlay\nbr_netfilter\n' | sudo tee /etc/modules-load.d/k8s.conf

3sudo modprobe overlay && sudo modprobe br_netfilter

4cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf

5net.bridge.bridge-nf-call-iptables = 1

6net.bridge.bridge-nf-call-ip6tables = 1

7net.ipv4.ip_forward = 1

8EOF

9sudo sysctl --system

2. Every node: containerd

containerd with systemd cgroups

SHELL

1sudo apt-get update && sudo apt-get install -y containerd

2sudo mkdir -p /etc/containerd

3containerd config default | sudo tee /etc/containerd/config.toml >/dev/null

4sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml

5sudo systemctl restart containerd && sudo systemctl enable containerd

3. Every node: kubeadm, kubelet, kubectl

pkgs.k8s.io repo

SHELL

1K8S=v1.34 # set to the current minor version (kubernetes.io/releases)

2sudo apt-get install -y apt-transport-https ca-certificates curl gpg

3curl -fsSL https://pkgs.k8s.io/core:/stable:/$K8S/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

4echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/$K8S/deb/ /" | sudo tee /etc/apt/sources.list.d/kubernetes.list

5sudo apt-get update && sudo apt-get install -y kubelet kubeadm kubectl

6sudo apt-mark hold kubelet kubeadm kubectl

7sudo systemctl enable --now kubelet

4. Control plane: init + CNI

kubeadm init

SHELL

1sudo kubeadm init --pod-network-cidr=10.244.0.0/16

2mkdir -p $HOME/.kube

3sudo cp /etc/kubernetes/admin.conf $HOME/.kube/config

4sudo chown $(id -u):$(id -g) $HOME/.kube/config

6# Flannel matches 10.244.0.0/16

7kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml

5. Workers: join

kubeadm join

SHELL

1# on the control plane (regenerates the command any time)

2kubeadm token create --print-join-command

4# on each worker

5sudo kubeadm join <cp-ip>:6443 --token <token> --discovery-token-ca-cert-hash sha256:<hash>

7# verify from the control plane

8kubectl get nodes -o wide

Nodes stay NotReady until a CNI is installed. For NetworkPolicy support use Calico or Cilium instead of Flannel.

Kubernetes/Setup/Multi-node cluster with kubeadm on Ubuntu

ProcedureUpdated 3 weeks ago

Multi-node cluster with kubeadm on Ubuntu

Bootstrap a real multi-node cluster on Ubuntu VMs with kubeadm and containerd: prepare nodes, init the control plane, install a CNI, join workers.

#kubernetes #setup #kubeadm #linux

Steps

1 control plane + N workers. Minimum 2 CPU / 2 GB RAM per node, unique hostnames, ports 6443 and 10250 open.

STEP 1 OF 7

Every node: disable swap, load overlay + br_netfilter, set sysctl

1. Every node: prerequisites

swap, modules, sysctl

SHELL

1sudo swapoff -a && sudo sed -i '/ swap / s/^/#/' /etc/fstab

2printf 'overlay\nbr_netfilter\n' | sudo tee /etc/modules-load.d/k8s.conf

3sudo modprobe overlay && sudo modprobe br_netfilter

4cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf

5net.bridge.bridge-nf-call-iptables = 1

6net.bridge.bridge-nf-call-ip6tables = 1

7net.ipv4.ip_forward = 1

8EOF

9sudo sysctl --system

2. Every node: containerd

containerd with systemd cgroups

SHELL

1sudo apt-get update && sudo apt-get install -y containerd

2sudo mkdir -p /etc/containerd

3containerd config default | sudo tee /etc/containerd/config.toml >/dev/null

4sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml

5sudo systemctl restart containerd && sudo systemctl enable containerd

3. Every node: kubeadm, kubelet, kubectl

pkgs.k8s.io repo

SHELL

1K8S=v1.34 # set to the current minor version (kubernetes.io/releases)

2sudo apt-get install -y apt-transport-https ca-certificates curl gpg

3curl -fsSL https://pkgs.k8s.io/core:/stable:/$K8S/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

4echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/$K8S/deb/ /" | sudo tee /etc/apt/sources.list.d/kubernetes.list

5sudo apt-get update && sudo apt-get install -y kubelet kubeadm kubectl

6sudo apt-mark hold kubelet kubeadm kubectl

7sudo systemctl enable --now kubelet

4. Control plane: init + CNI

kubeadm init

SHELL

1sudo kubeadm init --pod-network-cidr=10.244.0.0/16

2mkdir -p $HOME/.kube

3sudo cp /etc/kubernetes/admin.conf $HOME/.kube/config

4sudo chown $(id -u):$(id -g) $HOME/.kube/config

6# Flannel matches 10.244.0.0/16

7kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml

5. Workers: join

kubeadm join

SHELL

1# on the control plane (regenerates the command any time)

2kubeadm token create --print-join-command

4# on each worker

5sudo kubeadm join <cp-ip>:6443 --token <token> --discovery-token-ca-cert-hash sha256:<hash>

7# verify from the control plane

8kubectl get nodes -o wide

Nodes stay NotReady until a CNI is installed. For NetworkPolicy support use Calico or Cilium instead of Flannel.