Kubernetes Cluster Creation with Kubeadm on RedHat 9 Derivatives

Note: I have tested this guide on CentOS 9, Rocky Linux 9.2, and Alma Linux 9.2.

Kubernetes is an open-source container orchestration platform that automates the deployment, scaling, and management of containerized applications. It provides a flexible and scalable solution for managing modern workloads in production environments. In this guide, we will walk you through the step-by-step process of setting up a Kubernetes cluster using kubeadm on Red Hat and CentOS.

Step 1: Setup Hostnames

Before starting the Kubernetes cluster setup, it is essential to ensure that the IP addresses of all the servers (Master and Workers) are properly mapped to their hostnames in the /etc/hosts file. This step ensures smooth communication among the nodes in the cluster. Open the /etc/hosts file on each server and add the following entries:

vi /etc/hosts

192.168.100.5  Master
192.168.100.6  Worker-1
192.168.100.7  Worker-2

Make sure to replace the IP addresses and hostnames with the actual values of your servers. This step will prevent any hostname resolution issues during the Kubernetes cluster setup.

Step 2: Update and Configure the Servers

Before we start installing the required packages, let’s update the package manager cache and install any pending updates:

dnf makecache --refresh
dnf update -y
reboot

The servers will reboot to apply the updates.

Step 3: Configure SELinux

Next, we need to configure some system settings for Kubernetes to work properly. Let’s start by disabling SELinux temporarily and modifying the configuration file to set SELinux to permissive mode:

setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

Step 4: Configure Kernel Modules

Now, we need to prepare the servers for Kubernetes installation and configure essential kernel modules. These steps are crucial to ensure proper functioning and communication within the Kubernetes cluster.

Load the necessary kernel modules required by Kubernetes:

modprobe overlay
modprobe br_netfilter

To make these modules load at system boot, create a configuration file:

cat > /etc/modules-load.d/k8s.conf << EOF
overlay
br_netfilter
EOF

Step 5: Configure Sysctl Settings

Next, we’ll set specific sysctl settings that Kubernetes relies on:

cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF
 
sysctl --system

These sysctl settings enable IP forwarding and enable bridged network traffic to pass through iptables. This ensures that Kubernetes networking functions as expected.

Step 6: Disable Swap

Kubernetes requires that swap be disabled on all cluster nodes to ensure optimal performance and stability. Follow these steps to disable swap on each server:

swapoff -a
sed -e '/swap/s/^/#/g' -i /etc/fstab

Step 7: Install CRI (Containerd)

In this step, we will install Containerd on our servers. This container runtime is essential for managing and running containers, which are the building blocks of Kubernetes applications.

What is Containerd?

Containerd is an industry-standard container runtime that provides the core functionality for managing containers on the host system. It is designed to be embedded into higher-level container systems, such as Docker, Kubernetes, and others.

1) Add Docker CE Repository:

dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
dnf makecache

2) Install Containerd.io:

dnf install -y containerd.io

3) Configure Containerd:

The configuration file is located at /etc/containerd/config.toml. We’ll make a small adjustment to enable Systemd Cgroup support.

sudo mkdir /etc/containerd
sudo sh -c "containerd config default > /etc/containerd/config.toml"
sudo sed -i 's/            SystemdCgroup = false/            SystemdCgroup = true/' /etc/containerd/config.toml

Enable and restart the containerd service:

systemctl enable --now containerd.service
sudo systemctl restart containerd.service

Step 8: Configure Firewall Rules

We need to allow specific ports used by Kubernetes components through the firewall. Execute the following commands to add the necessary rules:

firewall-cmd --permanent --add-port={6443,2379,2380,10250,10251,10252,5473}/tcp
firewall-cmd --reload

Step 9: Install Kubernetes Components

Next, we’ll install Kubernetes components, including kubelet, kubeadm, and kubectl.

Create the Kubernetes repository configuration file:

cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v1.28/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v1.28/rpm/repodata/repomd.xml.key
EOF

Refresh the package cache and install components:

dnf makecache
dnf install -y kubelet-1.28.3-0 kubeadm-1.28.3-0 kubectl-1.28.3-0 --disableexcludes=kubernetes
systemctl enable --now kubelet.service

Note: The above steps are applicable to both master and worker nodes. The following steps are only applicable to master nodes.

Step 10: Initialize the Kubernetes Control Plane

Now it’s time to initialize the Kubernetes control plane on the master node:

sudo kubeadm config images pull
sudo kubeadm init --pod-network-cidr=10.10.0.0/16

The kubeadm init command will take some time to complete. Once it’s done, you’ll see a message with instructions on how to join worker nodes to the cluster.

Step 11: Copy Configuration File to User’s Directory

On the master node, copy the configuration file to the user’s directory:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Step 12: Install CNI (Calico)

To enable inter-pod communication, we’ll use Calico.

kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.26.1/manifests/tigera-operator.yaml
curl https://raw.githubusercontent.com/projectcalico/calico/v3.26.1/manifests/custom-resources.yaml -O
sed -i 's/cidr: 192\.168\.0\.0\/16/cidr: 10.10.0.0\/16/g' custom-resources.yaml
kubectl create -f custom-resources.yaml

Step 13: Join Worker Nodes

Get the Join Command on the Master Node:

kubeadm token create --print-join-command

Join Worker Nodes:

On each worker node, execute the join command obtained from the previous step:

kubeadm join 10.10.0.1:6443 --token abcdef.1234567890abcdef \
    --discovery-token-ca-cert-hash sha256:1234...

Verify Worker Node Joining:

Back on the master node, you can verify that the worker nodes have successfully joined:

kubectl get nodes

Conclusion

Congratulations! You have successfully set up a Kubernetes cluster using kubeadm on Red Hat and CentOS derivatives. Kubernetes provides a powerful platform for deploying and managing containerized applications at scale.

Happy Kubernetes clustering!