Kubernetes Cluster with Kubespray

Minikube is all well and good for developing Kubernetes pods and deployments and testing them locally. But how can you install a real Kubernetes cluster in your own data center?

Of course, this can be done the hard way or via the installation with kubeadm. Especially the first variant is very complex and error-prone.

The installation is particularly easy with kubespray. This is an Ansible-based solution that creates a Kubernetes cluster with certain predefined parameters.

As a starting point, you should have the following system:

• At least 3 hosts on which Kubernetes is to run (3x Node with master and worker role).
• A bastion host from which Ansible is run

The hosts can be virtual machines, of course.

To create a cluster with kubespray, proceed as follows:

Set up SSH key access from Bastion host to all nodes
Install Ansible and Git on the Bastion host
Clone the Kubespray repository
Prepare inventory files for Kubespray
Run Ansible Playbook

Set up SSH key access

To do this, an SSH keypair is first created on the Bastion host:

ssh-keygen

Here the default values can be confirmed and no password should be assigned for the key. After that the private and public key is located under ~/.ssh/id_rsa und ~/.ssh/id_rsa.pub.

Now the public key in id_rsa.pub must be distributed to all nodes. To do this, a file ~/.ssh/authorized_keys is created on each node if it does not already exist. The content from id_rsa.pub can simply be copied into it.

Afterwards, the SSH access without password, with the key from the Bastion host to all other nodes should be tested once.

Installing Ansible and Git on the Bastion Host

To install Ansible on a CentOS 7 server, the appropriate repository must be enabled.

sudo yum install centos-release-ansible-29.noarch

Ansible can then be installed.

sudo yum install ansible.noarch

Git can be installed as well.

sudo yum install git.x86_64

After that, python-pip must also be installed.

sudo yum install python3-pip

Cloning the kubespray repository

Now the Git repository of kubespray must be cloned. To do this, copy the link from the GitHub page, or execute the following directly:

git clone https://github.com/kubernetes-sigs/kubespray.git

Perpare installation

When that’s done, we navigate to the directory:

cd kubespray

After that, we need to install the prerequisites for the Ansible Kubernetes installation using kubespray.

sudo pip3 install -r requirements.txt

Then we copy the sample inventory into a new inventory, which we call „mycluster“.

cp -rfp inventory/sample inventory/mycluster

Now we create the inventory, according to our host IP addresses.

declare -a IPS=(10.90.1.11 10.90.1.12 10.90.1.13 10.90.1.14 10.90.1.15)
CONFIG_FILE=inventory/mycluster/hosts.yaml python3 contrib/inventory_builder/inventory.py ${IPS[@]}

Now we need to change one more value in the „inventory/mycluster/group_vars/k8s-cluster/k8s-cluster.yml“ if we use CentOS 7 hosts.

vi inventory/mycluster/group_vars/k8s-cluster/k8s-cluster.yml

We search for the entry „kube_proxy_mode: ipvs“ and change it to

kube_proxy_mode: iptables

Install cluster

Now we can start with the cluster installation. We run the Ansible playbook from the Bastion host.

ansible-playbook -i inventory/mycluster/hosts.yaml --become cluster.yml

After that, the playbook should run through. This can take up to 45 minutes. Depending on how many hosts are configured and how fast the hardware is.

Manage Kubernetes clusters

To be able to manage the cluster with kubectl, we have to set the config file. For this we connect to the first master, in my case „node-11“ and copy the Kubernetes config to „~/.kube/config“.

sudo cat /etc/kubernetes/admin.conf >> ~/.kube/config

After that we can manage the cluster via kubectl.

kubectl get nodes
kubectl get pod -n kube-system