Blog

Installing RKE Government in Air-gap Environments

Written by Mike D'amato | December 3, 2020
There are many deployment environments where internet access is just not possible. RKE Government was designed from the ground up with cases like this in mind. Out of the box, RKE Government supports several ways of injecting its dependencies for easy installations and upgrades in air-gapped networks.

In this tutorial, we’ll walk through a simple example of preparing a host from scratch and deploying RKE Government configured to use only internally available dependencies without needing an internet connection.

Prerequisites

This tutorial targets Centos or RHEL 7. You will need two hosts in total. One host that has an active internet collection that is used to gather the dependencies, and one you are targeting for your offline installation. The host with an internet connection needs to have Docker installed.

We will be focusing on launching a single node cluster with both agent and server roles creating a fully functional Kubernetes cluster. However, this cluster can easily be expanded.

Be sure to read more about requirements in the RKE requirements documentation.

Dependency collection

Note: The dependencies are available for download on GitHub but we have included the manual steps below.

The only difference between doing this installation in an environment with an internet connection and doing it in an air-gapped environment is we need to gather a few dependencies first and extract them into their designated locations on your offline host. This consists of the RKE2 RPMs and their dependencies, container images, and some SELinux RPMs.

We have written a simple bash script that collects all the dependencies and zips them into a single file.

Make an empty folder to mount into your container.

  mkdir RKE_Dependencies
 

Run this docker command on your online host and it will grab and zip up everything for you (change the image to centos:7 if you’re planning on running on RHEL/CentOS 7).

  docker run --rm \
  -v $(pwd)/RKE_Dependencies:/mnt \
  -w /mnt centos:8 \
  /bin/bash -c \
  "curl -sfL https://gist.githubusercontent.com/mddamato/45efeb226b5109fb72a7a7289a943bf3/raw/offline_rke2_tutorial_dep_collection.sh | bash -"
 

If you’re feeling responsible, run
docker run -it --rm -v $(pwd)/RKE_Dependencies:/mnt -w /mnt centos:8 /bin/bash
first. Then inside the container run
curl -sfL https://gist.githubusercontent.com/mddamato/45efeb226b5109fb72a7a7289a943bf3/raw/offline_rke2_tutorial_dep_collection.sh -o offline_rke2_tutorial_dep_collection.sh
so you can review the script contents before you
chmod +x offline_rke2_tutorial_dep_collection.sh and
bash ./offline_rke2_tutorial_dep_collection.sh

All this script does is download the dependencies RKE might need and places them into a tar file so that we can later expand them to build up our RKE Government cluster.

The main components are:

  • RKE Container Images.
  • RKE RPMs, and SELinux dependencies.

RKE Government Images

RKE is container-based so we need to ensure all images are available without needing to dial out to a registry. We will need to gather all the container images so that later we can place them on the filesystem of your offline host. These images are already packaged up by Rancher, we just need to download them from the releases page on Github. Our script does this for us, but you can download these from RKE2 releases page on github. In this tutorial, we are going to place the tar file on the offline host directly. There is also an option to use an internal private registry instead, which would enable more simplicity in standing up multiple nodes.

The RKE Government RPMs

The script installs the yum repo configuration needed to download the RKEGov RPMs plus related dependencies. This roughly comes from the RPM Installation Method in the RKE Documentation. With an online host, we can install these repos, download the packages along with their dependencies, then generate an offline repo so that we can install them later. We will generate this and place that in our tar file with everything else.

SELinux Policies

On CentOS and RHEL machines SELinux is set to enforcing by default in most cases. Rancher builds RPM packages that install SELinux policies for RKE to work as anticipated with SELinux in enforcing mode. These are also included in the yum repo we talked about earlier. We need to gather the RPM for RKE Government SELinux policies, as well as the other SELinux policies required like container_selinux. Luckily with the repo configuration we already added, these will get gathered along with everything else by the script.

Install RKE Government

Now that you have your tar file, we are ready to Install RKE onto your offline host. Follow these steps to install RKE.

  • Copy the tar file from the previous steps to your offline host.
  • Once you have the tar file on your host, ssh into the host and move into the directory where it’s stored. All remaining actions in this section need to be done as the root user.
  • Unpack the dependency file, then remove it if you like.
  tar xzvf rke-government-deps-*.tar.gz
 
  • Move the images files into the designated images directory for RKE to consume.
  mkdir -p /var/lib/rancher/rke2/agent/images/
      cp rke2-images-canal.linux-amd64.tar.zst /var/lib/rancher/rke2/agent/images/
      cp rke2-images-core.linux-amd64.tar.zst /var/lib/rancher/rke2/agent/images/
 
  • Install a local yum repo referencing the RPMs we downloaded earlier. This will let us make yum install commands using files on the local system.
  mkdir -p /var/lib/rancher/yum_repos
      tar xzf rke_rpm_deps.tar.gz -C /var/lib/rancher/yum_repos
      cat > /etc/yum.repos.d/rke_rpm_deps.repo <<EOF
      [rke_rpm_deps]
      name=rke_rpm_deps
      baseurl=file:///var/lib/rancher/yum_repos/rke_rpm_deps
      enabled=0
      gpgcheck=0
      EOF
 
  • Install RKE Government RPM. This will install rke2-server, container-selinux, rke2-common, rke2-selinux, and any other dependencies that might be missing from your system.
  yum install --disablerepo=* --enablerepo="rke_rpm_deps" rke2-server
 

Configure and launch RKE Government

RKE Government is now installed but before we launch we need to make sure we add our desired configuration. To launch we just need to start the systemd unit and enable it for automatic starting when the machine reboots. All actions in this section need to be done as the root user.

  • Create a directory for RKE so that we can place configuration options.
  mkdir -p /etc/rancher/rke2
 
  • Add configuration to /etc/rancher/rke2/config.yaml. More configuration options are available on the server config section of the docs. For this example, we’re just adding some basic options.
  cat > /etc/rancher/rke2/config.yaml <<EOF
      selinux: true
      write-kubeconfig-mode: "0644"
      node-label:
      - "exampleNodeLabel=true"
      EOF
 
  • Enable the rke2-server.service systemd unit so that it starts up with every reboot.
  systemctl enable rke2-server
 
  • Start the RKE Government service.
  systemctl start rke2-server
 
  • Follow the logs, if you like. The cluster will take a few minutes to get initiated.
  journalctl -u rke2-server -f
 
  • You can see all the RKE containers running with ctr.
  /var/lib/rancher/rke2/bin/ctr -a /run/k3s/containerd/containerd.sock -n k8s.io c ls
 
  • You can also interact with your cluster using kubectl. You should see your node with Ready status. Note that you also have a kubeconfig located at /etc/rancher/rke2/rke2.yaml. This contains your initial administrator credentials, be sure to protect this file.
/var/lib/rancher/rke2/bin/kubectl --kubeconfig /etc/rancher/rke2/rke2.yaml get no

You can add /var/lib/rancher/rke2/bin and /usr/local/bin/ to your path so that you don’t need to specify the entire path every time. Additionally, set KUBECONFIG=/etc/rancher/rke2/rke2.yaml .

What’s next?

You now have a fully functioning RKEGovernment Kubernetes cluster.

A token for adding more nodes to your cluster has been created at /var/lib/rancher/rke2/server/node-token. Use this to register new nodes by providing these RKE configuration keys:

  server: https://<kubernetes_controlplane_host>:9345
      token: my-shared-secret
      tls-san:
      - <kubernetes_controlplane_host>.<domain>
 

You could follow this exact procedure for adding more nodes, yum install rke2-agent will allow you to install the agents.

Your Kubernetes cluster is ready! Begin deploying your cloud-native applications or follow some other tutorials including how to configure your air-gap container registries.

Conclusion

RKE Government is a fully conformant Kubernetes distribution that focuses on security and compliance for the U.S. Federal Government. It is tailored specifically for Government use cases, which frequently include Air-Gapped installations. As we’ve seen from this tutorial, anyone can take advantage of the simplicity of RKE Government. Be sure to look at other Rancher resources and tutorials to get the most out of Rancher.

“This publication was prepared or accomplished by the author in a personal capacity. All opinions expressed by the author of this publication are solely their current opinions and do not reflect the opinions of Rancher Federal, Inc., respective parent companies, or affiliates with which the author is affiliated. The author's opinions are based upon information they consider reliable, but neither Rancher Federal, Inc., nor its affiliates, nor the companies with which the author is affiliated, warrant its completeness or accuracy, and it should not be relied upon as such.”
View full post