Overview and Introduction
In today's highly connected world, we sometimes forget that there are many instances, especially in the government, where environments need to be intentionally isolated from external connectivity, creating what is known as an air-gapped or disconnected environment. In some cases, these environments may have external connectivity, but that connectivity is extremely unreliable or is limited in it's communication, creating what is known as a denied, disrupted, intermittent, and limited impact environment.
After working in these types of environments for years, most teams spent a significant amount of time developing custom code for moving artifacts between these environments. Unfortunately, this is never an easy process and extremely time consuming. Here at Rancher Government, we are very familiar with the need to improve the process of working in these environments and recently released a tool that is purpose built to simplify the airgap experience.
Rancher Government Hauler
Hauler simplifies the airgap experience by providing a straightforward approach that doesn't require operators to adhere to a specific workflow. It is designed as a standalone binary with zero dependencies. Hauler represents assets as artifacts and supports container images, helm charts, and files. Additionally, Hauler groups these artifacts into content and collections, allowing operators to easily fetch, store, package, and distribute these assets using declarative manifests or through the command line.
Hauler achieves this by storing it's content and collections as OCI Compliant Artifacts and even provides bootstrapping utilities such as an embedded registry and embedded fileserver.
To better understand the core concepts of Hauler and how they fit together, let's take a look at the diagram below:
Now that you've been introduced to the core concepts and basics of Hauler, let's take a look at the typical airgapped or disconnected workflow used when working with Hauler:
Testing out Hauler
As we briefly mentioned earlier, Hauler has zero dependencies, besides itself, so we don't have any prerequistes besides two servers to simulate our airgapped environment.
Let's designate one server as the connected server and one server as the disconnected server. For me, you will notice I'm the root user on my connected-server and my disconnected-server.
Let's start by downloading and installing Hauler! As you can see, there is an easy to use installation script!
# sudo to root
[rocky@connected-server hauler] sudo su
# install latest release
[root@connected-server hauler] curl -sfL https://get.hauler.dev | bash
[INFO] Hauler: Starting Installation...
- Version: v1.0.0
- Platform: linux
- Architecture: amd64
[INFO] Hauler: Starting Checksum Verification...
- Expected Checksum: 966fc0fedbc445447f261ecfd03e145b615e764303f7ab685a5e42300beae36b
- Determined Checksum: 966fc0fedbc445447f261ecfd03e145b615e764303f7ab685a5e42300beae36b
- Successfully Verified Checksum: hauler_1.0.0_linux_amd64.tar.gz
[INFO] Hauler: Successfully Installed at /usr/local/bin/hauler
- Hauler v1.0.0 is now available for use!
- Documentation: https://hauler.dev
Once we have successfully installed it, let's take a look at all the available commands:
[root@connected-server hauler] hauler --help
Airgap Swiss Army Knife
Usage:
hauler [flags]
hauler [command]
Available Commands:
completion Generates completion scripts for various shells
download Download OCI content from a registry and populate it on disk
help Help about any command
login Log in to a registry
serve Run one or more of hauler's embedded servers types
store Interact with hauler's embedded content store
version Print the current version
Flags:
-h, --help help for hauler
-l, --log-level string (default "info")
Use "hauler [command] --help" for more information about a command.Feel free to continue exploring the command line, but we're going to move onto adding some artifacts to our store. We haven't mentioned it yet, but Hauler uses stores to logically seperate artifacts into directories and the default name is store.
We're going to stick with using the command line for today, but if you would like to see an example of adding these same artifacts to the store using a manifest, feel free to view it below!
Let's run the following commands below:
# add a image... defaults to latest and docker.io
[root@connected-server hauler] hauler store add image neuvector/scanner
3:31PM INF added 'image' to store at [index.docker.io/neuvector/scanner:latest]
# add a image with a specific platform and with supply chain artifacts
# may not work for all users due to the specified registry
[root@connected-server hauler] hauler store add image rgcrprod.azurecr.us/longhornio/longhorn-ui:v1.6.0 --platform linux/amd64 --key carbide-key.pub
3:32PM INF signature verified for image [rgcrprod.azurecr.us/longhornio/longhorn-ui:v1.6.0]
3:32PM INF added 'image' to store at [rgcrprod.azurecr.us/longhornio/longhorn-ui:v1.6.0]
# add a helm chart with a specific version
[root@connected-server hauler] hauler store add chart rancher --repo https://releases.rancher.com/server-charts/stable --version 2.8.2
3:33PM INF added 'chart' to store at [hauler/rancher:2.8.2], with digest [sha256:27e742f51e66e32512509a95523bc9a531ec63f723c730b47685e7678cbc30d3]
# add a file and assign it a new name
[root@connected-server hauler] hauler store add file https://get.rke2.io --name install.sh
3:34PM INF added 'file' to store at [hauler/get.rke2.io:latest], with digest [sha256:0bffbf5fecf9dda70c112153a6ea90392cc2e67c55fa82cc7bb0679b03ef68e0]
cat << EOF >> hauler-manifest.yaml
apiVersion: content.hauler.cattle.io/v1alpha1
kind: Images
metadata:
name: hauler-content-images-example
spec:
images:
- name: neuvector/scanner:latest
- name: rgcrprod.azurecr.us/longhornio/longhorn-ui:v1.6.0
key: carbide-key.pub
platform: linux/amd64
---
apiVersion: content.hauler.cattle.io/v1alpha1
kind: Charts
metadata:
name: hauler-content-charts-example
spec:
charts:
- name: rancher
repoURL: https://releases.rancher.com/server-charts/stable
version: 2.8.2
---
apiVersion: content.hauler.cattle.io/v1alpha1
kind: Files
metadata:
name: hauler-content-files-example
spec:
files:
- path: https://get.rke2.io
name: install.sh
EOF# fetch the content from hauler manifest
[root@connected-server hauler] hauler store sync --files hauler-manifest.yaml
3:36PM INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Images] to store
3:36PM INF added 'image' to store at [index.docker.io/neuvector/scanner:latest]
3:36PM INF signature verified for image [rgcrprod.azurecr.us/longhornio/longhorn-ui:v1.6.0]
3:36PM INF added 'image' to store at [rgcrprod.azurecr.us/longhornio/longhorn-ui:v1.6.0]
3:36PM INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Charts] to store
3:36PM INF added 'chart' to store at [hauler/rancher:2.8.2], with digest [sha256:27e742f51e66e32512509a95523bc9a531ec63f723c730b47685e7678cbc30d3]
3:36PM INF syncing [content.hauler.cattle.io/v1alpha1, Kind=Files] to store
3:36PM INF added 'file' to store at [hauler/install.sh:latest], with digest [sha256:0a6317871b81a2fb0afe1369057aa69209c4668f9d359e38e79e7817f2a10107]
Once we've added all the above artifacts, we can verify and view all the content in our store by running the command below:
# view the content in the local hauler store
[root@connected-server hauler] hauler store info
+-------------------------------+-------+-------------+----------+----------+
| REFERENCE | TYPE | PLATFORM | LAYERS | SIZE |
+-------------------------------+-------+-------------+----------+----------+
| hauler/install.sh:latest | file | - | 1 | 24.7 kB |
| hauler/rancher:2.8.2 | chart | - | 1 | 15.0 kB |
| longhornio/longhorn-ui:v1.6.0 | image | linux/amd64 | 6 | 73.2 MB |
| | atts | - | 1 | 11.6 kB |
| | sbom | - | 1 | 2.5 MB |
| | sigs | - | 1 | 258 B |
| neuvector/scanner:latest | image | linux/amd64 | 4 | 174.3 MB |
| | image | linux/arm64 | 4 | 163.6 MB |
+-------------------------------+-------+-------------+----------+----------+
| TOTAL | 465.7 MB |
+-------------------------------+-------+-------------+----------+----------+
store and we are ready to save it as a haul, aka the compressed archive or tarball, by running the command below:# save and export the content in the local hauler store
[root@connected-server hauler] hauler store save --filename haul.tar.zst
3:40PM INF saved store [store] -> [/opt/hauler/haul.tar.zst]haul, we need to move it to the disconnected-server. I'm going to transfer the haul.tar.zst to my other server using scp.haul on the disconnected-server, let's run the command below to load the haul:# load and import the airgapped content to the new local hauler store
[root@disconnected-server hauler] hauler store load haul.tar.zst
3:43PM INF loading content from [haul.tar.zst] to [store]haul, let's verify and view the content in our store to ensure it all loaded correctly:# view the content in the local hauler store
[root@disconnected-server hauler] hauler store info
+-------------------------------+-------+-------------+----------+----------+
| REFERENCE | TYPE | PLATFORM | LAYERS | SIZE |
+-------------------------------+-------+-------------+----------+----------+
| hauler/install.sh:latest | file | - | 1 | 24.7 kB |
| hauler/rancher:2.8.2 | chart | - | 1 | 15.0 kB |
| longhornio/longhorn-ui:v1.6.0 | image | linux/amd64 | 6 | 73.2 MB |
| | atts | - | 1 | 11.6 kB |
| | sbom | - | 1 | 2.5 MB |
| | sigs | - | 1 | 258 B |
| neuvector/scanner:latest | image | linux/amd64 | 4 | 174.3 MB |
| | image | linux/arm64 | 4 | 163.6 MB |
+-------------------------------+-------+-------------+----------+----------+
| TOTAL | 465.7 MB |
+-------------------------------+-------+-------------+----------+----------+
As you can see, in a handful of commands, we now have all of the content from our connected-server, on our disconnected-server.
Bootstrapping Utilities
One more aspect of Hauler that really deserves to be highlighted. If you have worked in DevSecOps or Kubernetes or Docker space, you are probably very familiar with the difficult problem of boostrapping your environment(s). Not only does Hauler enable you to fetch, package and distribute your artifacts, but it enables you to serve your artifacts out to be used by your infrastructure or applications! I briefly mentioned it earlier, but Hauler does this with an embedded registry and embedded fileserver.
Let's head back to our disconnected-server and take a look by running the command below:
[root@disconnected-server hauler] hauler store serve --help
Expose the content of a local store through an OCI compliant registry or file server
Usage:
hauler store serve [flags]
hauler store serve [command]
Available Commands:
fileserver Serve the file server
registry Serve the embedded registry
Flags:
-h, --help help for serve
Global Flags:
--cache string Location of where to store cache data (defaults to $XDG_CACHE_DIR/hauler)
-l, --log-level string (default "info")
-s, --store string Location to create store at (default "store")
Use "hauler store serve [command] --help" for more information about a command.Embedded Registry
Now let's go ahead and server a registry by running the command below:
# serve the content as a registry from the hauler store
# defaults to :5000
[root@disconnected-server hauler] hauler store serve registry
3:49PM INF neuvector/scanner:latest
3:49PM INF longhornio/longhorn-ui:v1.6.0
3:49PM INF hauler/install.sh:latest
3:49PM INF hauler/rancher:2.8.2
3:49PM INF copied artifacts to [127.0.0.1:43273]
3:49PM INF starting registry on port [5000]
WARN[0003] No HTTP secret provided - generated random secret. This may cause problems with uploads if multiple registries are behind a load-balancer. To provide a shared secret, fill in http.secret in the configuration file or set the REGISTRY_HTTP_SECRET environment variable. go.version=go1.21.7 version=v3.0.0+unknown
INFO[0003] redis not configured go.version=go1.21.7 version=v3.0.0+unknown
INFO[0003] Starting upload purge in 38m0s go.version=go1.21.7 version=v3.0.0+unknown
INFO[0003] using inmemory blob descriptor cache go.version=go1.21.7 version=v3.0.0+unknownregistry is ready, on the default port of 5000, we can verify and view the available images with this awesome curl command:[root@disconnected-server hauler] curl -sfL localhost:5000/v2/_catalog | jq
{
"repositories": [
"hauler/install.sh",
"hauler/rancher",
"longhornio/longhorn-ui",
"neuvector/scanner"
]
}Embedded Fileserver
Now let's go ahead and server a fileserver by running the command below:
# serve the file content as a fileserver from the hauler store
# defaults to :8080
[root@disconnected-server hauler] hauler store serve fileserver
3:52PM INF copied artifacts to [store-files]
3:52PM INF starting file server on port [8080]fileserver is ready, on the default port of 8080, we can verify and view the available files with a curl command: # copy the content to a registry from the hauler store
# copies oci compliant artifacts
[root@disconnected-server hauler] hauler store copy registry://localhost:5000
3:55PM INF hauler/install.sh:latest
3:55PM INF longhornio/longhorn-ui:v1.6.0
3:55PM INF hauler/rancher:2.8.2
3:55PM INF neuvector/scanner:latest
3:55PM INF longhornio/longhorn-ui:v1.6.0
3:55PM INF copied artifacts to [localhost:5000]Existing Infrastructure?
What if you already have a registry or fileserver on your airgapped or disconnect environment? Well! Hauler does not have to server out a registry or fileserver. Instead we can use Hauler to copy the artifacts in our store to our registry or fileserver.
registry by running the command below:# copy the content to a directory from the hauler store
# copies non oci compliant artifacts
[root@disconnected-server hauler] hauler store copy dir://hauler-files
3:57PM INF copied artifacts to [hauler-files]What is Next?
Congratulations! You made it!! Hopefully, you realize how Hauler would be able to simplify and streamline your existing processes for fetching, packaging, validating, and distributing artifacts to airgapped, disconnected, or denied, disrupted, intermittent, and limited impact environments.
To learn more, please visit the GitHub Repository, check out the Official Documentation or contact our RGS team.
“This publication was prepared or accomplished by the author in a personal capacity. All opinions expressed by the author of this publication are solely their current opinions and do not reflect the opinions of Rancher Federal, Inc., respective parent companies, or affiliates with which the author is affiliated. The author's opinions are based upon information they consider reliable, but neither Rancher Federal, Inc., nor its affiliates, nor the companies with which the author is affiliated, warrant its completeness or accuracy, and it should not be relied upon as such.”