I wanted to make a standalone Rust binary that would run on a barebones Docker container (basically a chroot jail) that contained only that single file. I discovered it was not that straightforward.
According to my shallow understanding of Rust, one of its promises is that it compiles code into a single binary file that does not require a runtime environment or VM, unlike interpreted languages that compile down to bytecode such as Python and Java. Rust achieves this through a process called static linking, wherein the included Rust libraries and dependencies are all built into the compiled binary. In contrast, dynamic linking is a process in which a compiled binary points to shared system libraries that live on the operating system's filesystem, separate from the binary itself.
Rust offers static linking as one of its benefits. However, it is unfortunately not that simple. According to Rust's documentation,
"Pure-Rust dependencies are statically linked by default so you can use created binaries and libraries without installing Rust everywhere. By contrast, native libraries (e.g. libc and libm) are usually dynamically linked..."[1].
In other words, only Rust dependencies get statically linked by default; all of the non-Rust libraries like C libraries are dynamically linked. In most scenarios, this dynamic linking is not a problem. After all, the system that you copy your compiled executable to will likely have the required libraries. This is not the case for a barebones docker container, as we will see.
Docker containers come in a variety of flavors, most of which are based on lightweight Linux distros. There are containers for Ubuntu, Debian, and Alpine Linux, to name a few. Despite these containers being tiny operating systems unto themselves, most introductory readings about containers will tell you that treating a container like a virtual machine is an anti-pattern. VMs are meant to be fairly long-lived, emulate hardware, and they come with their own kernels. Contrast that with containers, which are meant to be ephemeral, emulate the operating system rather than hardware, and rely on the container server's kernel. Therefore, I was determined to make Docker "put its money where its mouth is" by spinning up a container that was a glorified chroot evironment whose only outside dependency was the kernel.
Using Rust's cargo command, one can make a hello world binary with just a few commands:
brenden$:> cargo new hello
brenden$:> cd hello
brenden$:> cargo build --release
The resultant binary will work on another computer of the same type as the one where it was compiled. If I compile it on macOS X Mojave, I can give just the opaque binary to someone else running Mojave and be pretty sure it will run. Because of this assurance, I had never really dug any deeper.
I used a multi-stage Dockerfile to define two containers-- one for building the binary and one for running it.
It looked like this:
# Create the build container to compile the hello world program
FROM rust:1.40-stretch as builder
ENV USER root
RUN cargo new hello
WORKDIR hello
RUN cargo build --release
# Create the execution container by copying the compiled hello world to it and running it
FROM scratch
COPY --from=builder /hello/target/release/hello /hello
CMD ["/hello"]
From everything I had read (which honestly wasn't much, as I am just starting to use Docker), this looked passable.
The build step seemed to work fine, as shown below:
brenden$:> docker build -t test/rust-hello:latest .
Sending build context to Docker daemon 307.2kB
Step 1/8 : FROM rust:1.40-stretch as builder
---> df5aae0d2dbf
Step 2/8 : ENV USER root
---> Using cache
---> afe8a4cfdde3
Step 3/8 : RUN cargo new hello
---> Using cache
---> 9a4826f41fca
Step 4/8 : WORKDIR hello
---> Using cache
---> 2980899a43c9
Step 5/8 : RUN cargo build --release
---> Using cache
---> 45520d0f5219
Step 6/8 : FROM scratch
--->
Step 7/8 : COPY --from=builder /hello/target/release/hello /hello
---> Using cache
---> 70634303a372
Step 8/8 : CMD ["/hello"]
---> Using cache
---> b3d523bdd185
Successfully built b3d523bdd185
Successfully tagged test/rust-hello:latest
But, the run resulted in an error:
brenden$:> docker run test/rust-hello:latest
standard_init_linux.go:207: exec user process caused "no such file or directory"
At first I thought this was an incompatibility issue between macOS X and linux, so I tried it on my System76 laptop running Debian 9-- the result was the same failure.
After some research, I found out that Rust statically links pure Rust dependencies, but it dynamically links C libraries.
One can see from the output of the ldd command (Linux) or the otool command (OS X) which libraries have been dynamically linked:
# on macOSX Mojave
brenden$:> otool -L hello/target/release/hello
hello/target/release/hello:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
/usr/lib/libresolv.9.dylib (compatibility version 1.0.0, current version 1.0.0)
# on Debian 9 (Linux)
brenden$:> ldd hello/target/release/hello
linux-vdso.so.1 (0x00007ffcab9e3000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fe82550b000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fe825501000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fe8254e0000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fe8254c6000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe825305000)
/lib64/ld-linux-x86-64.so.2 (0x00007fe825568000)
Obviously this was no good. I wanted to statically link every single dependency so I could have a truly standalone binary.
Some searching yielded two excellent results.
On Rust's official documentation, I found an explanation of an alternative to libccalled musl.
"Dynamic linking on Linux can be undesirable if you wish to use new library features on old systems or target systems which do not have the required dependencies for your program to run. Static linking is supported via an alternative libc, musl."[2]
So, musl is an alternative implementation of libc that developers use to package directly (read: statically link) with Rust binaries so that they are batteries-included.
On GitHub I found a repo called rust-musl-builder that is described as a "Docker container for easily building static Rust binaries".
Armed with a new technique, I made a second attempt that did away with the build container, choosing instead to build directly on my System76 laptop assisted by the rust-musl-builder container. The code to build it looked like this:
brenden$:> alias rust-musl-builder='sudo docker run --rm -it -v "$(pwd)":/home/rust/src ekidd/rust-musl-builder'
brenden$:> rust-musl-builder cargo build --release
# the output was sent to a different dir this time
# check it with ldd to show any dynamically linked libs
brenden$:> ldd hello/target/x86_64-unknown-linux-musl/release/hello
not a dynamic executable
Then it was time to write a simplified Dockerfile that just copied the build artifact to a scratch container and ran it:
FROM scratch
COPY hello/target/x86_64-unknown-linux-musl/release/hello /hello
CMD ["/hello"]
Build the container image:
brenden$:> sudo docker build -t test/rust-hello
Finally, it was time for a test run:
brenden$:> sudo docker run test/rust-hello
Hello, world!
Success!
For anyone more experienced with docker containers, this may seem obvious. For me, a person who backed into a DevOps career by way of sysadmin-ing, this was an unexpected challenge. I had always taken it on faith that Rust made binaries that were fully self-contained, but ultimately I found this to be only a partial truth. This was a fun learning experience, and I was proud to have figured it out.