Lately I’ve become quite a fan of Rust, a modern programming language with a focus on memory safety. I’m using it in different projects, mostly for it’s great speed and low resource usage compared to e.g. Python or Node.
In contrast to scripting languages, all Rust applications have to be compiled before they can be executed. This is a process that takes some time, depending on the size of the app and the number of dependencies.
During development, this isn’t an issue thanks to local caching and incremental compilation, which allows the compiler to only re-compile what has changed since the last run.
For my web applications, I use Kubernetes and Docker images for deployment, which means I have to build my applications using Docker as well. In this case, these tricks won’t help us.
The following example Dockerfile shows how to build an Rust application using Docker. We’re using a multi-stage build here, where the first stage uses an base image that contains the Rust compiler, while the second stage uses a slim Debian image which contains the necessary runtime libraries (this service uses PostgreSQL). This approach greatly reduces the final size of the image.
This example is based on the instructions of the Docker Rust base image.
This approach is easy, but has a great disadvantage: It’s really slow.
While Docker has some built-in layer caching, we can’t use it here. Even tricks that are used in other language platforms - like copying the dependency lockfile first and the rest of the app laster - cannot help us here.
Fortunately, developers at Mozilla found a lovely solution to share the build cache between systems, even if not using Docker: sccache.
sccache is a ccache-like compiler caching tool. It is used as a compiler wrapper and avoids compilation when possible, storing cached results either on local disk or in one of several cloud storage backends.
sccache allows different storage options, including the local disk, S3-compatible object storage and caching systems like Redis or Memcached. In my case, I chose an S3-based storage (MinIO), as it’s working well in externally running systems like CI runners.
As first step, we need to get sccache into the build container. Fortunately the maintainers provide pre-built binaries, that can be easily downloaded and used:
💡 Please check the latest version on sccache releases page instead of just copy-pasting this code, which might be already outdated when you’re reading it.
Additionally, we need to set some environment variables to configure sccache. Put these directly after the previous code block, the variables must be set before invoking
💡 These are set inside the build container only, and won’t leak into the final image.
For further details on configuring sccache, please check out their README.txt.
Securing up the storage
For security reasons, I recommend creating an own IAM user and limit it’s permissions to the bucket only. On my MinIO instance, I’ve used the following policy to only grant limited access:
Additionally, you might want to define a retention policy to auto-delete old files, else you’ll slowly build up a pile of unused data.
While the first build might take slightly longer (because of the additional I/O to upload the cache files), all further compilations should be considerable faster.
Autor Michael Mayr
letzte Änderung 03.07.2021
Lizenz Lizenziert unter CC BY-NC-SA 4.0.