Optimizing Compilation Speed and Disk Usage in Rust Development

In my actual development experience, two main issues have been troubling me in Rust development: slow compilation speed and high disk usage. This article will discuss these two problems and share some effective solutions.

Compilation Speed

When using certain third-party libraries, compilation speed can significantly slow down. For example, if you have ever used the async-stripe crate, you may have noticed that the project’s compilation time at least doubles, even though the project itself is very simple. The reason is that async-stripe generates a large amount of macro code during compilation (mainly from serialization/deserialization logic), causing the codebase to bloat to approximately 700,000 lines. In such cases, increased compilation time is inevitable, but we can mitigate this issue by optimizing dependencies and using caching tools.

Reducing Dependencies and Optimizing Dependency Tree

The number and complexity of dependencies directly impact compilation speed. Too many dependencies not only increase compilation time but also consume more disk space. Therefore, optimizing the dependency tree is crucial:

• Remove Unused Dependencies: The cargo-udeps tool can help detect and remove unused dependencies. This not only reduces compilation time but also decreases the project size. Install and use cargo-udeps:

bash
cargo install cargo-udeps
cargo udeps

• Choose Lightweight Alternatives: For certain functionalities, multiple crates may provide similar features. Choosing a lightweight crate can reduce the complexity of the dependency tree and the size of the compiled output. For instance, use smallvec instead of Vec, or once_cell instead of lazy_static.

• Avoid Duplicate Dependencies: Sometimes, multiple dependencies may introduce the same library, leading to redundant compilations. Use the cargo tree tool to view the dependency tree and identify and resolve duplicate dependencies. Run the following command to view the dependency tree:

bash
cargo tree

Using Caching Tools

sccache is a distributed compilation cache tool that caches compilation results and reuses them in subsequent compilations. sccache can even share compilation caches between different machines, which is particularly useful for CI/CD environments. Install and configure sccache:

bash
cargo install sccache
export RUSTC_WRAPPER=$(which sccache)

Using Fast Linker mold

mold is an efficient linker that speeds up the linking phase by optimizing the linking process. The linking phase is often one of the most time-consuming parts of the compilation process, especially when handling numerous dependencies. mold not only reduces memory usage but also shortens linking time. You can configure mold as the default linker as follows:

toml
[target.x86_64-unknown-linux-gnu]
linker = "clang"
rustflags = ["-C", "link-arg=-fuse-ld=/path/to/mold"]

Tip: Ensure the path to mold is correct, usually /usr/local/bin/mold or another installation path.

Disk Usage

In daily development, the disk usage of the target directory is a significant concern. Especially when your hard drive capacity is limited, frequent compilation operations can quickly fill up disk space. Here are the statistics after cleaning up the compilation cache in one of my projects:

bash
❯ cargo clean
Removed 237,637 files, 121.0 GiB total

On my Macbook with a 512GB hard drive, disk space is not abundant, and I develop multiple Rust projects simultaneously, so disk anxiety has always troubled me. To alleviate this issue, the following measures can be taken:

Unifying target Directory

By setting the environment variable CARGO_TARGET_DIR or configuring build.target-dir in ~/.cargo/config.toml, you can unify all projects’ target directories to an external location. This approach allows sharing certain intermediate layers of dependencies, reducing disk usage. The downside is that build artifacts are no longer located in the project root directory, which might inconvenience certain workflows.

toml
[build]
target-dir = "~/.cache/target"

Tip: Ensure the ~/.cache/target directory has sufficient permissions and regularly clean up unnecessary compilation artifacts.

Regularly Cleaning target Directory

Besides unifying the target directory, regularly cleaning it is also an effective strategy. You can manually clean it using the cargo clean command or automate the cleanup process in CI/CD pipelines. Additionally, cargo-sweep is a specialized tool for cleaning old dependencies, helping you further free up disk space.

bash
cargo install cargo-sweep
cargo sweep -a

Using File System Compression

If you use a file system that supports compression (such as Btrfs or ZFS), you can enable file system-level compression to reduce the disk usage of the target directory. For example, on Btrfs, you can enable compression with the following command:

bash
sudo btrfs property set /path/to/target compression zstd

Summary

Slow compilation speed and high disk usage are common pain points for Rust developers, but through reasonable optimization and tool usage, these issues can be effectively mitigated. Reducing dependencies, using caching tools, optimizing linkers, and managing the target directory properly are key means to improve development efficiency.

Of course, for those with ample resources, none of these issues would be a problem.