Full Stack Rust

A year ago, my selection of go-to languages looked like the following:

• Python for quick prototyping of high-level code, or for code that needs to leverage third-party functionality
• C/C++ for longer-lived low-level projeccts

At the time, I had only heard of Rust and briefly used it. My experience came from choosing to write a small utility in Rust to process a large file (> 4GB) of transactions and mine some statistics from them.

For that utility, I pulled in a library to map the file into memory and ran an analysis over it sequentially. There were some cool concepts, like the compiler statically enforced that the memory mapping could not be accessible whatsoever after it’s unmapped - an error that could occur in C++ if you weren’t careful.

However, it didn’t really draw me in at the time because it was just a small novelty.

What did the trick was when I added new functionality to pdblister to fetch many thousands of PDB files in parallel. Such a thing is nearly impossible in CPython thanks to the GIL (unless you want to deal with Python’s multiprocessing and the problems that come with that), and it’s extremely difficult to do in C/C++ without facing parallelism bugs.

However, with Rust it was a breeze. I added async driven by tokio, spawned new tasks with tokio::spawn to download PDBs, and fixed any compiler errors the Rust compiler came up with. Compiled, and it just worked.

The cherry on top? The Rust compiler spat out a binary that just runs anywhere with no run-time dependencies.

Rust instead of Python

Which comes to my first point. Rust is a great replacement for Python as a language for medium to long-lived tools.

The main benefit coming from Python is the large ecosystem of libraries and functionality, all accessible behind Python’s package manager pip. Want to quickly prototype something that needs to interact with an API? Python’s requests is a great choice - and if you already have it installed, you just have to type in import requests and you have it.

In addition, so does Rust with reqwest. Using cargo-edit, just type in cargo add reqwest in your project and then begin using it in your code!

However, Python begins losing points when you start moving into the longer-term lifetime. requests is a dependency that your users will need to fetch in order to use your program (unless you mess with pyinstaller - which I haven’t typically seen used in practice). In addition, they lose more points because of the weak typing and error handling ability (compared to Rust).

And at this point, I can write prototype tooling using Rust much quicker than I can with Python, and I can feel confident knowing that my tooling is easier to maintain and longer-lived than the equivalent Python.

For short-lived tools though, Python may still be better simply because it does not require project setup to get intellisense support in VSCode. Rust’s cargo-script gets close to pushing Rust into the realm of a scripting language, but unfortunately I haven’t found a plugin that integrates with it in VSCode.

Rust instead of C

Rust is also an outright replacement for C. It’s better in every way, and can natively interoperate with legacy C code for incremental replacement (a process called oxidation).

The biggest improvement with Rust is the ecosystem: as discussed in the previous section, it’s trivial to take advantage of pre-existing libraries in the Rust ecosystem. If you’ve never used the C language family, you’re lucky. In practice, the best way to use advanced functionality in C is to write it yourself.

The C ecosystem is fragmented, and fragile. There is no consistent standard for ABI or build systems:

• Because of the lack of ABI consistency, you cannot use the same binaries across platforms or operating systems. So you have to build the libraries from source.
• Because of the lack of a consistent build system, you can’t simply build a C library alongside your application. So you have to patch or rewrite the build system of the library you want to make it compatible with yours.
• C libraries are fundamentally handicapped because they can’t rely on third-party functionality in turn, such as abstractions around OS-specific functionality (most C libraries are locked to an OS, usually Linux).

And then you have the safety improvements - which you hear about in literally every blog post about Rust - so I’m not going to go over them.

But in my experience - the safety largely serves as a tool to make it easier for third-party library developers to force me to use their library correctly. Something that C libraries can’t do, apart from writing comments that can be freely ignored.

Full-stack Rust

So to bring it all together - I’ve been using Rust in the past year in all parts of the stack where I previously used other languages.

I’ve used Rust to implement a bootloader. I’ve used it for mirroring files via high-level HTTP/HTTPS and other technologies in pdblister and panamax. I’ve taken advantage of and contributed to the excellent gdbstub library for use in the control of VMs ran by a custom VMM.

These projects were all done at varying levels of the stack, and Rust has been an excellent fit for all levels. I’ve moved to exclusively using Rust in my personal projects, as well as pushing it for use in my work when it makes sense to do so.