Custom toolchain support

[cideM]

Through a weekend project I ended up trying to package Solana with Nix. The part that I'm struggling with is cargo-build-bpf. Essentially this downloads so-called BPF tools and then it uses rustup toolchain link to create a custom toolchain which is later used with cargo build +bpf. Now how do I make this work with Nix?

I first tried a devShell that uses Fenix with rustup in it. Curiously, if rustup is in the shell, cargo fails with error: no override and no default toolchain set. If I remove it, cargo (from Fenix) works again, but the command fails because it can't link the toolchain without rustup.

My other idea would be to patch out the rustup call and just do the toolchain linking myself. At the end of the day these just appear to be symlinks, more or less. But that's reimplementing a part of rustup which also doesn't sound great.

Any thoughts on supporting custom toolchains with Fenix, considering Fenix aims to replace rustup?

[wucke13]

I'd like to chip in and broaden the scope of this issue: Another interesting thing to have would be the esp32 rust toolchain

[tmayoff]

I'm trying to find a way to develop for the esp32 using rust on nix, what would it take to get it added to fenix

[ar3s3ru]

After quite some struggle, I managed to import the esp-rs/rust-build toolchain and make it work - albeit not through fenix per se.

The magic sauce to install the custom toolchain is in ./lib/mk-toolchain.nix, which is the function that creates the derivation that runs <toolchain-or-component>/install.sh.

mkToolchain is used behind all the public API, but isn't itself exported.

I've tried my best to make it work - by copying it into my project - but I just couldn't as I couldn't wrap my head around the 3-4 levels of nested function evaluation (toolchain, toolchain', toolchain'' and finally mkToolchain).

All I needed really was the stdenv.mkDerivation content (well, actually the installPhase to be precise) and to define the source params for fetchurl (to point to esp-rs/rust-build releases).

So I just ended up copying the stdenv.mkDerivation into two derivations; here you can see the result in my project:

1. ./nix/rust-esp.nix which provides cargo and rustc,
2. ./nix/rust-std-esp.nix which contains the stdlib,
3. Add the derivations to the ./nix/overlay.nix,
4. Use fenix.combine to join them together: https://github.com/openmoto-org/kontroller/blob/73a26aecb6ce875e8e69d07a81319759c97da6e4/flake.nix#L34-L36

There might be a way to do the same with the current public API, @figsoda can probably clarify that.

I assume it would be similar for other custom toolchains that come prebuilt.

[!IMPORTANT] If you're also targeting esp-idf, keep in mind you'll need more stuff to make it work than just the Rust toolchain.

You'll need nixpkgs-esp-dev too, which brings in a bunch of related tooling. One of which is esp-clang which is necessary for compilation and linking against esp-idf. You can install it like so: https://github.com/openmoto-org/kontroller/blob/73a26aecb6ce875e8e69d07a81319759c97da6e4/nix/overlay.nix#L5-L14

esp-clang is not visible as exported package through nixpkgs-esp-dev AFAIU, though it is added to the PATH when installed. If you use mkShellNoCC in your devShell, it should be the first clang path that gets resolved. Not the best, but 🤷🏻‍♂️

What's more, nixpkgs-esp-dev exports IDF_* environment variables to point to the esp-idf version it installs.

This will make esp-idf-sys angry, since that crate wants the esp-idf toolchain to be a Git repository (for whatever reason) and nixpkgs-esp-dev fetches it through fetchFromGithub, the esp-idf repo won't have Git info (which is good 'cause it's impure anyway).

Therefore you need to unset the exported IDF_* variables in your shellHook, like so: https://github.com/openmoto-org/kontroller/blob/73a26aecb6ce875e8e69d07a81319759c97da6e4/flake.nix#L58-L61

Among other things, esp-idf-sys is expecting some specific environment variables to be set (usually by espup), which are CLANG_PATH, LIBCLANG_PATH and LIBCLANG_BIN_PATH. Here is how I provided them through the devShell: https://github.com/openmoto-org/kontroller/blob/73a26aecb6ce875e8e69d07a81319759c97da6e4/flake.nix#L65-L69

Hope this was helpful, cheers!