nix-community / crate2nix

rebuild only changed crates in CI with crate2nix and nix
https://nix-community.github.io/crate2nix/
Apache License 2.0
338 stars 82 forks source link

Cross compilation improvements #274

Closed jordanisaacs closed 1 year ago

jordanisaacs commented 1 year ago

My goal for cross compilation was to get mustang targets working (which is successful with this patch). This differs a bit from standard nixpkgs cross compilation as it only needs to pass --target to rustc (non rust targets do not need to be cross compiled). Thus, the goal of these changes is to make cross compilation less reliant on the inner workings of nixpkgs build stages.

  1. The platform target now wholly relies on the stdenv passed in through calling Cargo.nix and acts similarily to buildRustCrate.
  2. mkBuiltByPackageIdByPkgs is now aware if it is a build dependency (or proc-macro) and can act accordingly. a. It uses a tiny patch to build-rust-crate for an argument to disable cross compilation. Cross compilation gets disabled for build dependencies b. A new argument, extraDepsIsBuild is provided as an override to .build. This lets you add the sysroot crates only to non build dependencies.

While it works for my use case not sure if it breaks other uses or if there is a better way to achieve this.

An example (albeit a bit messy) is seen here and the sysroot is here. Special thanks to alamgu for providing some guidance.

Ericson2314 commented 1 year ago

The rest of the changes I am not so sure about. So let's do the vendor stuff first and separately.

It sounds like the thing you are trying to avoid with doing it all Nixpkgs's way is building rustc itself? There are solutions to that that don't involve breaking with Nixpkgs so much.

jordanisaacs commented 1 year ago

It sounds like the thing you are trying to avoid with doing it all Nixpkgs's way is building rustc itself? There are solutions to that that don't involve breaking with Nixpkgs so much.

The reason for the changes is from my understanding the stdenv provided to the buildRustCrate function seems to stay constant whether it is a build or normal dependency. A different buildRustCrate isn't used for build and normal dependencies. (I may have missed something but this is my understanding of the code). I approached cross compilation by overriding the stdenv of the buildRustCrate function with the cross stdenv. Thus, all the build dependencies were getting cross compiled when they shouldn't have.

I tried to to approach it the alamgu way of going through buildPackages but it was failing to work at all with the oxalica's rust overlay.

jordanisaacs commented 1 year ago

Was able to figure out how to get it working the nixpkgs way. And the vendor was moved into its own PR.