nixpkgs update breaks empty_cross test due to missing `rust` arguement

kolloch commented 9 months ago

See https://github.com/nix-community/crate2nix/pull/318.

Reproduction Steps

nix develop (or nix-shell or direnv)
niv update nixpkgs

Build/run integration tests:

nix build -L \
  --option log-lines 100 --show-trace --out-link ./target/nix-result \
   -f ./tests.nix

Before Update

Everything builds without errors.

After Update

error message

Analysis

The test does some special setup steps in its default.nix including an override for buildRustCrate vscode.dev:

fun = pkgs.buildRustCrate.override {
          inherit stdenv;

          # Don't bother with cross compiler since we don't need stdlib
          inherit (pkgs.buildPackages.buildPackages) rust rustc cargo;
        };

If you check for differences between the working and non-working commit with git diff 0cbe9f69c234a7700596e943bfae7ef27a31b735 bd645e8668ec6612439a9ee7e71f7eac4099d4f6 -- pkgs/build-support/rust/, you get:

diff --git a/pkgs/build-support/rust/build-rust-crate/build-crate.nix b/pkgs/build-support/rust/build-rust-crate/build-crate.nix
index 37bf3ec26f77..e842b6a3f501 100644
--- a/pkgs/build-support/rust/build-rust-crate/build-crate.nix
+++ b/pkgs/build-support/rust/build-rust-crate/build-crate.nix
@@ -1,6 +1,5 @@
 { lib, stdenv
 , mkRustcDepArgs, mkRustcFeatureArgs, needUnstableCLI
-, rust
 }:

 { crateName,
@@ -21,7 +20,7 @@
         (mkRustcDepArgs dependencies crateRenames)
         (mkRustcFeatureArgs crateFeatures)
       ] ++ lib.optionals (stdenv.hostPlatform != stdenv.buildPlatform) [
-        "--target" (rust.toRustTargetSpec stdenv.hostPlatform)
+        "--target" stdenv.hostPlatform.rust.rustcTargetSpec
       ] ++ lib.optionals (needUnstableCLI dependencies) [
         "-Z" "unstable-options"
       ] ++ extraRustcOpts
@@ -41,6 +40,7 @@
     );

     binRustcOpts = lib.concatStringsSep " " (
+      [ "-C linker=${stdenv.cc}/bin/${stdenv.cc.targetPrefix}cc" ] ++
       baseRustcOpts
     );

diff --git a/pkgs/build-support/rust/build-rust-crate/default.nix b/pkgs/build-support/rust/build-rust-crate/default.nix
index d977fb9f7c27..1760ab6e99a6 100644
--- a/pkgs/build-support/rust/build-rust-crate/default.nix
+++ b/pkgs/build-support/rust/build-rust-crate/default.nix
@@ -10,7 +10,6 @@
 , fetchCrate
 , pkgsBuildBuild
 , rustc
-, rust
 , cargo
 , jq
 , libiconv

and more. So the rust parameter is not used anymore and instead stdenv is expected for the rustcTargetSpec. If we simply remove it from the code in default.nix, we get a weird error:

 ...
        … while calling anonymous lambda

         at /nix/store/m707dys0ykz12vnyfljx62dc6lv0hmx7-source/pkgs/build-support/rust/build-rust-crate/build-crate.nix:5:1:

            4|
            5| { crateName,
             | ^
            6|   dependencies,

       error: cannot coerce null to a string

Conclusion

Part of the complexity is due to the fact that for certain platforms the expected sharedLibrary extension is not provided. I want to fix that in nixpkgs in https://github.com/NixOS/nixpkgs/pull/278679.
This cannot coerce null to a string error is not very helpful and I'd appreciate help from someone knowing cross compilation.
During experimentation, I got to a state where it started to compile gcc etc for riscv. I think a lot of the overriding in the test case is to prevent that -- since these tools then get typically not used.

kolloch commented 9 months ago

@Ericson2314 I think that you wrote the original tests and know waaaay more about cross-building than I do :) If you could help, it would be appreciated.

kolloch commented 9 months ago

@flokli / @amjoseph-nixpkgs I'd really appreciate help here -- pinged you since you seem to know and care about cross-compiling.

Checkout crate2nix

Execute the test that is now skipped by default

nix build -L \
  --option log-lines 100 --show-trace --out-link ./target/nix-result \
   -f ./tests.nix empty_cross.forceSkipped

Check out what the heck is going wrong with https://github.com/nix-community/crate2nix/tree/master/sample_projects/empty_cross.

Thank you!!

flokli commented 9 months ago

I'm sorry, I only carried @amjoseph-nixpkgs ' patches to here, and am not too familiar with the cross architecture.

Ericson2314 commented 9 months ago

CC @alyssais I'll take a look at this, but I might need your help :)

ghost commented 8 months ago

I'm sorry, I only carried @amjoseph-nixpkgs ' patches

lolwut? not my commit:

$ git show --format=full 95e3997dc9b806090e5f8c7adff80acfee14714a | egrep 'Author|Commit'
Author: Florian Klink <flokli@flokli.de>
Commit: Florian Klink <flokli@flokli.de>

The test does some special setup steps in its default.nix

Yeah the monkeying around that it is doing with stdenv internals is totally crazy. IMHO it "worked by accident" before. You should drop that test or split it into a sequence of tests that each do one small thing -- it's trying to do too many things at once, so when it fails eval like this it really doesn't tell us anything except "something is bad".

If you replace -none-elf with -linux-gnu in the test the problem goes away. This issue affects only the -none-elf targets, which are weird ones to begin with. Could you please update the title of this issue to reflect that? This is a "kernel-less target" problem, not a general cross problem.

flokli commented 8 months ago

I'm sorry, I only carried @amjoseph-nixpkgs ' patches

lolwut? not my commit:
$ git show --format=full 95e3997dc9b806090e5f8c7adff80acfee14714a | egrep 'Author|Commit'
Author: Florian Klink <flokli@flokli.de>
Commit: Florian Klink <flokli@flokli.de>

Yes, you're not the author of the commits in crate2nix, but they originate from your CLs in depot, as you commented in https://github.com/nix-community/crate2nix/pull/309#issuecomment-1790296452.

I only meant to say I was doing that upstreaming and don't feel qualified enough with the nixpkgs cross architecture to help out with the specific issue discussed here (#319). Thanks for picking it up!

Ericson2314 commented 8 months ago

Hmm, I though I made the freestanding cross test :)

nix-community / crate2nix