numtide / flake-utils

Pure Nix flake utility functions [maintainer=@zimbatm]
MIT License
1.14k stars 78 forks source link

single arch+os override flake #80

Closed dzmitry-lahoda closed 1 year ago

dzmitry-lahoda commented 1 year ago

i am doing nix flake check and get

error: error: a 'aarch64-darwin' with features {} is required to build '/nix/store/22zx8fsi94pa9p9m3w1m21ylb7fb55xa-yarn.nix.drv', but I am a 'x86_64-linux' with features {benchmark, big-parallel, kvm, nixos-test}

may we have flake doing only linux x86 override to flake so that we can smoke test whole nix in ci.

without having arm test.

zimbatm commented 1 year ago

I don't think this is coming from flake-utils.

If yarn2nix uses Import-From-Derivation (IFD), it will try building the derivation it's supposed to import on the same arch where the code is evaluated. And since Flakes evaluates all the architectures, it's exposing that problem.

dzmitry-lahoda commented 1 year ago

flake evaluates only systems+arch which is provided by flake-utils. if flake-utils can be configured via override only to one it will wokr

dzmitry-lahoda commented 1 year ago

i have tested that. so it is mundane to support out of this repo.

read it to be akin to Rust Features or Rust targets. It has filter by specific target and featue, and has -all-targes and -all-features check

zimbatm commented 1 year ago

ok, this is not a free support channel.

dzmitry-lahoda commented 1 year ago

huh, you are weird. i do not need support. i said you tool can have 2 feautes and you closed both by asking me to pay. we do not need support. I have fork of this already.

crazy.

dzmitry-lahoda commented 1 year ago

yarn2nix is not relevant. relevant is single system nix check which is possible with flake override for single system. I did it and it worked. you cannot nix check flake with ustils without as it cross checks too much.

dzmitry-lahoda commented 1 year ago

i reported this repor is not well plays with nix check and devops and you closed issuess with some weird response.

dzmitry-lahoda commented 1 year ago

here is the code for thoose who will have same issue https://github.com/ComposableFi/composable/blob/nix/v1/.nix/override-input/flake-utils/flake.nix

zimbatm commented 1 year ago

who knows. it's either a communication or expectation issue between us. or maybe I'm just being grumpy. I don't have a lot of energy to give away when half of it is already spent trying to understand the messages.

dzmitry-lahoda commented 1 year ago

What we want - check that nix """compiles""" in CI.

How? Run https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake-check.html

But, it tries to evaluate arm/darwin. Our CI is not that.

How to fix?

--override-input with flake-utils from template which does only on ${system} in list.

What it gets?

You can compile whole nix and ensure it is typed. But you cannot cross compile because of some dependencies. So we wan to check nix lang is correct, but cannot because of the only way to check is have arm/darwin.

Can I you

--arg name expr
Pass the value expr as the argument name to Nix functions.

--argstr name string
Pass the string string as the argument name to Nix functions.

Seems flakes ignore this or do not allow (at least what I read, and may be there is my ack of knowledge - but as per items on nix gh - it is dissallowed to have arbitrary input because it violates isolation of nix flake is flake.nix + flake.lock).

So there only way to """compile""" nix in CI to check it before build is to override with flake for 1 and only one system.

Can we just build? No, we do not have mac or ARM. But if nix lang is wrong it is wrong for all systems (we talk here about """compilation""", not about side effecting derivaiton which is different on different systems).

dzmitry-lahoda commented 1 year ago

@zimbatm you likely should undrestand that flake.nix can be '""compiled""", that is hard to get as I see for people. eqactly same as complex rust/typescript monorepo is compiled.

I tell flake.nix and its deps, not arbitrary defaul.nix.

And the only way to do compilation now is to override flake-utils with current system of CI.

dzmitry-lahoda commented 1 year ago

But yeah, that can be a bug in NIX. who knows.

dzmitry-lahoda commented 1 year ago

nix flake check =>

error: error: a 'aarch64-darwin' with features {} is required to build '/nix/store/22zx8fsi94pa9p9m3w1m21ylb7fb55xa-yarn.nix.drv', but I am a 'x86_64-linux' with features {benchmark, big-parallel, kvm, nixos-test}

nix flake check --override flake-utils flake-utils-x86_64-linux -> no error.

sure. it can be different if there is if else on system. but that is usually not what users do or see. they use cross platform stuff.

these packages - kvm - for sure cannot do mac darwin, but how can I test all the flake nix which is not mac?

 but I am a 'x86_64-linux' with features {benchmark, big-parallel, kvm, nixos-test}

so that is feature.

zimbatm commented 1 year ago

flake.nix:

{
  outputs = { self, nixpkgs }: {
    checks.aarch64-darwin.hello = nixpkgs.legacyPackages.aarch64-darwin.hello;
    checks.aarch64-linux.hello = nixpkgs.legacyPackages.aarch64-linux.hello;
    checks.x86_64-darwin.hello = nixpkgs.legacyPackages.x86_64-darwin.hello;
    checks.x86_64-linux.hello = nixpkgs.legacyPackages.x86_64-linux.hello;
  };
}
$ nix flake check -v
evaluating flake...
running flake checks...

You can try this on your system and see that the issue is not the multi-arch support. Flakes evaluate all the systems, but only builds the current one. Adding more hacks to fix the hacks is not the right solution. Stop using IFD, and your problem will go away.

dzmitry-lahoda commented 1 year ago

check in nix does run check command. in many cases check actually builds to run tests. so solution was pass not build param during check (so no test are running).

nix flake metadata && \
nix flake show --allow-import-from-derivation  --show-trace --fallback --debug --print-build-logs --keep-failed --option sandbox relaxed && \
nix flake check --no-build --keep-going --allow-import-from-derivation  --show-trace --no-update-lock-file --fallback --debug --print-build-logs --keep-failed --impure --option sandbox relaxed

in case of strict sandbox it gives compiled nix kind of.