nix-community / crate2nix

rebuild only changed crates in CI with crate2nix and nix
https://nix-community.github.io/crate2nix/
Apache License 2.0
380 stars 89 forks source link

bloated dependencies due to `crate2nix_stable` #371

Open sellout opened 3 days ago

sellout commented 3 days ago

Since the flake was added, there has been a crate2nix_stable input that points at the previous[^1] release so that test runner can be built even if the current branch has broken stuff. However, this bloats the input graph, with master currently having four versions of each input as a consequence.

[^1]: I guess 0.13.0 got skipped, as the references are master → 0.14.1 → 0.14.0 → 0.12.0.

I can think of two solutions for this (in order of my own preference):

  1. Don’t reference the flake.lock to get this input, and thus remove the input completely. Since it references tagged versions, I feel like pulling that tag rather than the specific SHA is reasonable, but I get that that means the repo isn’t as pure as it could otherwise be.
  2. Just keep one level of _stable reference. Adding inputs.crate2nix_stable.inputs.crate2nix_stable.follows = "crate2nix_stable"; lets you still use the previous stable release to test the current branch, but then that stable release will refer to itself as its own stable release, rather than the infinite regress going on at the moment.

I’d be happy to implement either of these (or some other approach) if there’s something the devs are amenable to.

workaround

As a user of crate2nix, I don’t need to worry about the test runner, so in my own flakes I have

inputs.crate2nix.inputs.crate2nix_stable.follows = "crate2nix";

which ties the knot, eliminating the three extra sets of inputs[^2]. This is a trivial adaptation of solution 2 above, and could still be useful for users even if solution 2 is adopted by crate2nix (solution 1 would obviate this workaround, which is part of why I prefer that one).

[^2]: inputs.crate2nix.inputs.crate2nix_stable.follows = "" could also work (crate2nix already does this to “discard” inputs in other cases), but I’m not a fan, as it actually points crate2nix_stable at the current flake[^3], which for users isn’t the right flake at all and even for crate2nix itself points it at the “unstable” version.

[^3]: It would be great if there were something like .follows = null to remove a reference, but it does seem like .follows = "" is the closest thing that exists for now.

sellout commented 3 days ago

As, I’m thinking about this, I wonder if the _stable approach might solve some of my own circular dependency woes. I have a set of flakes like

flowchart LR
  A --> B
  A --> C
  B --> A
  C --> A

where all of my projects depend on A, and the handful of my projects that A depends on end up with a circular dependency.

I originally had the infinite regression

flowchart LR
  A --> B
  A --> C
  B --> A'
  C --> A'
  A' --> B'
  A' --> C'
  B' --> A''
  C' --> A''

but then modified it to use self references like inputs.b.inputs.a.follows = "", resulting in the first diagram with actual circular dependencies. But this does cause issues with having to update multiple repos in tandem in some cases.

I wonder if an approach like solution 2

## in A’s flake.nix
inputs.a_stable.inputs.a_stable.follows = "a_stable";
inputs.b.inputs.a.follows = "a_stable";
inputs.c.inputs.a.follows = "a_stable";

## in B’s flake.nix
inputs.b_stable.inputs.b_stable.follows = "b_stable";
inputs.a.inputs.b.follows = "b_stable";

## in C’s flake.nix
inputs.c_stable.inputs.c_stable.follows = "c_stable";
inputs.a.inputs.c.follows = "c_stable";

resulting in

flowchart LR
  A --> B
  A --> C
  B --> B_stable
  C --> C_stable
  B --> A_stable
  A --> A_stable
  A_stable --> A_stable
  A_stable --> B_stable
  A_stable --> C_stable
  B_stable --> B_stable
  B_stable --> A_stable
  C --> A_stable
  C_stable --> C_stable
  C_stable --> A_stable

might defer the cycle enough to allow for more independent development (with enough upsides to justify the bloat).

I’m sure I haven’t gotten that quite right, and I foresee issues with managing breaking changes (which I suppose is the situation that currently causes tandem updates), but this might be something worth exploring … so thanks for the idea 😅