A convention for "channel" index derivations?

[copumpkin]

I've proposed something like this before, but this seems like it could improve Nix usability significantly:

nix-env -i and nix-env -q are both pretty slow. Even if the future nix command gets rid of name-centric tooling, we're still going to want something that supports queries against a package set.

I'm thinking of the following:

1. At the top level of nixpkgs (or some attribute path we all agree on), we add a derivation that builds an index of nixpkgs (using the path attribute or whatever).
2. That derivation gets built by Hydra, stuffed into the binary cache, and all the usual things.
3. nix-env (and eventually nix) looks for the index attribute, and either builds it (potential bad user experience) or substitutes it from the cache. It then just runs a quick query against that index to do its job.

This way, users of the official channels get a snappy experience with both queries and (if we keep them around) name-based operations. Developers who use queries or name-based operations will build their own indices on the fly (just like any other derivation), and doing that will likely not be much slower than waiting for Nix to traverse the entire tree anyway.

See https://github.com/NixOS/nix/issues/485 for the last time I proposed something like this, and for an issue where this matters.

[domenkozar]

Sounds like something we could ship together with https://github.com/NixOS/nixos-channel-scripts/issues/4

[copumpkin]

Oh yes, I'd forgotten about command-not-found. That could definitely use a similar mechanism, although I'd probably keep a separate index for it because that one actually needs to build packages (and thus takes a bajillion years), whereas this index only needs to evaluate the package set.

[cleverca22]

https://github.com/NixOS/nixpkgs/issues/12203#issuecomment-183547626

this command will take a release.nix file, and create a list of every single derivation that hydra can build, in json form

with a small tweak, you can also include the unfree packages, and skip the 32bit and 64bit separation of things

[Ericson2314]

Package's meta field could also contain a list of executables that are supposed to be provided, and at build time, nixpkgs could ensure that the those packages do indeed contain (at least) those executables. This would allow building a provisional command-not-found map.

[cleverca22]

a library list would also be valuable, then you can iterate over all of nixpkgs to find any binary or library

[stale[bot]]

I marked this as stale due to inactivity. → More info

[stale[bot]]

I closed this issue due to inactivity. → More info