Rewrite derivers in local DB for `keep-derivations=true`

[danbst]

https://discourse.nixos.org/t/how-to-get-a-missing-drv-file-for-a-derivation-from-nixpkgs/2300/8

When derivation is substituted it inherits deriver (.drv path) from binary cache. But because .drv are not stable (multiple derivations can produce same output), it isn't always same .drv that was used locally.

So, when keep-derivations=true, store DB should update deriver with local .drv path, not use substituted.

[lheckemann]

Or maybe more than one deriver per path should be allowed?

[bb010g]

Here's a workaround for nix-store {--query | -q} {--derivers | -d} for now that pulls using libstore's queryValidDerivers (via nix show-derivation) instead of queryPathInfo:

% () { local p=("${@:P}"); nix show-derivation "${p[@]}" | jq 'with_entries(. as {key:$k, value:$v} | {value: $k, key: ($ARGS.positional[] | select(startswith($k, $v.outputs[].path)))})' --args "${p[@]}" }  =systemctl "$(man -w systemctl)"

{
  "/nix/store/01ismaycmmibdnnmy3bxhgg8xpxy997m-systemd-239.20190219/bin/systemctl": "/nix/store/xr0dlfbcvrwrnzw4kn310r35nagcnm41-systemd-239.20190219.drv",
  "/nix/store/nxiyiyj4psf65mprq30nr3h8wflr4g55-systemd-239.20190219-man/share/man/man1/systemctl.1.gz": "/nix/store/xr0dlfbcvrwrnzw4kn310r35nagcnm41-systemd-239.20190219.drv"
}

This uses Zsh features and can be dropped straight on a command line, as shown.

Prettier, commented jq

```jq with_entries(. as {$key, $value} | # Change the mapping from {(derivation): outputs) to {(output): derivation}. { value: $key, # This produces a variable number of outputs, and thus a variable number # of entries in the resulting mapping, one for each output found in this # derivation. key: ( $ARGS.positional[] | # We filter from all the derivations found for our arguments down to # only those with outputs matching a provided output path, or # alternatively the derivation path itself (if provided; the paths would # be equal in this case). select(startswith($key, $value.outputs[].path)) ) } ) ```

If you're on Bash, the following should work:

nix-find-deriver() {
  local ps=(); for p; do ps+=("$(realpath "$p")"); done
  nix show-derivation "${ps[@]}" | \
  jq 'with_entries(. as {key:$k, value:$v} | {value: $k, key: ($ARGS.positional[] | select(startswith($k, $v.outputs[].path)))})' --args "${ps[@]}"
}

(If you do not have realpath(1) on your system, readlink -f is fine.)

If you don't want the mapping to JSON behavior, and simply want plain derivation paths in corresponding order, as with nix-store -qd, use jq -r 'BODY' --args … with this body instead:

. as $d | $ARGS.positional[] | first((. as $p | $d | keys_unsorted[] | . as $k | select($p | startswith($k, $d[$k].outputs[].path))), "unknown-deriver")

Prettier, commented jq

```jq # We're producing based on argument order, so change our input. . as $drvs | $ARGS.positional[] | # This first function gives us a nice fallback and doesn't make us process # more input than necessary. first( # We want to check all the derivations for every argument, so change inputs. (. as $path | $drvs | # We're already going to output whatever the first output from this # sub-expression is, so we can just split out into all the derivation # paths already. keys_unsorted[] | # This last bit of input footwork decides whether to output this # derivation path by running the current argument path through all the # candidate path prefixes and outputting if it's a match. (This normally # would cause issues relating to outputing the same value multiple times, # but the first match we have will catch on our first(...) and break out.) . as $key | select($path | startswith($key, $drvs[$key].outputs[].path)) ), # By generating a constant after our main filter's outputs, we can smoothly # fail on unknowns with the same string nix-store uses. "unknown-deriver" ) ```

If you'd like the more nix-store -qd-style behavior of not talking about the derivation path you just gave, remove "$k," from $k, $v.outputs[].path in either jq expression.

[stale[bot]]

I marked this as stale due to inactivity. → More info

[stale[bot]]

I closed this issue due to inactivity. → More info

[fzakaria]

This is really confusing to understand. I try to --query --deriver often and I don't get the right DRV. Kind of confusing my understanding of Nix.

[fzakaria]

Here is what I am observing:

I posted https://discourse.nixos.org/t/why-isnt-deriver-and-realize-identity-functions-see-example/47490 which is maybe related.

This UX is terrible for anyone trying to understand Nix.

> which htop
/Users/fzakaria/.nix-profile/bin/htop

> nix derivation show $(which htop) | jq -r "keys[0]"
/nix/store/hgnz7dz10vi0n81s341iihqm1azq4lb6-htop-3.2.2.drv

> nix-store --query --deriver $(which htop)
/nix/store/nshxi52ldsv03sc8k810kw5j3hwa69bv-htop-3.2.2.drv

Those two derivations are different and only one exists on my system.

[kjeremy]

Here is what I am observing:

I posted https://discourse.nixos.org/t/why-isnt-deriver-and-realize-identity-functions-see-example/47490 which is maybe related.

This UX is terrible for anyone trying to understand Nix.

> which htop
/Users/fzakaria/.nix-profile/bin/htop

> nix derivation show $(which htop) | jq -r "keys[0]"
/nix/store/hgnz7dz10vi0n81s341iihqm1azq4lb6-htop-3.2.2.drv

> nix-store --query --deriver $(which htop)
/nix/store/nshxi52ldsv03sc8k810kw5j3hwa69bv-htop-3.2.2.drv

Those two derivations are different and only one exists on my system.

I just ran into this last week and am also very confused.