nix search with `-f` works poorly with "installable" semantics

[samueldr]

Describe the bug

According to the help text, a search query has to be given after an installable.

.../channels/nixpkgs $ nix --extra-experimental-features nix-command search --help | head
    Warning: This program is experimental and its interface is subject to change.

Name

    nix search - search for packages

Synopsis

    nix search [option...] installable regex...

With -f use, which is the non-flakes escape hatch, nix search cannot actually be used to search all the packages.

Steps To Reproduce

~ $ nix --extra-experimental-features nix-command search -f '<nixpkgs>' 'firefox'
*  (94.0.1)

~ $ nix --extra-experimental-features nix-command search -f '<nixpkgs>' 'fire'
error: attribute 'fire' in selection path 'fire' not found

(Ignore the lack of attribute names in results...)

Expected behavior

Compare with (purposefully redundant) use with Nix 2.3

 $ nix-shell -I nixpkgs=channel:nixos-21.05 -p nix

[nix-shell:.../channels/nixpkgs]$ nix search -f '<nixpkgs>' fire | head
warning: using cached results; pass '-u' to update the cache
* arrayfire (arrayfire)
  A general-purpose library for parallel and massively-parallel computations

* blackfire (blackfire-agent)
  Blackfire Profiler agent and client

* bonfire (bonfire-unstable)
  CLI Graylog Client with Follow Mode

* chrome-token-signing (chrome-token-signing-1.1.5)

[nix-shell:~]$ 

nix-env --version output

~ $ nix-env --version
nix-env (Nix) 2.4

~ $ readlink -f $(which nix)
/nix/store/ag01yxbp4z0bksh7qc67xc31rgk91yaa-nix-2.4/bin/nix

[thufschmitt]

So here’s my suspicion as to what is happening (here and in #5641):

• nix search -f '<nixpkgs>' firefox doesn’t search for firefox in nixpkgs but for the empty string (matching everything) in (import <nixpkgs> {}).firefox. So you only get one result. And because what nix search prints isn’t the package name but its attribute path (relative to the root of the search), it will print the empty attribute path.
• Likewise, nix search -f '<nixpkgs>' fire will search in (import <nixpkgs> {}).fire wihch doesn’t exist, hence the error
• The “correct” syntax should be nix search -f '<nixpkgs>' . firefox, but it doesn’t work because it expects import <nixpkgs> to be an attribute set (there’s some logic to magically call things if they’re a function, but it probably isn’t applied to the last element of the attribute path)
• nix search -f <nixpkgs> pkgs firefox nearly works, but fails at the first evaluation error (which by design is the very first package that Nix encounters). That’s because nix search has a logic to ignore eval errors, but it’s only enabled if the first element of the attribute path is legacyPackages (which is why nixpkgs is exposed under legacyPackages and not packages in its flake) − https://github.com/NixOS/nix/blob/master/src/nix/search.cc#L164-L165
• Regardless of that, recurseForDerivations is only taken into account if the first element of the attribute path is legacyPackages and the depth is bigger than 2 − https://github.com/NixOS/nix/blob/master/src/nix/search.cc#L157-L161 . So the search will recurse into a lot of stuff that it shouldn’t have to.

At the end of the day, search can be made to work with:

$ cat <<EOF > default.nix
{ legacyPackages.pkgs = import <nixpkgs> {}; }
EOF
$ nix search -f . . firefox-beta-bin
* legacyPackages.pkgs.firefox-beta-bin (94.0b2)
  Mozilla Firefox, free web browser (binary package)

* legacyPackages.pkgs.firefox-beta-bin-unwrapped (94.0b2)
  Mozilla Firefox, free web browser (binary package)

(which is a lot of hops to go through ;) )

[nbraud]

With -f use, which is the non-flakes escape hatch, nix search cannot actually be used to search all the packages.

Why is there a “non-flakes escape hatch”? flakes are an optional, experimental feature, and presumably shouldn't be involved without explicit user opt-in.

[thufschmitt]

Why is there a “non-flakes escape hatch”? flakes are an optional, experimental feature, and presumably shouldn't be involved without explicit user opt-in.

So is the new cli (including nix search). And part of the reason for that is indeed that it’s strongly tied to flakes and the non-flake use-case is a second-class citizen there

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nix-2-4-and-what-s-next/16257/1

[thufschmitt]

Relevant: #7668

[Ericson2314]

CC @infinisil

[tomberek]

Fixed by: https://github.com/NixOS/nix/pull/7668 (edit: duplicate with above, sorry)