vimPluginsUpdater doesn't work outside nixpkgs

[jlesquembre]

Describe the bug

I get an error when I follow the steps described here: https://github.com/NixOS/nixpkgs/blob/34ea795b4ba7168607098689ac4728bf697d191b/doc/languages-frameworks/vim.section.md?plain=1#L248-L262

Steps To Reproduce

Steps to reproduce the behavior:

Run:

mkdir -p /tmp/my-plugins
cd /tmp/my-plugins
echo -e "repo,branch,alias\npwntester/octo.nvim,," >vim-plugin-names
git init
nix run github:NixOS/nixpkgs/nixos-unstable#vimPluginsUpdater -- -i vim-plugin-names -o generated.nix --no-commit

Fails with:

error:
       … <borked>

         at «none»:0: (source not available)

       … while calling the 'import' builtin

         at «string»:1:6:

            1| with import <localpkgs> {};
             |      ^
            2| let

       error: opening file '/tmp/my-plugins/default.nix': No such file or directory
Traceback (most recent call last):
  File "/nix/store/5zvxiglpn43k71jrlh68j0krgr84zdm7-vim-plugins-updater-0.1/bin/..vim-plugins-updater-wrapped-wrapped", line 162, in <module>
    main()
  File "/nix/store/5zvxiglpn43k71jrlh68j0krgr84zdm7-vim-plugins-updater-0.1/bin/..vim-plugins-updater-wrapped-wrapped", line 158, in main
    editor.run()
  File "/nix/store/5zvxiglpn43k71jrlh68j0krgr84zdm7-vim-plugins-updater-0.1/lib/pluginupdate.py", line 560, in run
    getattr(self, command)(args)
  File "/nix/store/5zvxiglpn43k71jrlh68j0krgr84zdm7-vim-plugins-updater-0.1/bin/..vim-plugins-updater-wrapped-wrapped", line 129, in update
    pluginupdate.update_plugins(self, args)
  File "/nix/store/5zvxiglpn43k71jrlh68j0krgr84zdm7-vim-plugins-updater-0.1/lib/pluginupdate.py", line 783, in update_plugins
    update = editor.get_update(args.input_file, args.outfile, fetch_config)
  File "/nix/store/5zvxiglpn43k71jrlh68j0krgr84zdm7-vim-plugins-updater-0.1/lib/pluginupdate.py", line 423, in get_update
    cache: Cache = Cache(self.get_current_plugins(self.nixpkgs), self.cache_file)
  File "/nix/store/5zvxiglpn43k71jrlh68j0krgr84zdm7-vim-plugins-updater-0.1/lib/pluginupdate.py", line 407, in get_current_plugins
    data = run_nix_expr(self.get_plugins, nixpkgs)
  File "/nix/store/5zvxiglpn43k71jrlh68j0krgr84zdm7-vim-plugins-updater-0.1/lib/pluginupdate.py", line 345, in run_nix_expr
    out = subprocess.check_output(cmd)
  File "/nix/store/pzf6dnxg8gf04xazzjdwarm7s03cbrgz-python3-3.10.12/lib/python3.10/subprocess.py", line 421, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/nix/store/pzf6dnxg8gf04xazzjdwarm7s03cbrgz-python3-3.10.12/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['nix', 'eval', '--extra-experimental-features', 'nix-command', '--impure', '--json', '--expr', 'with import <localpkgs> {};\nlet\n  inherit (vimUtils.override {inherit vim;}) buildVimPlugin;\n  inherit (neovimUtils) buildNeovimPlugin;\n\n  generated = callPackage <localpkgs/pkgs/applications/editors/vim/plugins/generated.nix> {\n    inherit buildNeovimPlugin buildVimPlugin;\n  } {} {};\n  hasChecksum = value:\n    lib.isAttrs value && lib.hasAttrByPath ["src" "outputHash"] value;\n  getChecksum = name: value:\n    if hasChecksum value then {\n      submodules = value.src.fetchSubmodules or false;\n      sha256 = value.src.outputHash;\n      rev = value.src.rev;\n    } else null;\n  checksums = lib.mapAttrs getChecksum generated;\nin\n  lib.filterAttrs (n: v: v != null) checksums\n', '--nix-path', 'localpkgs=/tmp/my-plugins']' returned non-zero exit status 1.

Additional context

Before https://github.com/NixOS/nixpkgs/pull/253714, I was able to run

~/nixpkgs/pkgs/applications/editors/vim/plugins/update.py \
  -i vim-plugin-names -o generated.nix --no-commit                   

Notify maintainers

@teto

Metadata

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.54, NixOS, 23.11 (Tapir), 23.11.20230919.5ba549e`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.17.0`

[teto]

what if you set --nixpkgs ~/nixpkgs ? The program is supposed to be indenpendant of nixpkgs so once it is in /tmp/my-plugins it's normal it can't find some files.

NB: it you wanna review https://github.com/NixOS/nixpkgs/pull/260096

[jlesquembre]

@teto yes, that fixes the issue, maybe can we add it to the documentation? I can open a PR if that makes sense, but feel free to add it yourself if you prefer.

[teto]

I will add it to the example but the error you had was bad UX. Also at this point it shouldn't need any nixpkgs ? I think what happens is that the script tries to read the checksums from existing plugins in nixpkgs to fillup its cache. Maybe it could disable the cache instead

[jlesquembre]

Talking about documentation and UX, docs say that to reference the generated plugins we can do this:

myVimPlugins = pkgs.vimPlugins.extend (
  (pkgs.callPackage ./generated.nix {})
);

but it will fail, we have to pass a reference to buildVimPlugin and buildNeovimPlugin, it's more like this:

myVimPlugins = pkgs.vimPlugins.extend (
  (pkgs.callPackage ./generated.nix {
      inherit (pkgs.vimUtils) buildVimPlugin;
      inherit (pkgs.neovimUtils) buildNeovimPlugin;
  })
);

[teto]

sry I've been too busy + it's not a feature I use currently. Anyone feel free to pick it up, I can review.