Vim plugin fails to create virtualenv on NixOS

[sbruder]

Describe the bug

When opening vim with black installed for the first time, black tries to create the virtualenv. On my system it fails with the following error:

Error detected while processing function provider#python3#Call:
line   18:
Error invoking 'python_execute' on channel 3 (python3-script-host):
Traceback (most recent call last):
  File "<string>", line 96, in <module>
  File "<string>", line 72, in _initialize_black_env
  File "/nix/store/z65l1jqvxa58zzwwa3bvglb6asj4y8cv-python3-3.8.5/lib/python3.8/venv/__init__.py", line 391, in create
    builder.create(env_dir)
  File "/nix/store/z65l1jqvxa58zzwwa3bvglb6asj4y8cv-python3-3.8.5/lib/python3.8/venv/__init__.py", line 68, in create
    self._setup_pip(context)
  File "/nix/store/z65l1jqvxa58zzwwa3bvglb6asj4y8cv-python3-3.8.5/lib/python3.8/venv/__init__.py", line 289, in _setup_pip
    subprocess.check_output(cmd, stderr=subprocess.STDOUT)
  File "/nix/store/z65l1jqvxa58zzwwa3bvglb6asj4y8cv-python3-3.8.5/lib/python3.8/subprocess.py", line 411, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/nix/store/z65l1jqvxa58zzwwa3bvglb6asj4y8cv-python3-3.8.5/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/simon/.local/share/nvim/black/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.

When uncommenting shutil.rmtree(virtualenv_path) (line 128) and running /home/simon/.local/share/nvim/black/bin/python3 -Im ensurepip --upgrade --default-pip, the following error is shown:

Looking in links: /tmp/tmpkcyc_0a6
Processing /tmp/tmpkcyc_0a6/setuptools-47.1.0-py3-none-any.whl
Processing /tmp/tmpkcyc_0a6/pip-20.1.1-py2.py3-none-any.whl
Installing collected packages: setuptools, pip
ERROR: Could not install packages due to an EnvironmentError: [Errno 30] Read-only file system: '/nix/store/x718m62fiyfx6xfqrv2f33i7m3pfbrmi-python3-3.8.5-env/lib/python3.8/site-packages/easy_install.py'

To Reproduce

1. Install the black vim plugin with vim-plug on neovim on NixOS
2. Open neovim

Expected behavior

The plugin creates the virtualenv in the specified location.

Environment

• Version: every commit from 68a3c75eb2a7096a7d5b3cfa6a6f3a5b894a1d72
• OS and Python version: NixOS/Python 3.8.5

Does this bug also happen on master? Yes

Additional context

When installing neovim on NixOS, the python module in neovim includes pynvim and msgpack for which nix creates an own python installation (in my case /nix/store/x718m62fiyfx6xfqrv2f33i7m3pfbrmi-python3-3.8.5-env/bin/python3.8). The normal python installation without any modules in my case is /nix/store/z65l1jqvxa58zzwwa3bvglb6asj4y8cv-python3-3.8.5/bin/python3.8). The python3.8 executable in vim’s python environment is just a shell script wrapper that sets environment variables and then calls the normal python executable.

This leads to sys._base_executable (pointing to normal python) being different from sys.executable (pointing to vim’s python) in vim’s python environment. Since the plugin sets both to the same executable (vim’s python’s), creating the virtualenv fails.

I’m not sure how to fix this without reintroducing #1379.

[nacnudus]

I get the same error on Arch Linux.

[MatrixManAtYrService]

It's not perfect, but this workaround works for me. It just ignores the extension entirely and invokes the black cli against the current file on save.

home.packages = with pkgs; [
    neovim
    black
]

programs.neovim = {
  extraConfig = ''
    autocmd BufWritePost *.py execute '! black %'
  ''
}

[sbruder]

Thanks, that looks like a reasonable alternative. However, I have switched to formatting with pylsp’s black plugin instead, which does not require this plugin to be installed.