No longer works in Python 3.12

[HiPhish]

I just updated my system Python to 3.12 and I have two issues:

• pynvim cannot be installed outside of a venv
• Inside a venv I can install it but I get an error when I run UpdateRemotePlugins

---

PEP 688 introduces the concept of externally managed environments. The idea is to prevent users from messing up their system by installing packages which might conflict with system packages, even when using the --user option. The only way to use pip is either as pipx (which is not applicable for pynvim) or in a venv.

Here is the error message I get:

$ pip install --user pynvim
error: externally-managed-environment

× This environment is externally managed
╰─>
    This system-wide Python installation is managed by the Void Linux package
    manager, XBPS. Installation of Python packages from other sources is not
    normally allowed.

    To install a Python package not offered by Void Linux, consider using a virtual
    environment, e.g.:

    python3 -m venv /path/to/venv
    /path/to/venv/pip install <package>

    Appending the flag --system-site-packages to the first command will give the
    virtual environment access to any Python package installed via XBPS.

    Invoking python, pip, and executables installed by pip in /path/to/venv/bin
    should automatically use the virtual environment. Alternatively, source its
    activation script to add the environment to the command search path for a shell:

    . /path/to/venv/activate

    After activation, running

    deactivate

    will remove the environment from the search path without destroying it.

    The XBPS package python3-pipx provides pipx, a convenient tool to automatically
    manage virtual environments for individual Python applications.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

---

Next I tried creating a venv and running pip install pynvim in it. This worked fine, but when I executed :UpdateRemotePlugins I got the following error:

function remote#host#UpdateRemotePlugins[6]..<SNR>92_RegistrationCommands[15]..remote#host#Require[10]..provider#pythonx#Require[1
2]..provider#Poll, line 7
Vim(if):Error invoking 'poll' on channel 3:^@Invalid channel: 3
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/hiphish/Desktop/derp/.venv/lib/python3.12/site-packages/neovim/__init__.py", line 5, in <module>
    import pynvim
  File "/home/hiphish/Desktop/derp/.venv/lib/python3.12/site-packages/pynvim/__init__.py", line 9, in <module>
    from pynvim.api import Nvim, NvimError
  File "/home/hiphish/Desktop/derp/.venv/lib/python3.12/site-packages/pynvim/api/__init__.py", line 7, in <module>
    from pynvim.api.buffer import Buffer
  File "/home/hiphish/Desktop/derp/.venv/lib/python3.12/site-packages/pynvim/api/buffer.py", line 2, in <module>
    from pynvim.api.common import Remote
  File "/home/hiphish/Desktop/derp/.venv/lib/python3.12/site-packages/pynvim/api/common.py", line 6, in <module>
    from pynvim.compat import unicode_errors_default
  File "/home/hiphish/Desktop/derp/.venv/lib/python3.12/site-packages/pynvim/compat.py", line 5, in <module>
    from imp import find_module as original_find_module
ModuleNotFoundError: No module named 'imp'
function remote#host#UpdateRemotePlugins[6]..<SNR>92_RegistrationCommands[15]..remote#host#Require[10]..provider#pythonx#Require[12]..provider#Poll, line 17
Failed to load python3 host. You can try to see what happened by starting nvim with $NVIM_PYTHON_LOG_FILE set and opening the generated log file. Also, the host stderr is available in messages.
Vim(if):Error invoking 'Poll' on channel 4:^@Invalid channel: 4
function remote#host#UpdateRemotePlugins[6]..SNR>92_RegistrationCommands[15]..remote#host#Require[10]..lisp#RequireLispHost, line 26
remote/host: generated rplugin manifest: /home/hiphish/.local/share/nvim/rplugin.vim

---

The first problem is unfortunate because it means any Python-based plugin is now broken unless I am working on a Python project. Remote plugins have fallen somewhat out of favor since we have Lua, but they still have value if I really want to use a Python library. The second problem however breaks Python remote plugins entirely.

[justinmk]

Did you try the unreleased version? https://github.com/neovim/pynvim/issues/538 mentions an issue with imp, which should be fixed by the next pynvim release.

[HiPhish]

Yes, the master branch fixes the second issue. Should I close this one?

[wookayin]

Yes this has already been fixed by #534. Until released, you can install the master version:

pip install 'pynvim @ git+https://github.com/neovim/pynvim'

[HiPhish]

I can override the installation issue by passing the --break-system-packages flag to pip. So the full command is

pip install --break-system-packages --user 'pynvim @ git+https://github.com/neovim/pynvim'

The name --break-system-packages sounds scary but it won't break the system as long as --user is used to install packages to the user's home directory instead of globally. It might break the user's personal configuration, but that can be solved by throwing away the Python files:

# Remove files manually, may leave residue files in ~/.local/bin
rm -rf ~/.local/include/python3.* ~/.local/lib/python3.*

This is good enough for me.