Error: html5lib not present when doing pip install inside a virtualenv

[vfrico]

Description

I've created a virtualenv using the 3.10 interpreter, but pip does not work inside it.

➜  ~ cd /tmp 
➜  /tmp virtualenv --python=/usr/bin/python3.10 v2     

created virtual environment CPython3.10.0.final.0-64 in 189ms
  creator CPython3Posix(dest=/tmp/v2, clear=False, global=False)
  seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, pkg_resources=latest, via=copy, app_data_dir=/home/victor/.local/share/virtualenv/seed-app-data/v1.0.1.debian.1)
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
➜  /tmp source v2/bin/activate
(v2) ➜  /tmp python
Python 3.10.0 (default, Oct  4 2021, 22:09:55) [GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()
(v2) ➜  /tmp pip install requests
Traceback (most recent call last):
  File "/tmp/v2/bin/pip", line 8, in <module>
    sys.exit(main())
  File "/tmp/v2/lib/python3.10/site-packages/pip/_internal/cli/main.py", line 73, in main
    command = create_command(cmd_name, isolated=("--isolated" in cmd_args))
  File "/tmp/v2/lib/python3.10/site-packages/pip/_internal/commands/__init__.py", line 96, in create_command
    module = importlib.import_module(module_path)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/tmp/v2/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 24, in <module>
    from pip._internal.cli.req_command import RequirementCommand
  File "/tmp/v2/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 15, in <module>
    from pip._internal.index.package_finder import PackageFinder
  File "/tmp/v2/lib/python3.10/site-packages/pip/_internal/index/package_finder.py", line 21, in <module>
    from pip._internal.index.collector import parse_links
  File "/tmp/v2/lib/python3.10/site-packages/pip/_internal/index/collector.py", line 12, in <module>
    from pip._vendor import html5lib, requests
ImportError: cannot import name 'html5lib' from 'pip._vendor' (/tmp/v2/lib/python3.10/site-packages/pip/_vendor/__init__.py)
(v2) ➜  /tmp 

os information

I have installed all the packages available for Ubuntu 20.04 in the PPA repository

lsb_release -a

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.3 LTS
Release:        20.04
Codename:       focal

uname -a

Linux ***machine-name*** 5.11.0-38-generic #42~20.04.1-Ubuntu SMP Tue Sep 28 20:41:07 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

[asottile]

this is an unfortunate packaging decision on the part of debian, I would suggest using virtialenv from pypi instead (which won't have the devendoring blunder)

they've fixed this for the next lts but nothing can really be done for 20.04

[RixzZ]

I'm getting this same behavior using python3.10 -m pip outside a virtualenv too. @asottile Is that for the same reason?

I was about to do a python3.10 -m pip install --user virtualenv to follow your previous advice. How would you go about that?

[ntoxeg]

I also got that behaviour from preinstalled pip, maybe it’s because of the old version (20.0.2 vs 21.3.1)? Unfortunately get-pip.py doesn’t work when used for the global install — for some reason it just breaks pip (I get no module named “pip” afterwards, assuming you set Python 3.10 as default using update-alternatives, otherwise it installs pip for the default python even if you invoke with python3.10). I seem to have got around this by just user-installing pip: python3.10 get-pip.py --user.

[asottile]

yeah that's tracked in #182 -- basically cpython changed the sysconfig initialization again and I haven't copied the relevant patch yet from debian

[RixzZ]

yeah that's tracked in #182 -- basically cpython changed the sysconfig initialization again and I haven't copied the relevant patch yet from debian

Oh, okay. Thanks.

I seem to have got around this by just user-installing pip: python3.10 get-pip.py --user.

I'll use this then meanwhile. Thanks.

[khink]

For the benefit (maybe) of other Debian / Ubuntu users who find this issue after a web search: Another workaround is to create the virtualenv with python3.10 -m venv $ENV_NAME. For me, this solved it.