Continuing the topic in #182 to provide some background. I got here because people have started coming to pip’s issue tracker reporting this as a pip bug. pypa/pip#10710
Traditionally, pip uses distutils to decide the location to install packages, including pip itself. The default “prefix site” is (as the name suggests) in where Python is installed (/usr/lib.../site-packages), but since Debian wants to separate system-provided and externally installed packages, it patches distutils to use /usr/local/lib/.../site-packages, while also patching sys.path to use dist-packages under /usr/lib to indicate those packages are provided by the distribution.
However, to anticipate Python deprecating distutils in 3.10, pip switched to use sysconfig on Python 3.10 and later to find the locations. This means that Debian needs to patch sysconfig instead to make pip do the same thing.[^1] But from what I can tell, deadsnakes does not currently patch sysconfig, so a pip installed against its 3.10 would still be installing packages under the default, unpatched location. But since sys.path is still patched (the mechanism there does not change in 3.10), those installed packages cannot be found correctly.
The solution here would be, I believe, to patch sysconfig like distutils. Debian’s patch should work as a good starting point, if not used verbatim.
asottile
commented
2 years ago
Continuing the topic in #182 to provide some background. I got here because people have started coming to pip’s issue tracker reporting this as a pip bug. pypa/pip#10710
Traditionally, pip uses
distutilsto decide the location to install packages, including pip itself. The default “prefix site” is (as the name suggests) in where Python is installed (/usr/lib.../site-packages), but since Debian wants to separate system-provided and externally installed packages, it patchesdistutilsto use/usr/local/lib/.../site-packages, while also patchingsys.pathto usedist-packagesunder/usr/libto indicate those packages are provided by the distribution.However, to anticipate Python deprecating
distutilsin 3.10, pip switched to usesysconfigon Python 3.10 and later to find the locations. This means that Debian needs to patchsysconfiginstead to make pip do the same thing.[^1] But from what I can tell, deadsnakes does not currently patchsysconfig, so a pip installed against its 3.10 would still be installing packages under the default, unpatched location. But sincesys.pathis still patched (the mechanism there does not change in 3.10), those installed packages cannot be found correctly.The solution here would be, I believe, to patch
sysconfiglikedistutils. Debian’s patch should work as a good starting point, if not used verbatim.[^1]: Current patch: https://salsa.debian.org/cpython-team/python3/-/blob/master/debian/patches/sysconfigdata-name.diff