Open aorumbayev opened 2 months ago
import pkg_resources
should not be broken if setuptools is installed. You have an old version somehow, which probably means setuptools was capped in some package to a version that doesn't support Python 3.12. What version of setuptools is in your pipx environment?
Ahh, this is getting the surrounding environment. What is your PYTHONPATH set to? Setting that is one way to break pipx, I think.
@henryiii, could you please confirm if you've had a chance to execute the public repository that demonstrates the bug in a GitHub Codespace (specifically created to allow folks to easily reproduce the bug)?
Here's the sys.path
output from the pipx environment for the problematic package:
Python 3.12.2 (main, Apr 23 2024, 13:31:25) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print(sys.path)
['', '/home/codespace/.pyenv/versions/3.12.2/lib/python312.zip', '/home/codespace/.pyenv/versions/3.12.2/lib/python3.12', '/home/codespace/.pyenv/versions/3.12.2/lib/python3.12/lib-dynload', '/usr/local/py-utils/venvs/algokit/lib/python3.12/site-packages', '/usr/local/py-utils/shared/lib/python3.10/site-packages']
Setuptools is already installed in the virtual environment using Python 3.12.2:
Requirement already satisfied: setuptools in /home/codespace/.local/lib/python3.12/site-packages (69.2.0)
It appears that Python 3.10 directories are being added to sys.path
, likely due to pipx. There have been recommendations in similar pipx issues to perform a full reinstall of pipx to reset the shared virtual environment (created with Python 3.10). This suggests a possible challenge with packages requiring Python >=3.12 in GitHub Codespaces, as users might need to completely uninstall the preinstalled pipx and reinstall it with Python 3.12.
Additionally, using pipx reinstall-all
does not seem to update the shared virtual environment if it was initially set up with Python 3.10—pipx retains it. This behavior is unexpected and complicates environment management a lot :/
'/usr/local/py-utils/venvs/algokit/lib/python3.12/site-packages', '/usr/local/py-utils/shared/lib/python3.10/site-packages'
is a broken sys.path. There's no guarantee any package will work when you have two versions of site-packages for different Python's in your path. Pipx won't add another Python to your sys.path, and it will place directories at the top of the PATH, so that's already present on your system. You happen to hit it here because we don't require setuptools, but check to see if it's present. It is present, but in your Python 3.10 site packages, so it's mixing with the Python 3.12 environment and the whole thing breaks down.
If you fix your sys.path, then this problem goes away. You should not have Python 3.10 in your sys.path if you are using Python 3.12.
There are a few things that could be done to alleviate the problem (basically just push it off on some other package) from our side, though:
Of those, the most effort is the third one, but my preferred solution.
I really don't see where that python3.10 is sneaking in. Everything seems fine in the container, pyvenv.cfg
looks correct, no 3.10
's in the environment. Maybe there's an issue using pyenv
in the container that's already configured for 3.10? I don't really trust pyenv or tools to support it very well (due to the messy shim system). If you can use a better image in devcontainer.json
, or use a different mechanism to install Python (PDM and Hatch both can install Python, unlike Poetry), this might be resolved.
Same issue with pdm:
pipx install pdm
pdm python install cpython@3.12.2
pipx install . --force --python /home/codespace/.local/share/pdm/python/cpython@3.12.2/bin/python3
pipx-bug-cli
Though it's many times faster since these tools (pdm, hatch, rye) don't build Python from source like pyenv does.
I notice the Codespaces is detecting the Poetry file and doing some setup, not sure if that affects anything.
Also, POSIX has a "C" locale which we should also handle. We can easily fix that, which solves this if the locale is not set to non-English.
@henryiii thanks for the investigation so far! Curious if #680 fixed the issue? Will try testing it out in the codespace as well
Will finish up #680 after PyCon.
Describe the problem
The following repo was specifically created to outline a reproducible environment with the bug https://github.com/aorumbayev/pipx_bug.
The root cause is incompatibility with python 3.12. And this particular line is the root cause. I will also open this issue on pipx repo, so far i mentioned it on copier where copier maintainers forwarded me to this repo.
To Reproduce
Refer to readme on https://github.com/aorumbayev/pipx_bug
Logs
Expected behavior
plumbum works when used as a transitive dependency in packages installed via pipx
Operating system
Linux
Operating system distribution and version
ubuntu 20
Plumbum version
1.8.2
Python version
3.12.2
Installation method
plumbum is introduced as a transient dependency of copier. https://github.com/copier-org/copier/blob/85311ec658ea7162673489f70e7ec5a8d6b5367b/poetry.lock#L862