Closed jaraco closed 1 month ago
I've confirmed the issue can be replicated simply:
draft 🐚 pip download setuptools
Collecting setuptools
Using cached setuptools-74.1.2-py3-none-any.whl.metadata (6.7 kB)
Using cached setuptools-74.1.2-py3-none-any.whl (1.3 MB)
Saved ./setuptools-74.1.2-py3-none-any.whl
Successfully downloaded setuptools
draft 🐚 py -c 'import setuptools'
Traceback (most recent call last):
File "<string>", line 1, in <module>
import setuptools
ModuleNotFoundError: No module named 'setuptools'
draft [1] 🐚 env PYTHONPATH=./setuptools-74.1.2-py3-none-any.whl py -c 'import setuptools'
Traceback (most recent call last):
File "<string>", line 1, in <module>
import setuptools
File "/Users/jaraco/draft/setuptools-74.1.2-py3-none-any.whl/setuptools/__init__.py", line 27, in <module>
from .dist import Distribution
File "/Users/jaraco/draft/setuptools-74.1.2-py3-none-any.whl/setuptools/dist.py", line 18, in <module>
from . import (
...<3 lines>...
)
File "/Users/jaraco/draft/setuptools-74.1.2-py3-none-any.whl/setuptools/_entry_points.py", line 5, in <module>
from jaraco.functools import pass_none
ModuleNotFoundError: No module named 'jaraco'
I've confirmed that the path is being set correctly.
> /Users/jaraco/draft/setuptools-74.1.2-py3-none-any.whl/setuptools/__init__.py(21)<module>()
-> sys.modules.pop("backports", None)
(Pdb) sys.path
['', '/Users/jaraco/draft/setuptools-74.1.2-py3-none-any.whl', '/opt/python/lib/python313t.zip', '/opt/python/lib/python3.13t', '/opt/python/lib/python3.13t/lib-dynload', '/opt/python/lib/python3.13t/site-packages', '/Users/jaraco/draft/setuptools-74.1.2-py3-none-any.whl/setuptools/_vendor']
According to the zipimport docs:
The ZIP archive can contain a subdirectory structure to support package imports, and a path within the archive can be specified to only import from a subdirectory. For example, the path
example.zip/lib/
would only import from thelib/
subdirectory within the archive.
I've found that even explicitly setting that path doesn't allow the vendored imports to be imported:
draft [1] 🐚 env PYTHONPATH=setuptools-74.12.2-py3-none-any.whl/setuptools/_vendor py -c 'import packaging'
Traceback (most recent call last):
File "<string>", line 1, in <module>
import packaging
ModuleNotFoundError: No module named 'packaging'
It seems to be a bug in zipimporter and not Setuptools.
This is duplicate of https://github.com/pypa/setuptools/issues/4640
Interestingly, I'm unable to replicate the behavior using a simple archive:
draft 🐚 rm -r *
draft 🐚 mkdir lib
draft 🐚 mkdir lib/packaging
draft 🐚 touch lib/packaging/__init__.py
draft 🐚 py -m zipfile --create libs.zip lib
draft 🐚 py -m zipfile --list libs.zip
File Name Modified Size
lib/ 2024-09-09 11:15:38 0
lib/packaging/ 2024-09-09 11:15:44 0
lib/packaging/__init__.py 2024-09-09 11:15:44 0
draft 🐚 env PYTHONPATH=libs.zip/lib py -c "import packaging"
draft 🐚
I seem to recall something recently about zipimporter having trouble with zip files whose directories don't have explicit entries. Let's see if that's a factor.
I've confirmed that the setuptools wheel doesn't have directory entries:
I've found one reference indicating that zipimport can't load such archives.
@gaborbernat Can you confirm that the archives used in virtualenv have the same issue (no explicit directory entries)?
It's a known bug in Python (https://github.com/python/cpython/issues/59110), fixed in Python 3.14 apparently. I've been using Python 3.13.0rc1+.
Prior to that change, Python considered zip files with no explicit directories to be "broken".
Be that so, the change to the simplified vendoring is breaking existing behavior :D and 3.13 EOL is very far :D
Downloading the zip archive from GitHub, which has the explicit directories, doesn't encounter the issue:
draft 🐚 http -q --download https://github.com/pypa/setuptools/archive/refs/tags/v74.1.2.zip
draft 🐚 env PYTHONPATH=setuptools-74.1.2.zip/setuptools-74.1.2 py -c 'import setuptools' && echo done
done
The workarounds/solutions are several:
setuptools[core]
dependencies explicitly in the archive (may still not work for the namespace package without explicit directory entries)I don't think there's anything Setuptools can or should do here. Reviving the re-write vendoring technique is not viable.
- include explicit directory entries in the relevant zip archive
If you are not controlling creating the archive, this will not work.
- install the
setuptools[core]
dependencies explicitly in the archive (may still not work for the namespace package without explicit directory entries)
You already highlighted why this might not work.
- rely on Python 3.14 or later
Not really a solution for at least another year for apps. Not a solution at all until another 6 for libraries.
- pin to Setuptools < 71
This can work but means no bug fixes or feature additions for people using this feature.
So none of the workarounds is an all around solution IMHO.
I'm not aware of any options that Setuptools could employ to overcome this situation. If someone else has suggestions, I'm open to ideas.
The previous vendoring solution did not manifest this issue, but I understand you do not want to back.
In https://github.com/pypa/setuptools/pull/4457#discussion_r1748703788_, @gaborbernat reported that the simplified vendoring strategy means vendored dependencies are no longer working for zipimporter archives.
Although the best recommended solution is for downstream integrators to incorporate the dependencies (
setuptools[core]
) naturally, it would be nice if the vendoring could continue to work until dependencies can be fully declared naturally (e.g. undersetuptools
).