jaraco
commented
3 years ago That error occurs when there is metadata left over from or found in an older version of Setuptools.
Closed kloczek closed 3 years ago
jaraco
commented
3 years ago That error occurs when there is metadata left over from or found in an older version of Setuptools.
jaraco
commented
3 years ago I'm able to replicate the issue by ensuring that an older Setuptools is present on the path and then running the build command:
setuptools main $ pip-run -q build wheel 'setuptools<58' -- -m build --no-isolation --wheel
* Getting dependencies for wheel...
Traceback (most recent call last):
File "/Users/jaraco/code/public/pypa/setuptools/pkg_resources/__init__.py", line 2458, in resolve
return functools.reduce(getattr, self.attrs, module)
AttributeError: type object 'Distribution' has no attribute '_finalize_2to3_doctests'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-axcfpfsg/pep517/in_process/_in_process.py", line 349, in <module>
main()
File "/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-axcfpfsg/pep517/in_process/_in_process.py", line 331, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-axcfpfsg/pep517/in_process/_in_process.py", line 117, in get_requires_for_build_wheel
return hook(config_settings)
File "/Users/jaraco/code/public/pypa/setuptools/setuptools/build_meta.py", line 154, in get_requires_for_build_wheel
return self._get_build_requires(
File "/Users/jaraco/code/public/pypa/setuptools/setuptools/build_meta.py", line 135, in _get_build_requires
self.run_setup()
File "/Users/jaraco/code/public/pypa/setuptools/setuptools/build_meta.py", line 150, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 87, in <module>
dist = setuptools.setup(**setup_params)
File "/Users/jaraco/code/public/pypa/setuptools/setuptools/__init__.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/distutils/core.py", line 108, in setup
_setup_distribution = dist = klass(attrs)
File "/Users/jaraco/code/public/pypa/setuptools/setuptools/dist.py", line 446, in __init__
_Distribution.__init__(
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/distutils/dist.py", line 292, in __init__
self.finalize_options()
File "/Users/jaraco/code/public/pypa/setuptools/setuptools/dist.py", line 821, in finalize_options
for ep in sorted(eps, key=by_order):
File "/Users/jaraco/code/public/pypa/setuptools/setuptools/dist.py", line 820, in <lambda>
eps = map(lambda e: e.load(), pkg_resources.iter_entry_points(group))
File "/Users/jaraco/code/public/pypa/setuptools/pkg_resources/__init__.py", line 2450, in load
return self.resolve()
File "/Users/jaraco/code/public/pypa/setuptools/pkg_resources/__init__.py", line 2460, in resolve
raise ImportError(str(exc)) from exc
ImportError: type object 'Distribution' has no attribute '_finalize_2to3_doctests'
ERROR Backend subproccess exited when trying to invoke get_requires_for_build_wheelIn fact, I encountered this error in the release. I worked around it by cutting the release locally in an environment without an existing Setuptools.
The problem goes away once one build succeeds, because then the metadata for the local build is present and takes precedence.
jaraco
commented
3 years ago Interestingly, and somewhat surprisingly, the issue also occurs with isolated builds.
setuptools main $ pip-run -q build 'setuptools<58' -- -m build --wheel
* Creating venv isolated environment...
* Installing packages in isolated environment... (wheel)
* Getting dependencies for wheel...
Traceback (most recent call last):
File "/Users/jaraco/code/public/pypa/setuptools/pkg_resources/__init__.py", line 2458, in resolve
return functools.reduce(getattr, self.attrs, module)
AttributeError: type object 'Distribution' has no attribute '_finalize_2to3_doctests'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-_2f120tf/pep517/in_process/_in_process.py", line 349, in <module>
main()
File "/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-_2f120tf/pep517/in_process/_in_process.py", line 331, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-_2f120tf/pep517/in_process/_in_process.py", line 117, in get_requires_for_build_wheel
return hook(config_settings)
File "/Users/jaraco/code/public/pypa/setuptools/setuptools/build_meta.py", line 154, in get_requires_for_build_wheel
return self._get_build_requires(
File "/Users/jaraco/code/public/pypa/setuptools/setuptools/build_meta.py", line 135, in _get_build_requires
self.run_setup()
File "/Users/jaraco/code/public/pypa/setuptools/setuptools/build_meta.py", line 150, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 87, in <module>
dist = setuptools.setup(**setup_params)
File "/Users/jaraco/code/public/pypa/setuptools/setuptools/__init__.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/distutils/core.py", line 108, in setup
_setup_distribution = dist = klass(attrs)
File "/Users/jaraco/code/public/pypa/setuptools/setuptools/dist.py", line 446, in __init__
_Distribution.__init__(
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/distutils/dist.py", line 292, in __init__
self.finalize_options()
File "/Users/jaraco/code/public/pypa/setuptools/setuptools/dist.py", line 821, in finalize_options
for ep in sorted(eps, key=by_order):
File "/Users/jaraco/code/public/pypa/setuptools/setuptools/dist.py", line 820, in <lambda>
eps = map(lambda e: e.load(), pkg_resources.iter_entry_points(group))
File "/Users/jaraco/code/public/pypa/setuptools/pkg_resources/__init__.py", line 2450, in load
return self.resolve()
File "/Users/jaraco/code/public/pypa/setuptools/pkg_resources/__init__.py", line 2460, in resolve
raise ImportError(str(exc)) from exc
ImportError: type object 'Distribution' has no attribute '_finalize_2to3_doctests'
ERROR Backend subproccess exited when trying to invoke get_requires_for_build_wheelI'm guessing that's because isolated builds still get a copy of some version of Setuptools (whatever venv bundles). Except no, because if setuptools<58 is not installed into the working environment, the build succeeds.
setuptools main $ pip-run -q build -- -m build --wheel
* Creating venv isolated environment...
* Installing packages in isolated environment... (wheel)
* Getting dependencies for wheel...
running egg_info
creating setuptools.egg-info
writing setuptools.egg-info/PKG-INFO
writing dependency_links to setuptools.egg-info/dependency_links.txt
writing entry points to setuptools.egg-info/entry_points.txt
writing requirements to setuptools.egg-info/requires.txt
writing manifest file 'setuptools.egg-info/SOURCES.txt'
package init file 'setuptools/tests/indexes/__init__.py' not found (or not a regular file)
package init file 'setuptools/tests/indexes/test_links_priority/__init__.py' not found (or not a regular file)
package init file 'setuptools/tests/indexes/test_links_priority/simple/__init__.py' not found (or not a regular file)
package init file 'setuptools/tests/indexes/test_links_priority/simple/foobar/__init__.py' not found (or not a regular file)
package init file 'pkg_resources/tests/data/__init__.py' not found (or not a regular file)
...
adding 'setuptools-58.0.0.post20210906.dist-info/top_level.txt'
adding 'setuptools-58.0.0.post20210906.dist-info/RECORD'
removing build/bdist.macosx-10.9-universal2/wheel
Successfully built setuptools-58.0.0.post20210906-py3-none-any.whlSo now I'm suspecting the issue is that by using PYTHONPATH to add setuptools<58 (that's what pip-run does), it gets added to the isolated build environment.
The root cause is that Setuptools (intentionally) removed the entry point in order to remove that behavior, but if the metadata for an older version of Setuptools is present, it takes precedence.
The question I'd like to answer is: how pervasive is the failure and how necessary is a workaround?
I can think of a few possible workarounds:
The first two workarounds have the advantage that they add lasting value (avoiding similar issues in the future), but they have the disadvantage of imposing a burden/constraint on the downstream builders.
The latter two options would provide a widespread workaround but have two disadvantages: it won't improve the situation for future similar changes, and it adds debt that will need to be removed at some unclear point in the future.
I suspect it won't be easy for downstream builders to generally avoid having Setuptools installed, so I'm leaning toward one of the Setuptools-owned workarounds.
kloczek
commented
3 years ago Interestingly, and somewhat surprisingly, the issue also occurs with isolated builds.
For me remove --no-isolation solved the issue.
And my understanding is that how setuptools should be build because dhis module sits on top of whole dependency pyramyd and running build with isolation woild be good because it would allow avoid kind of bootstrapping process.
After build I see that still there are some spinx (4.1.2) warnings on generate man page (but IIRC I've already reported that) and pytest shows some new issues.
Just retested and setuptools 57.5.0 still builds correctly in the same build env.
fransklaver
commented
3 years ago Hi,
Mostly FYI, and I'm not trying to suggest that keeping crossenv working is up to you. Still the information might be useful. So I am running into similar issues with a crossenv setup for aarch64, where a numpy install fails with the exact same error. I do come from an older version of setuptools, but it seems to me that once everything is updated, the older version shouldn't have any influence anymore.
It reproduces for me on python-3.8.8, I don't have a cross-built 3.9 on hand (yet).
It basically boils down to:
#!/bin/bash
buildpython=$1
hostpython=$2
tmpenv=$(mktemp -d)
crossenv=$(mktemp -d)
echo "Preparing initial crossenv venv"
$buildpython -m venv --clear $tmpenv
$tmpenv/bin/pip install --upgrade pip wheel setuptools crossenv
echo "generating cross-env python environment"
$tmpenv/bin/python -m crossenv $hostpython --clear $crossenv
$crossenv/bin/cross-pip install --upgrade pip wheel setuptools
$crossenv/bin/build-pip install --upgrade pip wheel setuptools
$crossenv/bin/build-pip install cython cffi
$crossenv/bin/cross-pip install numpy
# and cleanup again
rm -rf $crossenv
rm -rf $tmpenvAnd the output until the first numpy failure (it retries):
Preparing initial crossenv venv
Collecting pip
Using cached pip-21.2.4-py3-none-any.whl (1.6 MB)
Collecting wheel
Using cached wheel-0.37.0-py2.py3-none-any.whl (35 kB)
Collecting setuptools
Using cached setuptools-58.0.0-py3-none-any.whl (816 kB)
Collecting crossenv
Using cached crossenv-1.1.4-py3-none-any.whl (28 kB)
Installing collected packages: pip, wheel, setuptools, crossenv
Attempting uninstall: pip
Found existing installation: pip 20.2.3
Uninstalling pip-20.2.3:
Successfully uninstalled pip-20.2.3
Attempting uninstall: setuptools
Found existing installation: setuptools 49.2.1
Uninstalling setuptools-49.2.1:
Successfully uninstalled setuptools-49.2.1
Successfully installed crossenv-1.1.4 pip-21.2.4 setuptools-58.0.0 wheel-0.37.0
generating cross-env python environment
Collecting pip
Using cached pip-21.2.4-py3-none-any.whl (1.6 MB)
Collecting wheel
Using cached wheel-0.37.0-py2.py3-none-any.whl (35 kB)
Collecting setuptools
Using cached setuptools-58.0.0-py3-none-any.whl (816 kB)
Installing collected packages: pip, wheel, setuptools
Attempting uninstall: pip
Found existing installation: pip 20.2.3
Uninstalling pip-20.2.3:
Successfully uninstalled pip-20.2.3
Attempting uninstall: setuptools
Found existing installation: setuptools 49.2.1
Uninstalling setuptools-49.2.1:
Successfully uninstalled setuptools-49.2.1
Successfully installed pip-21.2.4 setuptools-58.0.0 wheel-0.37.0
Collecting pip
Using cached pip-21.2.4-py3-none-any.whl (1.6 MB)
Collecting wheel
Using cached wheel-0.37.0-py2.py3-none-any.whl (35 kB)
Collecting setuptools
Using cached setuptools-58.0.0-py3-none-any.whl (816 kB)
Installing collected packages: pip, wheel, setuptools
Attempting uninstall: pip
Found existing installation: pip 20.2.3
Uninstalling pip-20.2.3:
Successfully uninstalled pip-20.2.3
Attempting uninstall: setuptools
Found existing installation: setuptools 49.2.1
Uninstalling setuptools-49.2.1:
Successfully uninstalled setuptools-49.2.1
Successfully installed pip-21.2.4 setuptools-58.0.0 wheel-0.37.0
Collecting cython
Using cached Cython-0.29.24-cp38-cp38-manylinux1_x86_64.whl (1.9 MB)
Collecting cffi
Using cached cffi-1.14.6-cp38-cp38-manylinux1_x86_64.whl (411 kB)
Collecting pycparser
Using cached pycparser-2.20-py2.py3-none-any.whl (112 kB)
Installing collected packages: pycparser, cython, cffi
Successfully installed cffi-1.14.6 cython-0.29.24 pycparser-2.20
Collecting numpy
Using cached numpy-1.21.2.zip (10.3 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... error
ERROR: Command errored out with exit status 1:
command: /tmp/tmp.qy5LolDavD/cross/bin/python /tmp/tmp.qy5LolDavD/build/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpspvqwyg9
cwd: /tmp/pip-install-u_lblyd1/numpy_db56c39523754d9da4e849d605d03590
Complete output (44 lines):
Running from numpy source directory.
Traceback (most recent call last):
File "/tmp/tmp.qy5LolDavD/build/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2458, in resolve
return functools.reduce(getattr, self.attrs, module)
AttributeError: type object 'Distribution' has no attribute '_finalize_2to3_doctests'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/tmp/tmp.qy5LolDavD/build/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 349, in <module>
main()
File "/tmp/tmp.qy5LolDavD/build/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 331, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/tmp/tmp.qy5LolDavD/build/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 117, in get_requires_for_build_wheel
return hook(config_settings)
File "/tmp/tmp.qy5LolDavD/build/lib/python3.8/site-packages/setuptools/build_meta.py", line 154, in get_requires_for_build_wheel
return self._get_build_requires(
File "/tmp/tmp.qy5LolDavD/build/lib/python3.8/site-packages/setuptools/build_meta.py", line 135, in _get_build_requires
self.run_setup()
File "/tmp/tmp.qy5LolDavD/build/lib/python3.8/site-packages/setuptools/build_meta.py", line 258, in run_setup
super(_BuildMetaLegacyBackend,
File "/tmp/tmp.qy5LolDavD/build/lib/python3.8/site-packages/setuptools/build_meta.py", line 150, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 448, in <module>
setup_package()
File "setup.py", line 440, in setup_package
setup(**metadata)
File "/tmp/tmp.qy5LolDavD/build/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/opt/build-support/lib/python3.8/distutils/core.py", line 108, in setup
_setup_distribution = dist = klass(attrs)
File "/tmp/tmp.qy5LolDavD/build/lib/python3.8/site-packages/setuptools/dist.py", line 446, in __init__
_Distribution.__init__(
File "/opt/build-support/lib/python3.8/distutils/dist.py", line 292, in __init__
self.finalize_options()
File "/tmp/tmp.qy5LolDavD/build/lib/python3.8/site-packages/setuptools/dist.py", line 821, in finalize_options
for ep in sorted(eps, key=by_order):
File "/tmp/tmp.qy5LolDavD/build/lib/python3.8/site-packages/setuptools/dist.py", line 820, in <lambda>
eps = map(lambda e: e.load(), pkg_resources.iter_entry_points(group))
File "/tmp/tmp.qy5LolDavD/build/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2450, in load
return self.resolve()
File "/tmp/tmp.qy5LolDavD/build/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2460, in resolve
raise ImportError(str(exc)) from exc
ImportError: type object 'Distribution' has no attribute '_finalize_2to3_doctests'
----------------------------------------Hope this helps rather than confuses.
jaraco
commented
3 years ago I am running into similar issues with a crossenv setup...
Hope this helps rather than confuses.
Not your fault, but I have zero experience with crossenv, so it's not at all apparent what's going on above.
I suspect, though, that crossenv is doing something similar to what pip-run is doing - using PYTHONPATH to manipulate the presence of certain dependencies. With all these layers of isolation, it's really difficult to discern what factors trigger the behavior. Given this additional use-case, I'm now convinced a workaround in Setuptools is needed.
jaraco
commented
3 years ago pytest shows some new issues
Those issues should be reported separately.
fransklaver
commented
3 years ago Not your fault, but I have zero experience with crossenv, so it's not at all apparent what's going on above.
Can't blame you for that.
I suspect, though, that crossenv is doing something similar to what pip-run is doing - using PYTHONPATH to manipulate the presence of certain dependencies. With all these layers of isolation, it's really difficult to discern what factors trigger the behavior. Given this additional use-case, I'm now convinced a workaround in Setuptools is needed.
crossenv is a bit of a beast. In the example above I create a python env to install and setup crossenv. Crossenv then builds its own environment, but needs two of them, one to build the packages (build-*) and one to actually install in the target environment (cross-*). I'm sure your gut feeling about it is close to how it really works. I'm looking forward to an update to check.
Thanks.
kloczek
commented
3 years ago What about pytest failures? Should I open separated ticket?
jaraco
commented
3 years ago What about pytest failures? Should I open separated ticket?
Yes, please. Note that the errors aren't occurring on CI, so I'll need your assistance identifying the factors unique to your environment that lead to the failures.
fransklaver
commented
3 years ago Builds work again for me. Thanks.
setuptools version
58.0.0
Python version
3.8.12
OS
Linux/x86_64
Additional environment information
build version 0.6.0
Description
With
setuptoolsI was able to use build procedure consisting from:python3 -m build --no-isolation --wheelegg_infobypython3 setup.py egg_infoIn new version that procedure fails on first step.
Expected behavior
buildshould produce .whl file.How to Reproduce
SETUPTOOLS_SCM_PRETEND_VERSION=58.0.0 python3 -m build --no-isolation --wheelOutput
Code of Conduct