Closed stratakis closed 6 years ago
Thanks for the report.
We see six failures on Python 2 and seven on Python 3. Here's the first of the failures seen on the Python 2:
__________________________ test_dist_fetch_build_egg ___________________________
tmpdir = local('/tmp/pytest-of-mockbuild/pytest-0/test_dist_fetch_build_egg0')
def test_dist_fetch_build_egg(tmpdir):
"""
Check multiple calls to `Distribution.fetch_build_egg` work as expected.
"""
index = tmpdir.mkdir('index')
index_url = urljoin('file://', pathname2url(str(index)))
def sdist_with_index(distname, version):
dist_dir = index.mkdir(distname)
dist_sdist = '%s-%s.tar.gz' % (distname, version)
make_nspkg_sdist(str(dist_dir.join(dist_sdist)), distname, version)
with dist_dir.join('index.html').open('w') as fp:
fp.write(DALS(
'''
<!DOCTYPE html><html><body>
<a href="{dist_sdist}" rel="internal">{dist_sdist}</a><br/>
</body></html>
'''
).format(dist_sdist=dist_sdist))
sdist_with_index('barbazquux', '3.2.0')
sdist_with_index('barbazquux-runner', '2.11.1')
with tmpdir.join('setup.cfg').open('w') as fp:
fp.write(DALS(
'''
[easy_install]
index_url = {index_url}
'''
).format(index_url=index_url))
reqs = '''
barbazquux-runner
barbazquux
'''.split()
with tmpdir.as_cwd():
dist = Distribution()
resolved_dists = [
dist.fetch_build_egg(r)
> for r in reqs
]
setuptools/tests/test_dist.py:44:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
setuptools/dist.py:522: in fetch_build_egg
return cmd.easy_install(req)
setuptools/command/easy_install.py:671: in easy_install
return self.install_item(spec, dist.location, tmpdir, deps)
setuptools/command/easy_install.py:697: in install_item
dists = self.install_eggs(spec, download, tmpdir)
setuptools/command/easy_install.py:878: in install_eggs
return self.build_and_install(setup_script, setup_base)
setuptools/command/easy_install.py:1117: in build_and_install
self.run_setup(setup_script, setup_base, args)
setuptools/command/easy_install.py:1103: in run_setup
run_setup(setup_script, args)
setuptools/sandbox.py:257: in run_setup
raise
/usr/lib64/python2.7/contextlib.py:35: in __exit__
self.gen.throw(type, value, traceback)
setuptools/sandbox.py:199: in setup_context
yield
/usr/lib64/python2.7/contextlib.py:35: in __exit__
self.gen.throw(type, value, traceback)
setuptools/sandbox.py:170: in save_modules
saved_exc.resume()
setuptools/sandbox.py:145: in resume
six.reraise(type, exc, self._tb)
setuptools/sandbox.py:158: in save_modules
yield saved
setuptools/sandbox.py:199: in setup_context
yield
E RuntimeError: maximum recursion depth exceeded
!!! Recursion detected (same locals & position)
----------------------------- Captured stdout call -----------------------------
Searching for barbazquux-runner
Reading file:///tmp/pytest-of-mockbuild/pytest-0/test_dist_fetch_build_egg0/index/barbazquux-runner/
Best match: barbazquux-runner 2.11.1
Processing barbazquux-runner-2.11.1.tar.gz
Writing /tmp/easy_install-N4Cer1/setup.cfg
Running setup.py -q bdist_egg --dist-dir /tmp/easy_install-N4Cer1/egg-dist-tmp-sBSZb5
________ TestDistutilsPackage.test_bdist_egg_available_on_distutils_pkg ________
self = <setuptools.tests.test_easy_install.TestDistutilsPackage instance at 0x7f6b1e5e8908>
distutils_package = None
def test_bdist_egg_available_on_distutils_pkg(self, distutils_package):
> run_setup('setup.py', ['bdist_egg'])
/builddir/build/BUILD/setuptools-36.5.0/setuptools/tests/test_easy_install.py:323:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/builddir/build/BUILD/setuptools-36.5.0/setuptools/sandbox.py:257: in run_setup
raise
/usr/lib64/python2.7/contextlib.py:35: in __exit__
self.gen.throw(type, value, traceback)
/builddir/build/BUILD/setuptools-36.5.0/setuptools/sandbox.py:199: in setup_context
yield
/usr/lib64/python2.7/contextlib.py:35: in __exit__
self.gen.throw(type, value, traceback)
/builddir/build/BUILD/setuptools-36.5.0/setuptools/sandbox.py:170: in save_modules
saved_exc.resume()
/builddir/build/BUILD/setuptools-36.5.0/setuptools/sandbox.py:145: in resume
six.reraise(type, exc, self._tb)
/builddir/build/BUILD/setuptools-36.5.0/setuptools/sandbox.py:158: in save_modules
yield saved
/builddir/build/BUILD/setuptools-36.5.0/setuptools/sandbox.py:199: in setup_context
yield
E RuntimeError: maximum recursion depth exceeded
!!! Recursion detected (same locals & position)
I'm afraid I just don't see what might be going on here.
Can you bisect the changes between 36.2 and 36.5 to see in which release the problem began? If I had to guess, I would think it's 36.4.0.
Can you imagine what it might be about the Fedora environment that elicits this failure?
Hello and thanks for looking at the issue. I'll try to provide the information you requested.
Managed to figure out one of the culprits and it is pytest.
Tested so far only on python2 and it seems that since the release of pytest 3.0.0 those four test cases from test_easy_install shown at the build log, started failing with the maximum recursion error.
The same tests actually pass with pytest 2.9.2.
What version of pytest are you using exactly?
pytest 3.2.3 currently.
I can reproduce the issue using a virtualenv if I pin pytest-shutil to 1.2.6: rm -rf venv && python -m venv venv && ./venv/bin/pip install -r tests/requirements.txt 'pytest-shutil==1.2.6' -e . && ./venv/bin/pytest setuptools/tests
:
============================= test session starts ==============================
platform linux -- Python 3.6.3, pytest-3.2.3, py-1.4.34, pluggy-0.4.0
rootdir: /home/bpierre/progs/src/setuptools, inifile: pytest.ini
plugins: virtualenv-1.2.11, shutil-1.2.6, flake8-0.9.1
collected 240 items / 2 skipped
setuptools/tests/test_archive_util.py X
setuptools/tests/test_bdist_egg.py .
setuptools/tests/test_build_clib.py .
setuptools/tests/test_build_ext.py ..
setuptools/tests/test_build_py.py .
setuptools/tests/test_config.py .....................
setuptools/tests/test_dep_util.py .
setuptools/tests/test_depends.py .
setuptools/tests/test_develop.py ss....
setuptools/tests/test_dist.py F
setuptools/tests/test_dist_info.py ..
setuptools/tests/test_easy_install.py ......F......FFFF.........
setuptools/tests/test_egg_info.py ...................x..........
setuptools/tests/test_find_packages.py ............
setuptools/tests/test_install_scripts.py .s.s
setuptools/tests/test_integration.py sssssssss
setuptools/tests/test_manifest.py ...................................................................
setuptools/tests/test_namespaces.py ...
setuptools/tests/test_packageindex.py ..................
setuptools/tests/test_sandbox.py .........F
setuptools/tests/test_sdist.py ..........
setuptools/tests/test_setuptools.py ...
setuptools/tests/test_test.py .
setuptools/tests/test_unicode_utils.py .
setuptools/tests/test_upload_docs.py ..
setuptools/tests/test_virtualenv.py ...
setuptools/tests/test_windows_wrappers.py sss
I actually tried with shutil 1.2.11 and I still get the failures.
When switching to 1.2.11, the only difference is pytest-shutil itself (same versions for all other packages):
apipkg==1.4
contextlib2==0.5.5
execnet==1.5.0
flake8==3.5.0
mccabe==0.6.1
mock==2.0.0
path.py==10.5
pbr==3.1.1
py==1.4.34
pycodestyle==2.3.1
pyflakes==1.6.0
pytest==3.2.3
pytest-fixture-config==1.2.11
pytest-flake8==0.9.1
pytest-shutil==1.2.11
pytest-virtualenv==1.2.11
six==1.11.0
virtualenv==15.1.0
What versions are you using for those?
Differences: execnet==1.4.1 path.py==5.2 pytest-shutil==1.2.6
Those packages do not exist in the buildroot, we don't pull them to build setuptools: flake8==3.5.0 mccabe==0.6.1 pycodestyle==2.3.1 pyflakes==1.6.0 pytest-flake8==0.9.1
The rest are the same.
I get failures with this environment:
apipkg==1.4
contextlib2==0.5.5
execnet==1.4.1
mock==2.0.0
path.py==5.2
pbr==3.1.1
py==1.4.34
pytest==3.2.3
pytest-fixture-config==1.2.11
pytest-shutil==1.2.6
pytest-virtualenv==1.2.11
six==1.11.0
virtualenv==15.1.0
but it works if pytest-shutil is updated to 1.2.11. Note again that this is with Python 3, Python 2 works fine in both cases.
How are all those dependencies provided? From distribution packages? Because another difference could be unvendoring...
Those are from distribution packages yes, so there might be some deviations from the upstream code in some of those packages but not something that might affect that behaviour, AFAIK.
Also unvendoring of what could possibly cause this? We do not unvendor what setuptools carries within though.
Noting that for the subsequent comments/tests after the initial report, were running on python2, as I'd like to address that first. Will see if I'm able to reproduce it in a virtualenv.
Also @mcyprian you might wanna take a look at it.
If however pin pytest to 2.9.2 with rm -rf venv && python -m venv venv && ./venv/bin/pip install -r tests/requirements.txt 'pytest==2.9.2' -e . && ./venv/bin/py.test setuptools/tests
and remove as well the requirement for pytest-flake8 due to it requiring pytest>=3.2, the tests pass.
Don't you get an error with test_egg_info when using pytest==2.9.2:
________________________________________________________________________________ ERROR collecting setuptools/tests/test_egg_info.py ________________________________________________________________________________
setuptools/tests/test_egg_info.py:24: in <module>
class TestEggInfo(object):
setuptools/tests/test_egg_info.py:368: in TestEggInfo
mismatch_marker_alternate=mismatch_marker_alternate,
setuptools/tests/test_egg_info.py:216: in parametrize
argvalues.append(pytest.param(requires, use_cfg,
E AttributeError: module 'pytest' has no attribute 'param'
?
And using the same command line without pinning pytest, you get the same failures as originally reported?
@benoit-pierre Yep I get the error that you mention.
If I try however to reproduce the issues outside the build system, and replicate your environment I get the exact same behaviour that you get. Invoking the tests under python 2 is fine, while under python3 I get the recursion error if I pin shutil to 1.2.6.
I think the actual traceback is hidden due to pytest's recursion error. Will see if I can manage to extract a more meaningful traceback.
cc'ing @hroncok as well.
Funny thing:
(I have Fedora 27 with all the BuildRequires from our python-setuptools packages installed (the missing ones I got from Fedora 28.)
$ python3 -m venv __venv__
$ . __venv__/bin/activate
(__venv__) $ python -m pip install -r tests/requirements.txt
(__venv__) $ PYTHONPATH=$(pwd) python -m pytest -k easy_install
all good
(__venv__) $ deactivate
$ python3 -m venv __venv__ --system-site-packages
$ . __venv__/bin/activate
(__venv__) $ PYTHONPATH=$(pwd) python -m pytest -k easy_install
boom
This makes me think that something from my system is interfering with the tests. Will try to find out what. (EDIT: It was RPM installed pytest-flakes
.)
using --assert=plain
workarounds the issue, PYTHONDONTWRITEBYTECODE=1
also workarounds this issue
pytest is calling os.mkdir(cache_dir)
when setuptools.sandbox
is imported. Using --assert=plain
makes it not do "assert magic" at all, using PYTHONDONTWRITEBYTECODE=1
makes it not store the files and call mkdir. Setuptool's DirectorySandbox is trying to prevent the mkdir call. When it does, it imports from sandbox again and that triggers pytest to try to do the whole thing from start. Hence the infinite recursion.
Still no idea why it blows up only when system site packages are allowed :(
pytest's AssertionRewritingHook._must_rewrite
:
{'pytest_virtualenv', 'pytest_shutil', 'pytest_flake8', 'setup', 'setuptools.tests.fixtures'}
(notice setup
){'pytest_shutil', 'pytest_virtualenv', 'pytest_flake8', 'setuptools.tests.fixtures'}
as setuptools.sandbox
startswith setup
, this is causing a problem later at https://github.com/pytest-dev/pytest/blob/77bd0aa02ffc7f8734682c2e353135af5d94ab76/_pytest/assertion/rewrite.py#L170
I have found the difference with system site-packages:
import pprint
import pkg_resources
# from pytest's _mark_plugins_for_rewrite
metadata_files = 'RECORD', 'SOURCES.txt'
package_files = (
entry.split(',')[0]
for entrypoint in pkg_resources.iter_entry_points('pytest11')
for metadata in metadata_files
for entry in entrypoint.dist._get_metadata(metadata)
)
pprint.pprint(list(package_files))
If you install a pytest plugin (such as pytest-virtualenv or pytest-flake8) from a wheel, the list does not include setup.py
and pytest.cfg
. If you however install trough sdist, it does. This puts setup
to pytest's AssertionRewritingHook._must_rewrite
.
A reproducer:
$ python3 -m venv __venv_reproducer__
$ . __venv_reproducer__/bin/activate
(__venv_reproducer__) $ pip install https://pypi.python.org/packages/e5/87/345c1423d3dd7c27247b61c71192c9b94a5f980647389e622ac41ff92a3d/pytest-flake8-0.9.1.tar.gz#md5=f16c46d0fa88e07606b0c444d697d815
...
Successfully installed flake8-3.5.0 mccabe-0.6.1 py-1.5.2 pycodestyle-2.3.1 pyflakes-1.6.0 pytest-3.2.5 pytest-flake8-0.9.1
(__venv_reproducer__) $ pip install -r tests/requirements.txt
...
Requirement already satisfied: pytest-flake8 in ./__venv_reproducer__/lib/python3.6/site-packages (from -r tests/requirements.txt (line 3))
...
Successfully installed apipkg-1.4 contextlib2-0.5.5 execnet-1.5.0 mock-2.0.0 path.py-10.5 pbr-3.1.1 pytest-fixture-config-1.2.11 pytest-shutil-1.2.11 pytest-virtualenv-1.2.11 six-1.11.0 virtualenv-15.1.0
(__venv_reproducer__) $ PYTHONPATH=$(pwd) python -m pytest -k test_unicode_filename_in_sdist -v --tb=line
...
setuptools/tests/test_easy_install.py::TestEasyInstallTest::test_unicode_filename_in_sdist FAILED
...
<frozen importlib._bootstrap>:968: RecursionError: maximum recursion depth exceeded
...
Our RPM packages are installed using python setup.py install
, not trough wheels, that's why we are hitting this problem.
I believe this is a pytest bug.
_pytest.config._mark_plugins_for_rewrite
should not register setup
when it sees setup.py
_pytest.assertion.rewrite._should_rewrite
should probably match things that startswith mark + '.'
, not just mark
(although I'm not very sure here)Could you please help here @nicoddemus? This comes from https://github.com/pytest-dev/pytest/commit/6711b1d6abf3adc2356e3b58c0b0f664bbbb77d2
Hi @hroncok, thanks for the investigation!
Indeed your assessment seems correct. Created pytest-dev/pytest#2939 to track it, I will try to get to it soon.
pytest-dev/pytest#2939 has been fixed and will be published in the next pytest release. Should we close this given that there's nothing for setuptools
to do about it?
Fedora package has been "fixed" by using PYTHONDONTWRITEBYTECODE=1
. I agree we can close this (I cannot close).
Kudos to everyone involved in the investigation of this issue!
Trying to update the package in Fedora from 36.2.0 to 36.5.0 produces some test failures with the message:
RuntimeError: maximum recursion depth exceeded while calling a Python object
Attaching two different build logs where tests are invoked under python2 and under python3 respectively.
py2build.log
py3build.log