Open jaraco opened 4 years ago
Hmm... I'm also on MacOS and everything passes. Are you using a newer virtualenv (virtualenv >= 20.0)? We know that pip's tests aren't compatible with that but idk what the failure modes are (#7698).
I realized I have tox-venv installed, meaning that tox envs get installed using venv, so I disabled that by creating a separate environment for running tox, but tests are still failing:
I can see in the output that virtualenv==16.7.10
is used (both in this run an the previous one).
Running the tests with the --use-venv
option, 11 of the previous test failures are skipped:
pip master $ tox -- --use-venv --lf
GLOB sdist-make: /Users/jaraco/code/public/pypa/pip/setup.py
python inst-nodeps: /Users/jaraco/code/public/pypa/pip/.tox/.tmp/package/1/pip-20.1.dev0.zip
python installed: apipkg==1.5,atomicwrites==1.3.0,attrs==19.3.0,cffi==1.14.0,coverage==5.0.3,cryptography==2.8,csv23==0.1.6,execnet==1.7.1,freezegun==0.3.15,mock==4.0.1,more-itertools==8.2.0,pip==20.1.dev0,pluggy==0.13.1,pretend==1.0.9,py==1.8.1,pycparser==2.20,pytest==3.8.2,pytest-cov==2.8.1,pytest-forked==1.1.3,pytest-rerunfailures==6.0,pytest-timeout==1.3.4,pytest-xdist==1.27.0,python-dateutil==2.8.1,PyYAML==5.3,scripttest==1.3,setuptools==41.2.0,six==1.14.0,virtualenv==16.7.10,Werkzeug==0.16.0,wheel==0.34.2
python run-test-pre: PYTHONHASHSEED='2897048640'
python run-test-pre: commands[0] | python -c 'import shutil, sys; shutil.rmtree(sys.argv[1], ignore_errors=True)' /Users/jaraco/code/public/pypa/pip/tests/data/common_wheels
python run-test-pre: commands[1] | python /Users/jaraco/code/public/pypa/pip/tools/tox_pip.py wheel -w /Users/jaraco/code/public/pypa/pip/tests/data/common_wheels -r /Users/jaraco/code/public/pypa/pip/tools/requirements/tests-common_wheels.txt
Collecting setuptools>=40.8.0
Using cached setuptools-45.2.0-py3-none-any.whl (584 kB)
Saved ./tests/data/common_wheels/setuptools-45.2.0-py3-none-any.whl
Collecting wheel
Using cached wheel-0.34.2-py2.py3-none-any.whl (26 kB)
Saved ./tests/data/common_wheels/wheel-0.34.2-py2.py3-none-any.whl
Skipping setuptools, due to already being wheel.
Skipping wheel, due to already being wheel.
python run-test: commands[0] | pytest --timeout 300 --use-venv --lf
========================================================================== test session starts ===========================================================================
platform darwin -- Python 3.8.1, pytest-3.8.2, py-1.8.1, pluggy-0.13.1
rootdir: /Users/jaraco/code/public/pypa/pip, inifile: setup.cfg
plugins: timeout-1.3.4, rerunfailures-6.0, forked-1.1.3, cov-2.8.1, xdist-1.27.0
timeout: 300.0s
timeout method: signal
timeout func_only: False
collected 1833 items / 1817 deselected
run-last-failure: rerun previous 16 failures
tests/functional/test_freeze.py F [ 6%]
tests/functional/test_install.py Fs [ 18%]
tests/functional/test_install_reqs.py F [ 25%]
tests/functional/test_install_user.py ssssss [ 62%]
tests/functional/test_install_wheel.py s [ 68%]
tests/functional/test_uninstall.py FF [ 81%]
tests/functional/test_uninstall_user.py sss [100%]
================================================================================ FAILURES ================================================================================
======================================================================== short test summary info =========================================================================
FAIL tests/functional/test_freeze.py::test_freeze_svn
FAIL tests/functional/test_install.py::test_basic_install_editable_from_svn
FAIL tests/functional/test_install_reqs.py::test_multiple_requirements_files
FAIL tests/functional/test_uninstall.py::test_uninstall_editable_from_svn
FAIL tests/functional/test_uninstall.py::test_uninstall_from_reqs_file
SKIP [1] tests/functional/test_install.py:1680: Incompatible with test venv
SKIP [1] tests/functional/test_install_user.py:44: Incompatible with test venv
SKIP [1] tests/functional/test_install_user.py:60: Incompatible with test venv
SKIP [1] tests/functional/test_install_user.py:99: Incompatible with test venv
SKIP [1] tests/functional/test_install_user.py:123: Incompatible with test venv
SKIP [1] tests/functional/test_install_user.py:154: Incompatible with test venv
SKIP [1] tests/functional/test_install_user.py:184: Incompatible with test venv
SKIP [1] tests/functional/test_install_wheel.py:272: Incompatible with test venv
SKIP [1] tests/functional/test_uninstall_user.py:15: Incompatible with test venv
SKIP [2] tests/functional/test_uninstall_user.py: Incompatible with test venv
========================================================= 5 failed, 11 skipped, 1817 deselected in 5.71 seconds ==========================================================
The ones that remain failing are decorated with @need_svn
but fail with No such file or directory: 'svnadmin'
It seems that I have svn
installed but not svnadmin
, but the pip test suite assumes that if the former is installed that the latter is available. I'll open a separate issue for that.
After addressing the svn issue, I double-checked that virtualenv 16.7.10 is being invoked when the tests are failing:
python run-test: commands[0] | pytest --timeout 300 -x --lf
========================================================================== test session starts ===========================================================================
platform darwin -- Python 3.8.1, pytest-3.8.2, py-1.8.1, pluggy-0.13.1
rootdir: /Users/jaraco/code/public/pypa/pip, inifile: setup.cfg
plugins: timeout-1.3.4, rerunfailures-6.0, forked-1.1.3, cov-2.8.1, xdist-1.27.0
timeout: 300.0s
timeout method: signal
timeout func_only: False
collected 1833 items / 1823 deselected
run-last-failure: rerun previous 10 failures
tests/functional/test_install.py
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB set_trace (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /Users/jaraco/code/public/pypa/pip/tests/lib/venv.py(67)_create()
-> _virtualenv.create_environment(
(Pdb) _virtualenv.__file__
'/Users/jaraco/code/public/pypa/pip/.tox/python/lib/python3.8/site-packages/virtualenv.py'
(Pdb) import pkg_resources
(Pdb) pkg_resources.get_distribution('virtualenv').version
'16.7.10'
I then inspected first failing result:
pip bugfix/7823-need-svnadmin $ tox -- -x --lf --pdb
GLOB sdist-make: /Users/jaraco/code/public/pypa/pip/setup.py
python inst-nodeps: /Users/jaraco/code/public/pypa/pip/.tox/.tmp/package/1/pip-20.1.dev0.zip
python installed: apipkg==1.5,atomicwrites==1.3.0,attrs==19.3.0,cffi==1.14.0,coverage==5.0.3,cryptography==2.8,csv23==0.1.6,execnet==1.7.1,freezegun==0.3.15,mock==4.0.1,more-itertools==8.2.0,pip==20.1.dev0,pluggy==0.13.1,pretend==1.0.9,py==1.8.1,pycparser==2.20,pytest==3.8.2,pytest-cov==2.8.1,pytest-forked==1.1.3,pytest-rerunfailures==6.0,pytest-timeout==1.3.4,pytest-xdist==1.27.0,python-dateutil==2.8.1,PyYAML==5.3,scripttest==1.3,setuptools==41.2.0,six==1.14.0,virtualenv==16.7.10,Werkzeug==0.16.0,wheel==0.34.2
python run-test-pre: PYTHONHASHSEED='3501743582'
python run-test-pre: commands[0] | python -c 'import shutil, sys; shutil.rmtree(sys.argv[1], ignore_errors=True)' /Users/jaraco/code/public/pypa/pip/tests/data/common_wheels
python run-test-pre: commands[1] | python /Users/jaraco/code/public/pypa/pip/tools/tox_pip.py wheel -w /Users/jaraco/code/public/pypa/pip/tests/data/common_wheels -r /Users/jaraco/code/public/pypa/pip/tools/requirements/tests-common_wheels.txt
Collecting setuptools>=40.8.0
Using cached setuptools-45.2.0-py3-none-any.whl (584 kB)
Saved ./tests/data/common_wheels/setuptools-45.2.0-py3-none-any.whl
Collecting wheel
Using cached wheel-0.34.2-py2.py3-none-any.whl (26 kB)
Saved ./tests/data/common_wheels/wheel-0.34.2-py2.py3-none-any.whl
Skipping setuptools, due to already being wheel.
Skipping wheel, due to already being wheel.
python run-test: commands[0] | pytest --timeout 300 -x --lf --pdb
========================================================================== test session starts ===========================================================================
platform darwin -- Python 3.8.1, pytest-3.8.2, py-1.8.1, pluggy-0.13.1
rootdir: /Users/jaraco/code/public/pypa/pip, inifile: setup.cfg
plugins: timeout-1.3.4, rerunfailures-6.0, forked-1.1.3, cov-2.8.1, xdist-1.27.0
timeout: 300.0s
timeout method: signal
timeout func_only: False
collected 1833 items / 1823 deselected
run-last-failure: rerun previous 10 failures
tests/functional/test_install.py F
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> captured stderr >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
warning: no files found matching 'docs/docutils.conf'
warning: no previously-included files found matching '.coveragerc'
warning: no previously-included files found matching '.mailmap'
warning: no previously-included files found matching '.appveyor.yml'
warning: no previously-included files found matching '.travis.yml'
warning: no previously-included files found matching '.readthedocs.yml'
warning: no previously-included files found matching '.pre-commit-config.yaml'
warning: no previously-included files found matching 'tox.ini'
warning: no previously-included files found matching 'noxfile.py'
warning: no files found matching 'Makefile' under directory 'docs'
warning: no files found matching '*.rst' under directory 'docs'
warning: no files found matching '*.py' under directory 'docs'
warning: no files found matching '*.bat' under directory 'docs'
warning: no previously-included files found matching 'src/pip/_vendor/six'
warning: no previously-included files found matching 'src/pip/_vendor/six/moves'
warning: no previously-included files matching '*.pyi' found under directory 'src/pip/_vendor'
no previously-included directories found matching '.github'
no previously-included directories found matching '.azure-pipelines'
no previously-included directories found matching 'docs/build'
no previously-included directories found matching 'news'
no previously-included directories found matching 'tasks'
no previously-included directories found matching 'tests'
no previously-included directories found matching 'tools'
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
script = <tests.lib.PipTestEnvironment object at 0x10fa15fd0>
@pytest.mark.incompatible_with_test_venv
def test_user_config_accepted(script):
# user set in the config file is parsed as 0/1 instead of True/False.
# Check that this doesn't cause a problem.
config_file = script.scratch_path / 'pip.conf'
script.environ['PIP_CONFIG_FILE'] = str(config_file)
config_file.write_text("[install]\nuser = true")
result = script.pip_install_local('simplewheel')
assert "Successfully installed simplewheel" in result.stdout
relative_user = os.path.relpath(script.user_site_path, script.base_path)
> assert join(relative_user, 'simplewheel') in result.files_created
E AssertionError: assert 'venv/user/lib/python/site-packages/simplewheel' in {'venv/user/lib/python3.8': <FoundDir /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest...pytest-19/test_user_config_accepted0/workspace:venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info>, ...}
E + where 'venv/user/lib/python/site-packages/simplewheel' = join('venv/user/lib/python/site-packages', 'simplewheel')
E + and {'venv/user/lib/python3.8': <FoundDir /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest...pytest-19/test_user_config_accepted0/workspace:venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info>, ...} = <tests.lib.TestPipResult object at 0x10ec986d0>.files_created
tests/functional/test_install.py:1692: AssertionError
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /Users/jaraco/code/public/pypa/pip/tests/functional/test_install.py(1692)test_user_config_accepted()
-> assert join(relative_user, 'simplewheel') in result.files_created
(Pdb) result.files_created
{'venv/user/lib/python3.8': <FoundDir /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-19/test_user_config_accepted0/workspace:venv/user/lib/python3.8>, 'venv/user/lib/python3.8/site-packages': <FoundDir /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-19/test_user_config_accepted0/workspace:venv/user/lib/python3.8/site-packages>, 'venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info': <FoundDir /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-19/test_user_config_accepted0/workspace:venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info>, 'venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info/RECORD': <FoundFile /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-19/test_user_config_accepted0/workspace:venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info/RECORD>, 'venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info/metadata.json': <FoundFile /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-19/test_user_config_accepted0/workspace:venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info/metadata.json>, 'venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info/WHEEL': <FoundFile /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-19/test_user_config_accepted0/workspace:venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info/WHEEL>, 'venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info/DESCRIPTION.rst': <FoundFile /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-19/test_user_config_accepted0/workspace:venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info/DESCRIPTION.rst>, 'venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info/top_level.txt': <FoundFile /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-19/test_user_config_accepted0/workspace:venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info/top_level.txt>, 'venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info/INSTALLER': <FoundFile /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-19/test_user_config_accepted0/workspace:venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info/INSTALLER>, 'venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info/METADATA': <FoundFile /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-19/test_user_config_accepted0/workspace:venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info/METADATA>, 'venv/user/lib/python3.8/site-packages/simplewheel': <FoundDir /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-19/test_user_config_accepted0/workspace:venv/user/lib/python3.8/site-packages/simplewheel>, 'venv/user/lib/python3.8/site-packages/simplewheel/__init__.py': <FoundFile /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-19/test_user_config_accepted0/workspace:venv/user/lib/python3.8/site-packages/simplewheel/__init__.py>}
(Pdb) join(relative_user, 'simplewheel')
'venv/user/lib/python/site-packages/simplewheel'
(Pdb) !list(map(print, list(result.files_created)))
venv/user/lib/python3.8
venv/user/lib/python3.8/site-packages
venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info
venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info/RECORD
venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info/metadata.json
venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info/WHEEL
venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info/DESCRIPTION.rst
venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info/top_level.txt
venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info/INSTALLER
venv/user/lib/python3.8/site-packages/simplewheel-2.0-1.dist-info/METADATA
venv/user/lib/python3.8/site-packages/simplewheel
venv/user/lib/python3.8/site-packages/simplewheel/__init__.py
[None, None, None, None, None, None, None, None, None, None, None, None]
As you can see, the test is expecting venv/usr/lib/python
to be in the result files when what it finds is venv/usr/lib/python3.8
.
Here's a bit more detail about how that discrepancy comes about:
(Pdb) script.user_site_path
Path('/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-19/test_user_config_accepted0/workspace/venv/user/lib/python/site-packages')
(Pdb) import site
(Pdb) site.USER_SITE
'/Users/jaraco/Library/Python/3.8/lib/python/site-packages'
I suspect the pip test suite isn't accounting for this branch.
The commit in af9f744 works around the issue. Feel free to merge that in or address the underlying cause.
Is this still an active issue or is it resolved with above-mentioned commit?
I don't believe the aforementioned commit is in master (though it's not obvious to me from what I can see in GitHub). Even if merged into master, it's only a workaround and this ticket maybe should remain open to address the root cause.
Environment
Description
Running the test suite on a clean checkout (
tox
withTOXENV=python
), 16 tests fail consistently.Expected behavior
Test suite should pass on popular operating systems.
Output