search

pypa / pip

The Python package installer
https://pip.pypa.io/
MIT License
9.36k stars 2.98k forks source link

Test suite broken on PyPy3.10 (7.3.15) on Gentoo — pip seems to install straight into venv instead of tempdirs #12511

Open mgorny opened 4 months ago

mgorny commented 4 months ago

Description

When running the pip's test suite with PyPy3.10 7.3.15 on Gentoo (and older versions), lots of tests fail. Upon investigating, I've discovered that pip operates on the test venv rather than the tempdirs, like it does for CPython.

For example:

$ nox -e test-pypy3 -- -x
nox > Running session test-pypy3
nox > Creating virtual environment (virtualenv) using pypy3 in .nox/test-pypy3
nox > python tools/protected_pip.py wheel -w tests/data/common_wheels -r tests/requirements-common_wheels.txt
nox > python -m pip install setuptools
nox > python setup.py sdist --formats=zip --dist-dir /tmp/pip-24.0/.nox/test-pypy3/sdist
nox > python tools/protected_pip.py install /tmp/pip-24.0/.nox/test-pypy3/sdist/pip-24.0.zip
nox > python tools/protected_pip.py install -r tests/requirements.txt
nox > pytest -x
========================================================= test session starts =========================================================
platform linux -- Python 3.10.13[pypy-7.3.15-final], pytest-8.0.0, pluggy-1.4.0
rootdir: /tmp/pip-24.0
configfile: setup.cfg
plugins: xdist-3.5.0, rerunfailures-13.0, cov-4.1.0
collected 2545 items                                                                                                                  

tests/functional/test_bad_url.py .                                                                                              [  0%]
tests/functional/test_broken_stdout.py ...                                                                                      [  0%]
tests/functional/test_build_env.py ....F

============================================================== FAILURES ===============================================================
______________________________________________________ test_build_env_isolation _______________________________________________________

script = <tests.lib.PipTestEnvironment object at 0x00007fe02d042d40>

    @pytest.mark.usefixtures("enable_user_site")
    def test_build_env_isolation(script: PipTestEnvironment) -> None:
        # Create dummy `pkg` wheel.
        pkg_whl = create_basic_wheel_for_package(script, "pkg", "1.0")

        # Install it to site packages.
        script.pip_install_local(pkg_whl)

        # And a copy in the user site.
>       script.pip_install_local("--ignore-installed", "--user", pkg_whl)

tests/functional/test_build_env.py:241: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/lib/__init__.py:745: in pip_install_local
    return self.pip(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <tests.lib.PipTestEnvironment object at 0x00007fe02d042d40>
cwd = PosixPath('/tmp/pytest-of-mgorny/pytest-0/test_build_env_isolation0/workspace/scratch'), allow_stderr_error = False
allow_stderr_warning = False, allow_error = False, args = ('python', '-m', 'pip', 'install', '--no-index', '--find-links', ...)
kw = {'expect_stderr': True}, expect_error = None

    def run(
        self,
        *args: str,
        cwd: Optional[StrPath] = None,
        allow_stderr_error: Optional[bool] = None,
        allow_stderr_warning: Optional[bool] = None,
        allow_error: bool = False,
        **kw: Any,
    ) -> TestPipResult:
        """
        :param allow_stderr_error: whether a logged error is allowed in
            stderr.  Passing True for this argument implies
            `allow_stderr_warning` since warnings are weaker than errors.
        :param allow_stderr_warning: whether a logged warning (or
            deprecation message) is allowed in stderr.
        :param allow_error: if True (default is False) does not raise
            exception when the command exit value is non-zero.  Implies
            expect_error, but in contrast to expect_error will not assert
            that the exit value is zero.
        :param expect_error: if False (the default), asserts that the command
            exits with 0.  Otherwise, asserts that the command exits with a
            non-zero exit code.  Passing True also implies allow_stderr_error
            and allow_stderr_warning.
        :param expect_stderr: whether to allow warnings in stderr (equivalent
            to `allow_stderr_warning`).  This argument is an abbreviated
            version of `allow_stderr_warning` and is also kept for backwards
            compatibility.
        """
        if self.verbose:
            print(f">> running {args} {kw}")

        cwd = cwd or self.cwd
        if sys.platform == "win32":
            # Partial fix for ScriptTest.run using `shell=True` on Windows.
            args = tuple(re.sub("([&|<>^])", r"^\1", str(a)) for a in args)

        if allow_error:
            kw["expect_error"] = True

        # Propagate default values.
        expect_error = kw.get("expect_error")
        if expect_error:
            # Then default to allowing logged errors.
            if allow_stderr_error is not None and not allow_stderr_error:
                raise RuntimeError(
                    "cannot pass allow_stderr_error=False with expect_error=True"
                )
            allow_stderr_error = True

        elif kw.get("expect_stderr"):
            # Then default to allowing logged warnings.
            if allow_stderr_warning is not None and not allow_stderr_warning:
                raise RuntimeError(
                    "cannot pass allow_stderr_warning=False with expect_stderr=True"
                )
            allow_stderr_warning = True

        if allow_stderr_error:
            if allow_stderr_warning is not None and not allow_stderr_warning:
                raise RuntimeError(
                    "cannot pass allow_stderr_warning=False with "
                    "allow_stderr_error=True"
                )

        # Default values if not set.
        if allow_stderr_error is None:
            allow_stderr_error = False
        if allow_stderr_warning is None:
            allow_stderr_warning = allow_stderr_error

        # Pass expect_stderr=True to allow any stderr.  We do this because
        # we do our checking of stderr further on in check_stderr().
        kw["expect_stderr"] = True
        # Ignore linter check
        # B026 Star-arg unpacking after a keyword argument is strongly discouraged
>       result = super().run(cwd=cwd, *args, **kw)  # noqa: B026
E       AssertionError: Script returned code: 1

tests/lib/__init__.py:706: AssertionError
-------------------------------------------------------- Captured stdout call ---------------------------------------------------------
Script result: python -m pip install --no-index --find-links file:///tmp/pip-24.0/tests/data/packages --ignore-installed --user /tmp/pytest-of-mgorny/pytest-0/test_build_env_isolation0/workspace/scratch/pkg-1.0-py2.py3-none-any.whl
  return code: 1
-- stderr: --------------------
ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv.

========================================================== warnings summary ===========================================================
.nox/test-pypy3/lib/pypy3.10/site-packages/pip/_internal/metadata/pkg_resources.py:8
  /tmp/pip-24.0/.nox/test-pypy3/lib/pypy3.10/site-packages/pip/_internal/metadata/pkg_resources.py:8: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    from pip._vendor import pkg_resources

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
======================================================= short test summary info =======================================================
FAILED tests/functional/test_build_env.py::test_build_env_isolation - AssertionError: Script returned code: 1
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=============================================== 1 failed, 8 passed, 1 warning in 40.88s ===============================================
nox > Command pytest -x failed with exit code 1
nox > Session test-pypy3 failed.
$ ls .nox/test-pypy3/lib/pypy3.10/site-packages/
bar                             freezegun                     pluggy                               six-1.16.0.dist-info
bar-3.0.dist-info               freezegun-1.4.0.dist-info     pluggy-1.4.0.dist-info               six.py
coverage                        iniconfig                     __pycache__                          tomli
coverage-7.4.1.dist-info        iniconfig-2.0.0.dist-info     py.py                                tomli-2.0.1.dist-info
cryptography                    installer                     _pytest                              tomli_w
cryptography-42.0.2.dist-info   installer-0.7.0.dist-info     pytest                               tomli_w-1.0.0.dist-info
dateutil                        markupsafe                    pytest-8.0.0.dist-info               virtualenv
distlib                         MarkupSafe-2.1.5.dist-info    pytest_cov                           virtualenv-20.25.0.dist-info
distlib-0.3.8.dist-info         other                         pytest_cov-4.1.0.dist-info           _virtualenv.pth
_distutils_hack                 other-0.5.dist-info           pytest-cov.pth                       _virtualenv.py
distutils-precedence.pth        packaging                     pytest_rerunfailures-13.0.dist-info  werkzeug
exceptiongroup                  packaging-23.2.dist-info      pytest_rerunfailures.py              werkzeug-3.0.1.dist-info
exceptiongroup-1.2.0.dist-info  pip                           pytest_xdist-3.5.0.dist-info         wheel
execnet                         pip-24.0.dist-info            python_dateutil-2.8.2.dist-info      wheel-0.42.0.dist-info
execnet-2.0.2.dist-info         pkg                           scripttest-1.3.dist-info             wheel-0.42.0.virtualenv
filelock                        pkg-1.0.dist-info             scripttest.py                        xdist
filelock-3.13.1.dist-info       pkg_resources                 setuptools
foo                             platformdirs                  setuptools-69.0.3.dist-info
foo-2.0.dist-info               platformdirs-4.2.0.dist-info  setuptools-69.0.3.virtualenv

Note bar and foo packages. If I let the test suite run longer, soon enough pip actually gets uninstalled from there.

Now, I don't know why it happens on our install and not on CI. We aren't patching PyPy in any relevant way. The install is reasonably straightforward. It involves, roughly:

/usr/bin/pypy3 -> pypy3.10
/usr/bin/pypy3.10 -> pypy3.10-c-7.3.15
/usr/bin/pypy3.10-c-7.3.15
/usr/lib/pypy3.10/**
/usr/include/pypy3.10/**

I'd be glad to debug it if you give me some hints.

Expected behavior

Most of the tests passing, and not wreaking havoc in venv.

pip version

24.0

Python version

PyPy3.10 7.3.15

OS

Gentoo Linux amd64

How to Reproduce

  1. Get a Gentoo install ;-).
  2. emerge pypy3 nox
  3. nox -e test-pypy3

Output

No response

Code of Conduct

uranusjr commented 4 months ago

Does this happen if you use a PyPy installation not provided by the system packaging manager? I highly suspect this is due to Gentoo patches to the Python stdlib that do not account for this use case.

mgorny commented 4 months ago

Yes, it does. I've even removed the system install, in case it affected it somehow.

I've done:

wget https://downloads.python.org/pypy/pypy3.10-v7.3.15-linux64.tar.bz2
tar -xf pypy3.10-v7.3.15-linux64.tar.bz2
export PATH=${PWD}/pypy3.10-v7.3.15-linux64/bin:$PATH
cd .../pip
nox -e test-pypy3 -- -x
pfmoore commented 4 months ago

Reproduced on Ubuntu:

docker run --rm -it ubuntu

apt-get -y update
apt-get install -y wget git nox
useradd -ms /bin/bash test
su - test
wget https://downloads.python.org/pypy/pypy3.10-v7.3.15-linux64.tar.bz2
tar -xf pypy3.10-v7.3.15-linux64.tar.bz2
export PATH=${PWD}/pypy3.10-v7.3.15-linux64/bin:$PATH
git clone https://github.com/pypa/pip
cd pip
virtualenv .venv
. .venv/bin/activate
pip install nox
nox -s test-pypy3 -- -x