search

pypa / flit

Simplified packaging of Python modules
https://flit.pypa.io/
BSD 3-Clause "New" or "Revised" License
2.17k stars 137 forks source link

Improve isolation of tests for `flit install` #609

Open kloczek opened 2 years ago

kloczek commented 2 years ago

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

Here is pytest output:

```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-flit-3.8.0-6.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-flit-3.8.0-6.fc35.x86_64/usr/lib/python3.8/site-packages + /usr/bin/pytest -ra =========================================================================== test session starts ============================================================================ platform linux -- Python 3.8.15, pytest-7.1.3, pluggy-1.0.0 rootdir: /home/tkloczko/rpmbuild/BUILD/flit-3.8.0 collected 205 items flit_core/flit_core/tests/test_build_thyself.py ... [ 1%] flit_core/flit_core/tests/test_buildapi.py ......... [ 5%] flit_core/flit_core/tests/test_common.py .................... [ 15%] flit_core/flit_core/tests/test_config.py ............................................. [ 37%] flit_core/flit_core/tests/test_sdist.py ...... [ 40%] flit_core/flit_core/tests/test_versionno.py . [ 40%] flit_core/flit_core/tests/test_wheel.py ..... [ 43%] tests/test_build.py ..... [ 45%] tests/test_command.py .. [ 46%] tests/test_config.py . [ 47%] tests/test_find_python_executable.py ....... [ 50%] tests/test_init.py .................. [ 59%] tests/test_install.py ...F..F..........F.F........ [ 73%] tests/test_sdist.py ............ [ 79%] tests/test_tomlify.py . [ 79%] tests/test_upload.py ....... [ 82%] tests/test_validate.py ................ [ 90%] tests/test_vcs.py . [ 91%] tests/test_wheel.py .................. [100%] ================================================================================= FAILURES ================================================================================= ____________________________________________________________________ InstallTests.test_install_data_dir ____________________________________________________________________ self = def test_install_data_dir(self): > Installer.from_ini_path( core_samples_dir / 'with_data_dir' / 'pyproject.toml', ).install_directly() tests/test_install.py:296: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ flit/install.py:310: in install_directly self.install_requirements() flit/install.py:263: in install_requirements check_call(cmd) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ popenargs = (['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', ...],), kwargs = {}, retcode = 1 cmd = ['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', ...] def check_call(*popenargs, **kwargs): """Run command with arguments. Wait for command to complete. If the exit code was zero then return, otherwise raise CalledProcessError. The CalledProcessError object will have the return code in the returncode attribute. The arguments are the same as for the call function. Example: check_call(["ls", "-l"]) """ retcode = call(*popenargs, **kwargs) if retcode: cmd = kwargs.get("args") if cmd is None: cmd = popenargs[0] > raise CalledProcessError(retcode, cmd) E subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', '/tmp/tmp7kwihz9srequirements.txt']' returned non-zero exit status 1. /usr/lib64/python3.8/subprocess.py:364: CalledProcessError --------------------------------------------------------------------------- Captured stdout call --------------------------------------------------------------------------- Requirement already satisfied: requests>=2.18 in /usr/lib/python3.8/site-packages (from -r /tmp/tmp7kwihz9srequirements.txt (line 1)) (2.28.1) Requirement already satisfied: docutils in /usr/lib/python3.8/site-packages (from -r /tmp/tmp7kwihz9srequirements.txt (line 2)) (0.18.1) Requirement already satisfied: idna>=2.5 in /usr/lib/python3.8/site-packages (from requests>=2.18->-r /tmp/tmp7kwihz9srequirements.txt (line 1)) (3.4) Requirement already satisfied: charset-normalizer>=2 in /usr/lib/python3.8/site-packages (from requests>=2.18->-r /tmp/tmp7kwihz9srequirements.txt (line 1)) (3.0.0) Requirement already satisfied: urllib3>=1.21.1 in /usr/lib/python3.8/site-packages (from requests>=2.18->-r /tmp/tmp7kwihz9srequirements.txt (line 1)) (1.26.12) --------------------------------------------------------------------------- Captured stderr call --------------------------------------------------------------------------- ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/home/tkloczko/.local/lib/python3.8/site-packages' Check the permissions. _________________________________________________________________ InstallTests.test_install_module_pep621 __________________________________________________________________ self = def test_install_module_pep621(self): > Installer.from_ini_path( core_samples_dir / 'pep621_nodynamic' / 'pyproject.toml', ).install_directly() tests/test_install.py:60: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ flit/install.py:310: in install_directly self.install_requirements() flit/install.py:263: in install_requirements check_call(cmd) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ popenargs = (['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', ...],), kwargs = {}, retcode = 1 cmd = ['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', ...] def check_call(*popenargs, **kwargs): """Run command with arguments. Wait for command to complete. If the exit code was zero then return, otherwise raise CalledProcessError. The CalledProcessError object will have the return code in the returncode attribute. The arguments are the same as for the call function. Example: check_call(["ls", "-l"]) """ retcode = call(*popenargs, **kwargs) if retcode: cmd = kwargs.get("args") if cmd is None: cmd = popenargs[0] > raise CalledProcessError(retcode, cmd) E subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', '/tmp/tmpfoibwf0erequirements.txt']' returned non-zero exit status 1. /usr/lib64/python3.8/subprocess.py:364: CalledProcessError --------------------------------------------------------------------------- Captured stdout call --------------------------------------------------------------------------- Requirement already satisfied: requests>=2.18 in /usr/lib/python3.8/site-packages (from -r /tmp/tmpfoibwf0erequirements.txt (line 1)) (2.28.1) Requirement already satisfied: docutils in /usr/lib/python3.8/site-packages (from -r /tmp/tmpfoibwf0erequirements.txt (line 2)) (0.18.1) Requirement already satisfied: idna>=2.5 in /usr/lib/python3.8/site-packages (from requests>=2.18->-r /tmp/tmpfoibwf0erequirements.txt (line 1)) (3.4) Requirement already satisfied: urllib3>=1.21.1 in /usr/lib/python3.8/site-packages (from requests>=2.18->-r /tmp/tmpfoibwf0erequirements.txt (line 1)) (1.26.12) Requirement already satisfied: charset-normalizer>=2 in /usr/lib/python3.8/site-packages (from requests>=2.18->-r /tmp/tmpfoibwf0erequirements.txt (line 1)) (3.0.0) --------------------------------------------------------------------------- Captured stderr call --------------------------------------------------------------------------- ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/home/tkloczko/.local/lib/python3.8/site-packages' Check the permissions. ---------------------------------------------------------------------------- Captured log call ----------------------------------------------------------------------------- WARNING flit_core.versionno:versionno.py:124 Version number normalised: '0.03' -> '0.3' (see PEP 440) ____________________________________________________________________ InstallTests.test_symlink_data_dir ____________________________________________________________________ self = def test_symlink_data_dir(self): if os.name == 'nt': raise SkipTest("symlink") > Installer.from_ini_path( core_samples_dir / 'with_data_dir' / 'pyproject.toml', symlink=True ).install_directly() tests/test_install.py:305: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ flit/install.py:310: in install_directly self.install_requirements() flit/install.py:263: in install_requirements check_call(cmd) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ popenargs = (['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', ...],), kwargs = {}, retcode = 1 cmd = ['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', ...] def check_call(*popenargs, **kwargs): """Run command with arguments. Wait for command to complete. If the exit code was zero then return, otherwise raise CalledProcessError. The CalledProcessError object will have the return code in the returncode attribute. The arguments are the same as for the call function. Example: check_call(["ls", "-l"]) """ retcode = call(*popenargs, **kwargs) if retcode: cmd = kwargs.get("args") if cmd is None: cmd = popenargs[0] > raise CalledProcessError(retcode, cmd) E subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', '/tmp/tmp1nztui2qrequirements.txt']' returned non-zero exit status 1. /usr/lib64/python3.8/subprocess.py:364: CalledProcessError --------------------------------------------------------------------------- Captured stdout call --------------------------------------------------------------------------- Requirement already satisfied: requests>=2.18 in /usr/lib/python3.8/site-packages (from -r /tmp/tmp1nztui2qrequirements.txt (line 1)) (2.28.1) Requirement already satisfied: docutils in /usr/lib/python3.8/site-packages (from -r /tmp/tmp1nztui2qrequirements.txt (line 2)) (0.18.1) Requirement already satisfied: idna>=2.5 in /usr/lib/python3.8/site-packages (from requests>=2.18->-r /tmp/tmp1nztui2qrequirements.txt (line 1)) (3.4) Requirement already satisfied: urllib3>=1.21.1 in /usr/lib/python3.8/site-packages (from requests>=2.18->-r /tmp/tmp1nztui2qrequirements.txt (line 1)) (1.26.12) Requirement already satisfied: charset-normalizer>=2 in /usr/lib/python3.8/site-packages (from requests>=2.18->-r /tmp/tmp1nztui2qrequirements.txt (line 1)) (3.0.0) --------------------------------------------------------------------------- Captured stderr call --------------------------------------------------------------------------- ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/home/tkloczko/.local/lib/python3.8/site-packages' Check the permissions. _________________________________________________________________ InstallTests.test_symlink_module_pep621 __________________________________________________________________ self = def test_symlink_module_pep621(self): if os.name == 'nt': raise SkipTest("symlink") > Installer.from_ini_path( core_samples_dir / 'pep621_nodynamic' / 'pyproject.toml', symlink=True ).install_directly() tests/test_install.py:160: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ flit/install.py:310: in install_directly self.install_requirements() flit/install.py:263: in install_requirements check_call(cmd) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ popenargs = (['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', ...],), kwargs = {}, retcode = 1 cmd = ['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', ...] def check_call(*popenargs, **kwargs): """Run command with arguments. Wait for command to complete. If the exit code was zero then return, otherwise raise CalledProcessError. The CalledProcessError object will have the return code in the returncode attribute. The arguments are the same as for the call function. Example: check_call(["ls", "-l"]) """ retcode = call(*popenargs, **kwargs) if retcode: cmd = kwargs.get("args") if cmd is None: cmd = popenargs[0] > raise CalledProcessError(retcode, cmd) E subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', '/tmp/tmpkabqdk4prequirements.txt']' returned non-zero exit status 1. /usr/lib64/python3.8/subprocess.py:364: CalledProcessError --------------------------------------------------------------------------- Captured stdout call --------------------------------------------------------------------------- Requirement already satisfied: requests>=2.18 in /usr/lib/python3.8/site-packages (from -r /tmp/tmpkabqdk4prequirements.txt (line 1)) (2.28.1) Requirement already satisfied: docutils in /usr/lib/python3.8/site-packages (from -r /tmp/tmpkabqdk4prequirements.txt (line 2)) (0.18.1) Requirement already satisfied: idna>=2.5 in /usr/lib/python3.8/site-packages (from requests>=2.18->-r /tmp/tmpkabqdk4prequirements.txt (line 1)) (3.4) Requirement already satisfied: charset-normalizer>=2 in /usr/lib/python3.8/site-packages (from requests>=2.18->-r /tmp/tmpkabqdk4prequirements.txt (line 1)) (3.0.0) Requirement already satisfied: urllib3>=1.21.1 in /usr/lib/python3.8/site-packages (from requests>=2.18->-r /tmp/tmpkabqdk4prequirements.txt (line 1)) (1.26.12) --------------------------------------------------------------------------- Captured stderr call --------------------------------------------------------------------------- ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/home/tkloczko/.local/lib/python3.8/site-packages' Check the permissions. ---------------------------------------------------------------------------- Captured log call ----------------------------------------------------------------------------- WARNING flit_core.versionno:versionno.py:124 Version number normalised: '0.03' -> '0.3' (see PEP 440) ========================================================================= short test summary info ========================================================================== FAILED tests/test_install.py::InstallTests::test_install_data_dir - subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', ... FAILED tests/test_install.py::InstallTests::test_install_module_pep621 - subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'pip', 'install', '--user', '... FAILED tests/test_install.py::InstallTests::test_symlink_data_dir - subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'pip', 'install', '--user', '-r', ... FAILED tests/test_install.py::InstallTests::test_symlink_module_pep621 - subprocess.CalledProcessError: Command '['/usr/bin/python3', '-m', 'pip', 'install', '--user', '... ====================================================================== 4 failed, 201 passed in 6.96s ======================================================================= ```

Here is list of installed modules in build env

```console Package Version ----------------------------- ----------------- alabaster 0.7.12 appdirs 1.4.4 attrs 22.1.0 Babel 2.11.0 Brlapi 0.8.3 build 0.9.0 charset-normalizer 3.0.0 codespell 2.2.1 contourpy 1.0.5 cssselect 1.1.0 cycler 0.11.0 distro 1.7.0 dnspython 2.2.1 docutils 0.18.1 extras 1.0.0 fixtures 4.0.0 fonttools 4.38.0 gpg 1.17.1-unknown idna 3.4 imagesize 1.4.1 importlib-metadata 5.0.0 iniconfig 1.1.1 Jinja2 3.1.1 kiwisolver 1.4.4 libcomps 0.1.19 louis 3.23.0 lxml 4.9.1 MarkupSafe 2.1.1 matplotlib 3.6.0 numpy 1.23.1 olefile 0.46 packaging 21.3 pbr 5.9.0 pep517 0.13.0 Pillow 9.2.0 pip 22.2.2 pluggy 1.0.0 py 1.11.0 Pygments 2.13.0 PyGObject 3.42.2 pyparsing 3.0.9 pytest 7.1.3 python-dateutil 2.8.2 pytz 2022.4 requests 2.28.1 requests_download 0.1.2 responses 0.22.0 rpm 4.17.0 scour 0.38.2 six 1.16.0 snowballstemmer 2.2.0 Sphinx 5.3.0 sphinx-rtd-theme 1.1.0 sphinxcontrib-applehelp 1.0.2.dev20220730 sphinxcontrib-devhelp 1.0.2.dev20220730 sphinxcontrib-htmlhelp 2.0.0 sphinxcontrib-jsmath 1.0.1.dev20220730 sphinxcontrib-qthelp 1.0.3.dev20220730 sphinxcontrib-serializinghtml 1.1.5 testpath 0.6.0 testtools 2.5.0 toml 0.10.2 tomli 2.0.1 tomli_w 1.0.0 types-toml 0.10.8 urllib3 1.26.12 wheel 0.37.1 zipp 3.9.0 ```
gotmax23 commented 2 years ago

The tests are all failing with

ERROR: Could not install packages due to an OSError: [Errno 13] Permission denied: '/home/tkloczko/.local/lib/python3.8/site-packages'
Check the permissions.

This seems to be a problem with your system and not a problem with the tests.

kloczek commented 2 years ago

Yep .. and question ios why test wiote is truing to install something?

pradyunsg commented 2 years ago

The test filename is:

tests/test_install.py

That should provide a hint for why its trying to install something. Flit has a flit install command, which is what is being tested in those tests.

pradyunsg commented 2 years ago

Closing this since this is an environment configuration issue.

takluyver commented 2 years ago

Actually, I think there is a genuine issue with the tests here that we hadn't spotted.

The tests do some monkeypatching to redirect installation to temporary directories:

https://github.com/pypa/flit/blob/f5704ea31f0fcc579b8518ea85d641651cba4f71/tests/test_install.py#L25-L30

But this doesn't affect the _auto_user method, which looks to see if it should do a user install or not:

https://github.com/pypa/flit/blob/f5704ea31f0fcc579b8518ea85d641651cba4f71/flit/install.py#L152-L154

And of course the monkeypatching also won't affect pip when the install code runs that as a subprocess.

So I'm going to reopen this :wink: . I'm not sure yet exactly what's the best way to properly isolate those tests - maybe we have to set up a venv for each test, even if that's slower - but I think they do need some work.