Pytest seems to search and create files in site-packages directory when not prompted to

[hmaarrfk]

It seems that if pytest is run in a development directory, it can create files in site-packages/.../__pycache__ which can cause modules to continue getting "imported" even when they are uninstalled.

Many packages use import X as proof of an X package is installed and available.

If you:

1. Install an optional package
2. Run pytest
3. Uninstall the optional package

It can leave your environment in a broken state.

Your input on "why" this is happening would be appreciated.

I wrote out my explanation and reproducer https://github.com/conda-forge/cupy-feedstock/issues/188#issuecomment-1449217000

• [x] a detailed description of the bug or problem you are having
• [x] output of pip list from the virtual environment you are using
• [x] pytest and operating system versions
• [x] minimal example if possible

[RonnyPfannschmidt]

The assertion rewriter currently caches next to the plugins for pytest plugins

I vaguely recall that pip would clean up the pyc files

If the tooling doesn't do that, we might need to start storing them somewhere else

[hmaarrfk]

Conda will not delete any files it didn't explicitly install. It may be the difference between it and pip

[WarrenWeckesser]

I think I have just run into the same issue. I am not using conda. I install a package with pip, run the tests with pytest, and then use pip to uninstall the package. The package directory still exists in .../site-packages, containing only the .pyc files generated by pytest in each of the tests/__pycache__ subdirectories. Is there a way to avoid this problem? I expect pip uninstall mypackage to completely remove all the files in .../site-packages.

[sanderr]

I can confirm this also causes issues with pip. Below is a reproduction, first reported on the pip issue tracker.

The pip maintainers have stated that they stand by their decision to not remove these .pyc files (at least not without user confirmation). I think they make a good argument. If I understand correctly, they do clean up Python's native .pyc files, but any other files are not considered owned by the package (unless specified in the package metadata), therefore pip does not remove them. In the reproduction below you'll see that uninstalling a pytest plugin removes the native .pyc files but not the ones created by pytest (with pytest in the name).

If the tooling doesn't do that, we might need to start storing them somewhere else

@RonnyPfannschmidt I would like to request this. I do not think the site-packages directory is intended for this.

Reproduction

(copied from comment on pip issue linked above)

To reproduce: install a pytest plugin, run pytest once, then uninstall the plugin again. In practice we have had issues with this specifically when moving from a normal install to an editable install, but the example reproduction doesn't go that far. Note how after running pytest once, a pyc file with pytest in the name appears in the site-packages' pycache dir. Uninstalling the plugin removes the other pyc files but not that one.

relevant versions:

Python 3.11.5
pip        23.2.1
pytest     7.4.2
setuptools 68.2.2

sander@bedevere:~$ mktmpenv --quiet  # uses virtualenvwrapper but you can just as well use the venv module directly
virtualenvwrapper.user_scripts creating /home/sander/.virtualenvs/tmp-2d931a2993df6fc/bin/predeactivate
virtualenvwrapper.user_scripts creating /home/sander/.virtualenvs/tmp-2d931a2993df6fc/bin/postdeactivate
virtualenvwrapper.user_scripts creating /home/sander/.virtualenvs/tmp-2d931a2993df6fc/bin/preactivate
virtualenvwrapper.user_scripts creating /home/sander/.virtualenvs/tmp-2d931a2993df6fc/bin/postactivate
virtualenvwrapper.user_scripts creating /home/sander/.virtualenvs/tmp-2d931a2993df6fc/bin/get_env_details
This is a temporary environment. It will be deleted when you run 'deactivate'.
(tmp-2d931a2993df6fc) sander@bedevere:~/.virtualenvs/tmp-2d931a2993df6fc$ pip install --quiet -U pip setuptools pytest
(tmp-2d931a2993df6fc) sander@bedevere:~/.virtualenvs/tmp-2d931a2993df6fc$ pip list | grep -e pip -e setuptools -e pytest
pip        23.2.1
pytest     7.4.2
setuptools 68.2.2
(tmp-2d931a2993df6fc) sander@bedevere:~/.virtualenvs/tmp-2d931a2993df6fc$ pip install --quiet pytest-repeat
(tmp-2d931a2993df6fc) sander@bedevere:~/.virtualenvs/tmp-2d931a2993df6fc$ tree lib/python3.11/site-packages/__pycache__/
lib/python3.11/site-packages/__pycache__/
├── py.cpython-311.pyc
├── pytest_repeat.cpython-311.pyc
└── _virtualenv.cpython-311.pyc

1 directory, 3 files
(tmp-2d931a2993df6fc) sander@bedevere:~/.virtualenvs/tmp-2d931a2993df6fc$ pip uninstall pytest-repeat  # observe normal uninstall behavior
Found existing installation: pytest-repeat 0.9.2
Uninstalling pytest-repeat-0.9.2:
  Would remove:
    /home/sander/.virtualenvs/tmp-2d931a2993df6fc/lib/python3.11/site-packages/pytest_repeat-0.9.2.dist-info/*
    /home/sander/.virtualenvs/tmp-2d931a2993df6fc/lib/python3.11/site-packages/pytest_repeat.py
Proceed (Y/n)? y
  Successfully uninstalled pytest-repeat-0.9.2
(tmp-2d931a2993df6fc) sander@bedevere:~/.virtualenvs/tmp-2d931a2993df6fc$ tree lib/python3.11/site-packages/__pycache__/
lib/python3.11/site-packages/__pycache__/
├── py.cpython-311.pyc
└── _virtualenv.cpython-311.pyc

1 directory, 2 files
(tmp-2d931a2993df6fc) sander@bedevere:~/.virtualenvs/tmp-2d931a2993df6fc$ pip install --quiet pytest-repeat
(tmp-2d931a2993df6fc) sander@bedevere:~/.virtualenvs/tmp-2d931a2993df6fc$ tree lib/python3.11/site-packages/__pycache__/
lib/python3.11/site-packages/__pycache__/
├── py.cpython-311.pyc
├── pytest_repeat.cpython-311.pyc
└── _virtualenv.cpython-311.pyc

1 directory, 3 files
(tmp-2d931a2993df6fc) sander@bedevere:~/.virtualenvs/tmp-2d931a2993df6fc$ pytest --help > /dev/null
(tmp-2d931a2993df6fc) sander@bedevere:~/.virtualenvs/tmp-2d931a2993df6fc$ tree lib/python3.11/site-packages/__pycache__/
lib/python3.11/site-packages/__pycache__/
├── py.cpython-311.pyc
├── pytest_repeat.cpython-311.pyc
├── pytest_repeat.cpython-311-pytest-7.4.2.pyc
└── _virtualenv.cpython-311.pyc

1 directory, 4 files
(tmp-2d931a2993df6fc) sander@bedevere:~/.virtualenvs/tmp-2d931a2993df6fc$ pip uninstall pytest-repeat  # observe uninstall behavior after pytest has run
Found existing installation: pytest-repeat 0.9.2
Uninstalling pytest-repeat-0.9.2:
  Would remove:
    /home/sander/.virtualenvs/tmp-2d931a2993df6fc/lib/python3.11/site-packages/pytest_repeat-0.9.2.dist-info/*
    /home/sander/.virtualenvs/tmp-2d931a2993df6fc/lib/python3.11/site-packages/pytest_repeat.py
Proceed (Y/n)? y
  Successfully uninstalled pytest-repeat-0.9.2
(tmp-2d931a2993df6fc) sander@bedevere:~/.virtualenvs/tmp-2d931a2993df6fc$ tree lib/python3.11/site-packages/__pycache__/
lib/python3.11/site-packages/__pycache__/
├── py.cpython-311.pyc
├── pytest_repeat.cpython-311-pytest-7.4.2.pyc
└── _virtualenv.cpython-311.pyc

1 directory, 3 files

[pelson]

If the tooling doesn't do that, we might need to start storing them somewhere else

FYI, the recommendation in https://github.com/pypa/packaging.python.org/pull/1423/files is to update the RECORD if you are going to modify the package. That is an alternative option.

[RonnyPfannschmidt]

pytest should never change installed packages

we do need to consider storing the cache somewhere else

[guillon]

I also have the issue in another setup (python 3.12, pytest 8.3.3) where we suffer from the same behavior as described in the initial comment of the issue.

Actually, as pytest generates additional cache files in installed package directories, when uninstalling the package, the install directory remains due to these undeclared <prefix>/lib/python3.12/<package>/__pycache__/*.cpython-312-pytest-8.3.3 cache files. This is problematic as then, even after uninstall of the package:

• importing the package may work (but it is then considered a namespace package with no attribute at all as it is empty), runtime errors come latter when accessing attributes,
• it can propagate down to any package dependency, breaking uninstall of unrelated packages as soon as they are imported at some point.

Does someone knows optionally at which version of pytest this behavior was introduced? It would help as a temporary workaround.

Also, does someone knows if there is some ongoing work on this or if we should propose a MR?

Note that I observed that python sources for which the cache entry was created in external package dirs contains *_test.py (which is not a test file, actually a source file of the package with this name).

I've changed the source tree to rename the *_test.py file to another name without test (actually *_check.py). Then, cleaned by hand the install trees and redo the experiment: install package, run tests with pytest, uninstall package. The problem did not appear anymore (no pytest __pycache__ entry have been created).

It may be that the caching heuristic used in pytest accounts also for the filename (it seems that it must contain test somewhere of what I observed).