Coverage.py fails to omit *third-party* coverage in a nested venv from the same implicit namespace package

[webknjaz]

Describe the bug

I have a case when I get unexpected coverage measurements from a tox-managed virtualenv existing within the source tree. Coveragepy is invoked via pytest-cov but all configuration is in coveragerc so I'm pretty sure this issue belongs here.

This reminds me of #876 and #905. Having stared at https://github.com/nedbat/coveragepy/commit/0285af966a3942d8bd63489bd285328e96221126 and https://github.com/nedbat/coveragepy/commit/5c2f614e01d35271f7907d85050115071cf24e87, I concluded that the fixes are still imperfect.

The reproducers in those issues showcased regular non-namespaces packages, which is why it wasn't on anybody's radar, I think. But I'm sure my case is legit too.

In the example below, awx_plugins is the namespace. The dependency is the awx_plugins.interfaces importable and the project itself contains awx_plugins.credentials and awx_plugins.inventory. The project's files are under src/awx_plugins/{credentials,inventory}/*.py and tests/*.py. The dependecy has src/awx_plugins/interfaces/*.py in the repo, but all are installed side-by-side as .tox/py/lib/python3.13/site-packages/awx_plugins/{credentials,interfaces,inventory}/*.py. .tox/py/lib/python3.13/site-packages/awx_plugins/__init__.py does not exist. .tox/py/lib/python3.13/site-packages/awx_plugins/{credentials,interfaces,inventory}/__init__.py do not exist either.

The project is configured with

[paths]
_site-packages-to-src-mapping =
  src
  */src
  *\src
  */lib/pypy*/site-packages
  */lib/python*/site-packages
  *\Lib\site-packages

which maps all the project-local source files properly (which is visible in the report correctly). The third-party ones remain unmatched and show up with site-packages paths.

There's also

[run]
relative_files = true
source =
  .
source_pkgs =
  awx_plugins

The generic paths and namespaces effectively give me the ability to copy the good coveragerc file over into similar packages of the same namespace and folder layout. This is intentional.

I'm almost sure changing source_pkgs to list awx_plugins.credentials and awx_plugins.inventory would limit what ends up in the report but that's not something I want given that I wish to have a reusable config across an ecosystem of similar projects (i.e. plugins to something else).

I've tested that adding

omit =
  .tox/**

is a functional workaround, following the suggestions in other issues.

But ultimately, I believe that this is a bug fixable within coveragepy and I shouldn't have to keep the omit settings in my configs.

To Reproduce

1. What version of Python are you using? A range of Python 3.11-3.13, I'm pretty sure the problem is agnostic, though.

2. What version of coverage.py shows the problem?

   👉 click to unfold 👈

   ```console $ .tox/py/bin/python -m coverage debug sys -- sys ------------------------------------------------------- coverage_version: 7.6.1 coverage_module: ~/src/github/ansible/awx-plugins/.tox/py/lib/python3.13/site-packages/coverage/__init__.py core: -none- CTracer: available plugins.file_tracers: -none- plugins.configurers: covdefaults.CovDefaults plugins.context_switchers: -none- configs_attempted: ~/src/github/ansible/awx-plugins/.coveragerc configs_read: ~/src/github/ansible/awx-plugins/.coveragerc config_file: ~/src/github/ansible/awx-plugins/.coveragerc config_contents: b'[html]\nshow_contexts = true\nskip_covered = false\n\n[paths]\n_site-packages-to-src-mapping =\n src\n */src\n *\\src\n */lib/pypy*/site-packages\n */lib/python*/site-packages\n *\\Lib\\site-packages\n\n[report]\n# `fail_under` is set here temporarily until it can be dropped:\nfail_under = 39.27\nskip_covered = true\nskip_empty = true\nshow_missing = true\nexclude_also =\n ^\\s*@pytest\\.mark\\.xfail\n\n[run]\nbranch = true\ncover_pylib = false\n# https://coverage.rtfd.io/en/latest/contexts.html#dynamic-contexts\n# dynamic_context = test_function # conflicts with `pytest-cov` if set here\nomit =\n .tox/**\nparallel = true\nplugins =\n covdefaults\nrelative_files = true\nsource =\n .\nsource_pkgs =\n awx_plugins\n' data_file: -none- python: 3.13.0b2 (main, Jun 23 2024, 13:25:09) [GCC 13.2.1 20240113] platform: Linux-6.6.13-gentoo-dist-x86_64-Intel-R-_Core-TM-_i7-9850H_CPU_@_2.60GHz-with-glibc2.38 implementation: CPython gil_enabled: True executable: ~/src/github/ansible/awx-plugins/.tox/py/bin/python def_encoding: utf-8 fs_encoding: utf-8 pid: 31740 cwd: ~/src/github/ansible/awx-plugins path: ~/src/github/ansible/awx-plugins ~/.pyenv/versions/3.13.0b2/lib/python313.zip ~/.pyenv/versions/3.13.0b2/lib/python3.13 ~/.pyenv/versions/3.13.0b2/lib/python3.13/lib-dynload ~/src/github/ansible/awx-plugins/.tox/py/lib/python3.13/site-packages ~/src/github/ansible/awx-plugins/src environment: HOME = ~ PYENV_ROOT = ~/.pyenv PYENV_SHELL = zsh PYENV_VIRTUALENV_INIT = 1 command_line: ~/src/github/ansible/awx-plugins/.tox/py/lib/python3.13/site-packages/coverage/__main__.py debug sys sqlite3_sqlite_version: 3.44.2 sqlite3_temp_store: 0 sqlite3_compile_options: ATOMIC_INTRINSICS=1, COMPILER=gcc-13.2.1 20240113, DEFAULT_AUTOVACUUM, DEFAULT_CACHE_SIZE=-2000, DEFAULT_FILE_FORMAT=4, DEFAULT_JOURNAL_SIZE_LIMIT=-1, DEFAULT_MMAP_SIZE=0, DEFAULT_PAGE_SIZE=4096, DEFAULT_PCACHE_INITSZ=20, DEFAULT_RECURSIVE_TRIGGERS, DEFAULT_SECTOR_SIZE=4096, DEFAULT_SYNCHRONOUS=2, DEFAULT_WAL_AUTOCHECKPOINT=1000, DEFAULT_WAL_SYNCHRONOUS=2, DEFAULT_WORKER_THREADS=0, ENABLE_API_ARMOR, ENABLE_BYTECODE_VTAB, ENABLE_COLUMN_METADATA, ENABLE_DBPAGE_VTAB, ENABLE_DBSTAT_VTAB, ENABLE_EXPLAIN_COMMENTS, ENABLE_FTS3, ENABLE_FTS3_PARENTHESIS, ENABLE_FTS4, ENABLE_FTS5, ENABLE_GEOPOLY, ENABLE_HIDDEN_COLUMNS, ENABLE_ICU, ENABLE_MATH_FUNCTIONS, ENABLE_MEMSYS5, ENABLE_NORMALIZE, ENABLE_OFFSET_SQL_FUNC, ENABLE_PREUPDATE_HOOK, ENABLE_RBU, ENABLE_RTREE, ENABLE_SESSION, ENABLE_STMTVTAB, ENABLE_STMT_SCANSTATUS, ENABLE_UNKNOWN_SQL_FUNCTION, ENABLE_UNLOCK_NOTIFY, ENABLE_UPDATE_DELETE_LIMIT, HAVE_ISNAN, MALLOC_SOFT_LIMIT=1024, MAX_ATTACHED=10, MAX_COLUMN=2000, MAX_COMPOUND_SELECT=500, MAX_DEFAULT_PAGE_SIZE=8192, MAX_EXPR_DEPTH=1000, MAX_FUNCTION_ARG=127, MAX_LENGTH=1000000000, MAX_LIKE_PATTERN_LENGTH=50000, MAX_MMAP_SIZE=0x7fff0000, MAX_PAGE_COUNT=1073741823, MAX_PAGE_SIZE=65536, MAX_SQL_LENGTH=1000000000, MAX_TRIGGER_DEPTH=1000, MAX_VARIABLE_NUMBER=32766, MAX_VDBE_OP=250000000, MAX_WORKER_THREADS=8, MUTEX_PTHREADS, SOUNDEX, SYSTEM_MALLOC, TEMP_STORE=1, THREADSAFE=1, USE_URI ```

3. What versions of what packages do you have installed?

   👉 click to unfold 👈

   ```console $ .tox/py/bin/python -m pip freeze adal==1.2.7 attrs==24.2.0 awx-plugins-core @ file://~/src/github/ansible/awx-plugins/.tox/.tmp/package/97/awx_plugins_core-0.0.1a5.dev213%2Bg8ea07661f9.d20241004-0.editable-py3-none-any.whl#sha256=3ca7389cb589da068511245e32662a41eefebf9ecfbbdfa7c41025be26f55c12 awx_plugins.interfaces @ git+https://github.com/ansible/awx_plugins.interfaces.git@3a78f59ffe922638f37bf63b4243aac41ed8abd8 azure-core==1.31.0 azure-identity==1.18.0 azure-keyvault==4.2.0 azure-keyvault-certificates==4.8.0 azure-keyvault-keys==4.9.0 azure-keyvault-secrets==4.8.0 boto3==1.35.31 botocore==1.35.31 certifi==2024.8.30 cffi==1.17.1 charset-normalizer==3.3.2 covdefaults==2.3.0 coverage==7.6.1 coverage-enable-subprocess==1.0 cryptography==43.0.1 execnet==2.1.1 hypothesis==6.112.2 idna==3.10 iniconfig==2.0.0 isodate==0.6.1 jmespath==1.0.1 msal==1.31.0 msal-extensions==1.2.0 msrest==0.7.1 msrestazure==0.6.4.post1 oauthlib==3.2.2 packaging==24.1 pluggy==1.5.0 portalocker==2.10.1 pycparser==2.22 PyJWT==2.9.0 pytest==8.3.3 pytest-cov==5.0.0 pytest-mock==3.14.0 pytest-xdist==3.6.1 python-dateutil==2.9.0.post0 python-dsv-sdk==1.0.4 python-tss-sdk==1.2.3 PyYAML==6.0.2 requests==2.32.3 requests-oauthlib==2.0.0 s3transfer==0.10.2 six==1.16.0 sortedcontainers==2.4.0 typing_extensions==4.12.2 urllib3==2.2.3 ```
4. What code shows the problem? https://github.com/ansible/awx-plugins/commit/8ea07661f9da09dedaa52b569978c448fb871931
5. What commands should we run to reproduce the problem?

   $ python3 -Im pip install tox
   $ git clone https://github.com/ansible/awx-plugins.git
   $ cd awx-plugins
   $ python3 -Im tox
   $ python3 -Im webbrowser .tox/py/tmp/htmlcov/index.html

That shows a couple of files under .tox/py/lib/python3.13/site-packages/awx_plugins/interfaces/**. They should not be listed.

Expected behavior

Coveragepy should not collect coverage in third-party modules from the same implicit namespace, installed in the same virtualenv.

Additional context

The concrete example is inspectable at https://app.codecov.io/gh/ansible/awx-plugins/commit/8ea07661f9da09dedaa52b569978c448fb871931/tree?flags%5B0%5D=pytest. The pre-selected pytest flag is what you want to have selected at all times since there's also MyPy coverage that's overlayed if you deselect it.

[webknjaz]

I've tested that adding

omit =
  .tox/**

is a functional workaround, following the suggestions in other issues.

Nope. It only appeared to be a full workaround, but it's not. It does what I expected in the local HTML/XML reports, but excludes all of the awx_plugins namespace files from the XML report in CI, making Codecov not show anything. I checked that codecov receives XML files with tests/ paths while locally I also have src/ paths. The coveragepy version is the same and other things should be as well, due to how I pin the environment deps. This is very weird...

[webknjaz]

I ended up changing that to a granular

omit =
  .tox/*/lib/pypy*/site-packages/awx_plugins/interfaces/**
  .tox/*/lib/python*/site-packages/awx_plugins/interfaces/**
  .tox\*\Lib\site-packages\awx_plugins\interfaces\**

for now, but I don't understand why running the same command in the CI and locally would produce different coverage reports. It seems like the environment somehow influences how omit works, which is rather confusing.