nedbat / coveragepy

The code coverage tool for Python
https://coverage.readthedocs.io
Apache License 2.0
3.01k stars 433 forks source link

adding ... (the ellipsis) to exclude_lines ignores all source #1042

Open cjw296 opened 4 years ago

cjw296 commented 4 years ago

Describe the bug

Adding this to .coveragerc results in all source being ignored:

[report]
exclude_lines =
  ...

To Reproduce How can we reproduce the problem? Please be specific. Don't just link to a failing CI job. Answer the questions below:

  1. What version of Python are you using? Reproduced 3.6 through 3.9
  2. What version of coverage.py are you using? Reproduced with 4.x through 5.3
  3. What code are you running? https://github.com/Simplistix/mush/pull/6/commits/8488a4726e0829988d8df5507b490a3ec73424d9
  4. What commands did you run? rm -f .coverage && coverage run -m pytest && coverage report

Expected behavior

Only lines of the following form to be excluded from coverage checking:

def some_example_function_that_will_never_be_called():
   ...

Additional context

The results of the output from this bug are super confusing:

Name                               Stmts   Miss  Cover
------------------------------------------------------
mush/__init__.py                       0      0   100%
mush/asyncio.py                        0      0   100%
mush/callpoints.py                     0      0   100%
mush/compat.py                         0      0   100%
mush/context.py                        0      0   100%
mush/declarations.py                   0      0   100%
mush/extraction.py                     0      0   100%
mush/markers.py                        0      0   100%
mush/modifier.py                       0      0   100%
mush/plug.py                           0      0   100%
mush/requirements.py                   0      0   100%
mush/resources.py                      0      0   100%
mush/runner.py                         0      0   100%
mush/tests/__init__.py                 0      0   100%
mush/tests/helpers.py                  0      0   100%
mush/tests/test_async_context.py       0      0   100%
mush/tests/test_async_runner.py        0      0   100%
mush/tests/test_callpoints.py          0      0   100%
mush/tests/test_context.py             0      0   100%
mush/typing.py                         0      0   100%
------------------------------------------------------
TOTAL                                  0      0   100%

...but I guess that makes sense in the context of the ellipsis causing all lines to be excluded.

AllanChain commented 3 years ago

https://coverage.readthedocs.io/en/coverage-5.3/excluding.html#advanced-exclusion

Coverage.py identifies exclusions by matching lines against a list of regular expressions.

... is recognized as a regular expression. You need to escape it :)

cjw296 commented 3 years ago

Sure, I wonder if there should be a blacklist of regexes?

nedbat commented 3 years ago

I could make a deny list that included "...", but:

  1. What else would go on it?
  2. Is there a cleverer way? If your pattern matches all three strings on a built-in "absurd list", then it probably matches everything.
  3. Or another cleverer way: if your pattern matches every line in any given source file, then it should be a warning.
cjw296 commented 3 years ago
  1. I guess this could grow as people report things? ... is problematic for me because it's a python literal that I use commonly in tests to indicate "this isn't import to the test" more explicitly than a pass statement.

  2. Another good safety net might be that all code in a file is excluded even though the file itself hasn't been excluded?

  3. Yes, that, but maybe for multiple exclude patterns? I can't think of a legit use case where a whole file would end up being excluded by ever line in it matching one or more exclude regexes.

HexDecimal commented 1 year ago

Since a corrected regular expression hasn't been mentioned yet I'd suggest the following: ellipsis with leading whitespace: ^\s*\.\.\.

Can be added to pyproject.toml like this:

[tool.coverage.report]
exclude_lines = ['^\s*\.\.\.']

Or .coveragerc like this:

[report]
exclude_lines =
  ^\s*\.\.\.

Excludes the ... commonly used in never-called function stubs, including ones with inline comments:

def foo():
    ... # comment

But doesn't exclude general uses of ... in code:

foo = ...
def baz(
   x = ...,
):
    """..."""
    pass # ...
baz(...)

The only problematic code I can think of is an ellipsis within a multi-line statement:

foo = (
    ...,
)
bar = (
    ...
)
"""
...
"""

Perhaps the docs could mention this regular expression as the correct way of excluding ....