search

nedbat / coveragepy

The code coverage tool for Python
https://coverage.readthedocs.io
Apache License 2.0
2.99k stars 429 forks source link

Feature Request - Function and class coverage in "coverage report" #1859

Open numbworks opened 3 weeks ago

numbworks commented 3 weeks ago

Problem

As stated in this thread, function and class coverage has been added in coverage v7.5.0!

Thank you @nedbat for this game-changer feature!

The problem is that it requires to open two HTML files ("class_index.html" and "function_index.html") in a browser at each run of coverage html, which doesn't suit terminal-based CI/CD pipelines and workflows.

Feature Request

I kindly request, if possible, to have the same content of "class_index.html" and "function_index.html" as output of the coverage report command? - i.e:

coverage report --functions
coverage report --classes

Current Workaround

The very inelegant workaround I'm using at the moment on my Debian 12-based devcontainer is the following one:

  1. Install the latest version of coverage (>= 7.5.0):
pip install coverage==7.6.1
  1. Install html2text:
wget http://ftp.de.debian.org/debian/pool/main/h/html2text/html2text_1.3.2a-28_amd64.deb
dpkg -i html2text_1.3.2a-28_amd64.deb
  1. Run the following commands in my tests folders:
cd tests
coverage run
coverage html

  1. A htmlcov sub-folder is created, containing the two HTML files we need ("class_index.html" and "function_index.html").

  2. Extract and format the table from the HTML file(s):

cd htmlcov
sed -n '/<table class="index" data-sortable>/,/<\/table>/p' class_index.html > class_index.txt
html2text -width 400 class_index.txt
  1. Done!

As you can imagine, it's not the most practical solution.

devdanzin commented 2 weeks ago

I have an old branch that adds something like this: https://github.com/nedbat/coveragepy/compare/master...devdanzin:coveragepy:report_on_regions. There's a sample of what it outputs (or used to, not sure it's working after recent refactorings) in issue #1793 (where it was decided to not add this information to the textual report then).

I won't be able to work on it for a couple of weeks, but would be glad to polish and update it if it's deemed desirable. Feel free to adapt it and submit as a PR before I'm back if you want to (and @nedbat agrees, of course).

numbworks commented 2 weeks ago

@devdanzin Thank you for your answer!

I won't be able to work on it for a couple of weeks, but would be glad to polish and update it if it's deemed desirable. Feel free to adapt it and submit as a PR before I'm back if you want to

I'm a Python developer myself, but I'm not familiar with the coverage.py codebase and it would take me some days in sequence to understand its structure (which I lack of at the moment) before being able to contribute.

I bet it will be way faster to wait for somebody else to add this feature! I'll keep using my workaround until then!

devdanzin commented 2 weeks ago

I bet it will be way faster to wait for somebody else to add this feature!

Good news: I've merged master into my branch and it works as-is. If you can, please install coverage from that branch and check whether it does everything you want, the way you'd like it to. I'm open to changing the output format and other features to better suit your needs.

For this to land as an official feature, besides being approved by the maintainer, it would still need some tests and some cleanup (mostly to mirror the cleaner code organization in jsonreport.py). I should be able to work on this during the weekend.

numbworks commented 2 weeks ago

@devdanzin Thank you for your time and work!

I'll try to find two hours to setup a temp devcontainer and try it out.

How can I install your updated version of coverage.py? Does pip install <github link> work?

In the meanwhile, I do provide to you a first quick answer!

I would expect your feature to output a formatted table like the one that I get from my latest workaround:

coveragepy_and_pandoc

The screenshot comes from the following project: nwreadinglist.

My latest workaround works by installing Pandoc in the dev environment (devcontainer):

FROM python:3.12.5-bookworm

# ...
RUN pip install coverage==7.6.1
# ...

RUN wget https://github.com/jgm/pandoc/releases/download/3.4/pandoc-3.4-1-amd64.deb \
    && dpkg -i pandoc-3.4-1-amd64.deb \
    && rm -f pandoc-3.4-1-amd64.deb

and asking it to format the table I extract from htmlcov:

cd tests
coverage run -m unittest nwreadinglisttests.py
coverage html --omit=nwreadinglisttests.py && sed -n '/<table class="index" data-sortable>/,/<\/table>/p' htmlcov/class_index.html | pandoc --from html --to plain && sleep 3 && rm -rf htmlcov
devdanzin commented 2 weeks ago

How can I install your updated version of coverage.py? Does pip install <github link> work?

I tested and pip install -U git+https://github.com/devdanzin/coveragepy.git@report_on_regions seems to work for that.

I would expect your feature to output a formatted table like the one that I get from my latest workaround:

coveragepy_and_pandoc

Currently output for coverage report --functions --classes (it's possible to pass either, both or none) is formatted like below. It should be simple to format like what you have, except that it's not currently possible to get the excluded lines data from the text report (see https://github.com/nedbat/coveragepy/issues/814).

Name                                              Stmts   Miss  Cover
---------------------------------------------------------------------
coverage/version.py                                  16      0   100%
coverage/version.py: function _make_url               1      0   100%
coverage/version.py: function _make_version           8      0   100%
setup.py                                             68     25    63%
setup.py: class BuildFailed                           2      2     0%
setup.py: class ve_build_ext                         12     12     0%
setup.py: function BuildFailed.__init__               2      2     0%
setup.py: function main                               8      5    38%
setup.py: function ve_build_ext.build_extension       8      8     0%
setup.py: function ve_build_ext.run                   4      4     0%
---------------------------------------------------------------------
TOTAL                                                84     25    70%
numbworks commented 1 week ago

@devdanzin Finally Sunday! I apologize for the slightly delayed answer!

except that it's not currently possible to get the excluded lines data from the text report

As far as there is the class name and the coverage % in a formatted table, it's good enough to quickly add tests where required!

Thank you for your work so far! I'll update this thread once I try it out!