ryanluker / vscode-coverage-gutters

Display test coverage generated by lcov and xml - works with many languages
https://marketplace.visualstudio.com/items?itemName=ryanluker.vscode-coverage-gutters
MIT License
460 stars 88 forks source link

Coverage gutters not showing for test scripts #389

Closed wardy3 closed 1 year ago

wardy3 commented 1 year ago

Describe the bug Coverage gutters seem to work fine on my actual script but not on the test script itself

> coverage report -m tests/isi_sdk_tests/test_isi_alloc.py isi_alloc.py                                                                                                           

Name                                    Stmts   Miss  Cover   Missing
---------------------------------------------------------------------
isi_alloc.py                              120     64    47%   174-182, 200-232, 247-258, 270-271, 289-292, 321-349
tests/isi_sdk_tests/test_isi_alloc.py     823      0   100%
---------------------------------------------------------------------
TOTAL                                     943     64    93%

The coverage file seems to be ok. These figures are correct. So I haven't written tests for the whole script but all the existing tests are being run.

However, the red/green colouring and the status bar value are not working for the test script.

It says "47% Coverage" for isi_alloc.py which matches the report. But for the test, I get "2% Coverage". The first few lines are marked green but then it seems to randomly highlight them, as if it's referring to a different file.

image

To Reproduce This is happening for all my test scripts but not for the actual scripts themselves. Even test scripts that are directly under tests/ are affected

Expected behaviour I would expect the gutters in the test to behave similar to the script.

Desktop (please complete the following information):

Additional context Add any other context about the problem here.

ryanluker commented 1 year ago

Thanks for the ticket @wardy3!

I vaguely remember other people having this issue, and it was due to how the extension matches coverage to source files.

Can you provide a copy of the coverage section structure that is printed in the debug logs?

cmd(ctrl)-shift-p -> Search Show Logs -> Extension Host

In addition to that, you could try using absolute paths instead of relative and that might fix your issue as well. (note, your coverage file will need to also provide absolute paths instead of relative as well) https://github.com/ryanluker/vscode-coverage-gutters/blob/master/package.json#L112-L126

See an example here of a cov.xml https://github.com/ryanluker/vscode-coverage-gutters/blob/master/example/python/cov.xml#L6

wardy3 commented 1 year ago

The extension host output is blank. I tried going via cmd-shift-p -> Developer: show logs... -> Extension host and it brings me to the same place

as far as coverage-gutters output though, there's this (with the source and test file I'm playing with now)

[1676359753825][coverageservice]: LOADING
[1676359758272][coverageservice]: Loading 2 file(s)
[1676359758272][coverageservice]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/cov.xml,/Users/wardy/Documents/EMC/gitlab.nosync/storage/tests/cov.xml
[1676359758275][coverageservice]: Loaded 2 data file(s)
[1676359758564][coverageservice]: Caching 179 coverage(s)
[1676359758564][coverageservice]: READY
[1676359758564][coverageservice]: RENDERING
[1676359758565][renderer][section test name]: isi_alloc.py
[1676359758565][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/isi_alloc.py
[1676359758565][renderer][section test name]: test_isi_alloc.py
[1676359758565][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/tests/isi_sdk_tests/test_isi_alloc.py
[1676359758566][renderer][section test name]: test_isi_alloc.py
[1676359758566][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/tests/isi_sdk_tests/test_isi_alloc.py
[1676359758566][coverageservice]: READY
[1676359758567][coverageservice]: Listening to file system at {/Users/wardy/Documents/EMC/gitlab.nosync/storage}/**/{lcov.info,cov.xml,coverage.xml,jacoco.xml,coverage.cobertura.xml}
[1676359770453][coverageservice]: RENDERING
[1676359770454][renderer][section test name]: isi_alloc.py
[1676359770454][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/isi_alloc.py
[1676359770455][renderer][section test name]: test_isi_alloc.py
[1676359770455][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/tests/isi_sdk_tests/test_isi_alloc.py
[1676359770456][coverageservice]: READY
[1676359803083][coverageservice]: RENDERING
[1676359803084][renderer][section test name]: isi_alloc.py
[1676359803084][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/isi_alloc.py
[1676359803084][renderer][section test name]: test_isi_alloc.py
[1676359803084][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/tests/isi_sdk_tests/test_isi_alloc.py
[1676359803085][renderer][section test name]: test_isi_alloc.py
[1676359803085][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/tests/isi_sdk_tests/test_isi_alloc.py
[1676359803085][coverageservice]: READY
[1676359831041][coverageservice]: RENDERING
[1676359831042][renderer][section test name]: isi_alloc.py
[1676359831042][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/isi_alloc.py
[1676359831042][renderer][section test name]: test_isi_alloc.py
[1676359831042][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/tests/isi_sdk_tests/test_isi_alloc.py
[1676359831043][coverageservice]: READY
[1676359848779][coverageservice]: RENDERING
[1676359848780][renderer][section test name]: isi_alloc.py
[1676359848780][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/isi_alloc.py
[1676359848781][renderer][section test name]: test_isi_alloc.py
[1676359848781][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/tests/isi_sdk_tests/test_isi_alloc.py
[1676359848781][renderer][section test name]: test_isi_alloc.py
[1676359848781][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/tests/isi_sdk_tests/test_isi_alloc.py
[1676359848781][coverageservice]: READY
[1676359848903][coverageservice]: RENDERING
[1676359848904][renderer][section test name]: isi_alloc.py
[1676359848904][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/isi_alloc.py
[1676359848905][renderer][section test name]: test_isi_alloc.py
[1676359848905][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/tests/isi_sdk_tests/test_isi_alloc.py
[1676359848905][coverageservice]: READY
[1676359863721][coverageservice]: RENDERING
[1676359863721][renderer][section test name]: isi_alloc.py
[1676359863721][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/isi_alloc.py
[1676359863722][renderer][section test name]: test_isi_alloc.py
[1676359863722][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/tests/isi_sdk_tests/test_isi_alloc.py
[1676359863723][renderer][section test name]: test_isi_alloc.py
[1676359863723][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/tests/isi_sdk_tests/test_isi_alloc.py
[1676359863723][coverageservice]: READY
[1676359879010][coverageservice]: RENDERING
[1676359879010][renderer][section test name]: isi_alloc.py
[1676359879010][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/isi_alloc.py
[1676359879011][renderer][section test name]: test_isi_alloc.py
[1676359879011][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/tests/isi_sdk_tests/test_isi_alloc.py
[1676359879012][coverageservice]: READY

Yeah, this seems to have fixed it

    "coverage-gutters.coverageBaseDir": "/Users/wardy/Documents/EMC/gitlab.nosync/storage/cov.xml",
    "coverage-gutters.manualCoverageFilePaths": [
        "/Users/wardy/Documents/EMC/gitlab.nosync/storage/cov.xml"
    ]

the cov.xml file already had a full path

    <sources>
        <source>/Users/wardy/Documents/EMC/gitlab.nosync/storage</source>
    </sources>

and

                <class name="test_isi_alloc.py" filename="tests/isi_sdk_tests/test_isi_alloc.py" complexity="0" line-rate="1" branch-rate="0">

main difference I can see in the log is this line:

[1676361128927][coverageservice]: Listening to file system at {/Users/wardy/Documents/EMC/gitlab.nosync/storage/cov.xml}

Was it getting confused with the glob pattern before?

(more log after it started working below)

[1676361128927][coverageservice]: READY
[1676361128927][coverageservice]: Listening to file system at {/Users/wardy/Documents/EMC/gitlab.nosync/storage/cov.xml}
[1676361140894][coverageservice]: RENDERING
[1676361140895][renderer][section test name]: test_isi_alloc.py
[1676361140895][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/tests/isi_sdk_tests/test_isi_alloc.py
[1676361140896][renderer][section test name]: test_isi_alloc.py
[1676361140896][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/tests/isi_sdk_tests/test_isi_alloc.py
[1676361140896][coverageservice]: READY
[1676361158626][coverageservice]: RENDERING
[1676361158627][coverageservice]: READY
[1676361158673][coverageservice]: RENDERING
[1676361158674][coverageservice]: READY
[1676361169382][coverageservice]: LOADING
[1676361169382][coverageservice]: Loading 1 file(s)
[1676361169382][coverageservice]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/cov.xml
[1676361169383][coverageservice]: Loaded 1 data file(s)
[1676361169383][coverageservice]: Caching 0 coverage(s)
[1676361169383][coverageservice]: READY
[1676361169383][coverageservice]: RENDERING
[1676361169383][coverageservice]: READY
[1676361170689][coverageservice]: RENDERING
[1676361170689][coverageservice]: READY
[1676361170732][coverageservice]: RENDERING
[1676361170733][coverageservice]: READY
[1676361173143][coverageservice]: LOADING
[1676361173144][coverageservice]: Loading 1 file(s)
[1676361173144][coverageservice]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/cov.xml
[1676361173145][coverageservice]: Loaded 1 data file(s)
[1676361173600][coverageservice]: Caching 178 coverage(s)
[1676361173600][coverageservice]: READY
[1676361173600][coverageservice]: RENDERING
[1676361173601][renderer][section test name]: test_isi_alloc.py
[1676361173601][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/tests/isi_sdk_tests/test_isi_alloc.py
[1676361173603][renderer][section test name]: test_isi_alloc.py
[1676361173603][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/tests/isi_sdk_tests/test_isi_alloc.py
[1676361173603][coverageservice]: READY
[1676361177818][coverageservice]: RENDERING
[1676361177819][renderer][section test name]: test_isi_alloc.py
[1676361177819][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/tests/isi_sdk_tests/test_isi_alloc.py
[1676361177820][renderer][section test name]: isi_alloc.py
[1676361177820][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/isi_alloc.py
[1676361177820][renderer][section test name]: isi_alloc.py
[1676361177820][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/isi_alloc.py
[1676361177820][coverageservice]: READY
[1676361183304][coverageservice]: RENDERING
[1676361183304][renderer][section test name]: test_isi_alloc.py
[1676361183304][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/tests/isi_sdk_tests/test_isi_alloc.py
[1676361183305][coverageservice]: READY
[1676361183343][coverageservice]: RENDERING
[1676361183343][renderer][section test name]: test_isi_alloc.py
[1676361183343][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/tests/isi_sdk_tests/test_isi_alloc.py
[1676361183344][coverageservice]: READY
[1676361213232][coverageservice]: RENDERING
[1676361213233][renderer][section test name]: test_isi_alloc.py
[1676361213233][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/tests/isi_sdk_tests/test_isi_alloc.py
[1676361213234][coverageservice]: READY
[1676361262494][coverageservice]: RENDERING
[1676361262495][renderer][section test name]: test_isi_alloc.py
[1676361262495][renderer][section file path]: /Users/wardy/Documents/EMC/gitlab.nosync/storage/tests/isi_sdk_tests/test_isi_alloc.py
[1676361262496][coverageservice]: READY
[1676361266262][coverageservice]: RENDERING
[1676361266263][coverageservice]: READY
[1676361332183][coverageservice]: RENDERING
[1676361332183][coverageservice]: READY

thanks!

ryanluker commented 1 year ago

Hmm odd that the shortcut for the developer logs didn't work... glad you found the logs another way though! I also think you are on the right track with Was it getting confused with the glob pattern before? as that might have been why it was confusing the two file paths and mixing up the coverages 🤔.

Thanks again for the output as well and I am glad that the absolute work around helped you out in your scenario 💪🏻.