ryanluker / vscode-coverage-gutters

Display test coverage generated by lcov and xml - works with many languages
https://marketplace.visualstudio.com/items?itemName=ryanluker.vscode-coverage-gutters
MIT License
454 stars 88 forks source link

cobertura Status Bar Percentage Wrong with C# Workspace #437

Open terryaney opened 5 months ago

terryaney commented 5 months ago

If I use lcov coverage format, everything seems to work. When using cobertura, the status bar percentage is wrong. Here is cobertura screen shot:

  1. Just tried highlighting that the Approve.cs file is in the correct project/folder that tests were ran on and that there is only a coverage.cobertura.xml file present.
  2. Side coverage highlighting seems to work properly.
  3. The percentage in status bar states 100% coverage (and I've toggled coverage on and off) with the following logging indicating that it seems to be processing file correctly:
[1707144812023][coverageservice]: READY
[1707144946921][coverageservice]: LOADING
[1707144949670][coverageservice]: Loading 3 file(s)
[1707144949670][coverageservice]: c:\BTR\Camelot\Api\WebService.Proxy\Api\tests\Camelot.Api.WebService.Proxy.Tests.Integration\TestResults\lcov.info,c:\BTR\Camelot\Api\DataLocker\Api\tests\Camelot.Api.DataLocker.Tests.Integration\TestResults\coverage.cobertura.xml,c:\BTR\Camelot\Api\WebService.Proxy\Api\tests\Camelot.Api.WebService.Proxy.Tests.Integration\TestResults\coverage.cobertura.xml
[1707144949677][coverageservice]: Loaded 3 data file(s)
[1707144949854][coverageservice]: Caching 951 coverage(s)
[1707144949854][coverageservice]: READY
[1707144949854][coverageservice]: RENDERING

image

If I change the lcov format, everything seems to display correctly:

image

Let me know if I can provide any more info. Thanks.

terryaney commented 5 months ago

Now, I'm getting the opposite on a Unit test project. I generated a lcov format which automatically was named to coverage.info as expected. The line number highlighting is correct (green for the GetDateForDay function is only function I wrote tests for), but the status bar says '0% Coverage'.

image

ryanluker commented 5 months ago

@terryaney Thanks for the ticket and the logs from the extension output. Definitely seems like an issue with the status % indicator, otherwise it might be coming from cobertura as that upstream library can have issues at times.

We should probably have more logging in this area of the code to assist with debugging your issues, just looking at the source code doesn't reveal anything immediately.

    public setCoverage(linePercentage: number | undefined ) {
        if (Number.isFinite(linePercentage)) {
            this.lineCoverage = `${linePercentage}%`;
        } else {
            this.lineCoverage = undefined;
        }
        this.update();
    }

Do you know if it is all files within your codebase or just a subset?

terryaney commented 5 months ago

Do you know if it is all files within your codebase or just a subset?

Unfortunately, it appears not. I wrote a few unit tests to test extension methods in one project and it seemed to display % correctly, but another few simple extension method tests in another project didn't work (second reference above where I mention the 'opposite was happening with lcov'). So I have:

  1. Integration tests (original post) that do not work with coverage.info but work with lcov
  2. Unit test in library A with lcov that does NOT work.
  3. Unit test in library B with lcov that DOES work.

Anyway to turn on meaningful logging I can give you?

ryanluker commented 4 months ago

Do you know if it is all files within your codebase or just a subset?

Unfortunately, it appears not. I wrote a few unit tests to test extension methods in one project and it seemed to display % correctly, but another few simple extension method tests in another project didn't work (second reference above where I mention the 'opposite was happening with lcov'). So I have:

1. Integration tests (original post) that do not work with `coverage.info` but work with `lcov`

2. Unit test in library A with `lcov` that does NOT work.

3. Unit test in library B with `lcov` that DOES work.

Anyway to turn on meaningful logging I can give you?

Sadly, no current way to introduce more logging easily (via an extension setting lets say). You would be able to add extra logging by using the existing channel in the compiled extension but that isn't a very fun experience. I can see about adding more logging to the existing channel pattern but it might not be in for next release sadly.

https://github.com/ryanluker/vscode-coverage-gutters/blob/master/src/extension.ts#L8-L17

terryaney commented 4 months ago

I'll wait :) My VS Code extension experience is close to NIL. Thanks for update.

ryanluker commented 4 months ago

I'll wait :) My VS Code extension experience is close to NIL. Thanks for update.

No worries, I always like to at least present the option, as sometimes people have expectations for how long a bug might take to fix. We will get to this issue eventually, thanks for using the extension!