Cannot visualize manual coverage generation

stdedos commented 1 year ago

Describe the bug

I am using Codium https://github.com/VSCodium/vscodium and Go / https://github.com/golang/vscode-go in my project (private).

https://github.com/golang/vscode-go can generate one-shot coverages for tests / packages etc, but I want to run specific tests (let's call them light integration tests) and generate coverage for them:

$ go install github.com/jstemmer/go-junit-report/v2@v2.0.0
$ go install github.com/boumenot/gocover-cobertura@v1.2
$ go test -coverprofile="$COVERAGE_FILE" -coverpkg=./... ./... 2>&1 | tee >(go-junit-report -set-exit-code -out "$TEST_REPORT")
$ gocover-cobertura < "$COVERAGE_FILE" > "$COBERTURA_REPORT"
$ head "$COBERTURA_REPORT"
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE coverage SYSTEM "http://cobertura.sourceforge.net/xml/coverage-04.dtd">
<coverage line-rate="0.5702541" branch-rate="0" version="" timestamp="1668431560233" lines-covered="4578" lines-valid="8028" branches-covered="0" branches-valid="0" complexity="0">
  <sources>
    <source>-path-/-proj-/git-submodule</source>
    <source>-path-/-proj-</source>
  </sources>
  <packages>
    <package name="a/b/git-submodule/d" line-rate="0.50127876" branch-rate="0" complexity="0">
      <classes>
$

and then visualize those.

"Nothing seems to work":

Enabling / Disabling coverage
Renaming the file with "one of the default picked-up file names"
Adding the generating filename to the "picked-up files"
Disabling the go extension (maybe the gutter/coverage system can only be hooked by one actor?)

Extensions' console logs "seem to do work".

Any ideas how can I troubleshoot my issue?

To Reproduce Steps to reproduce the behaviour:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behaviour A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

OS: [e.g. windows, linux, macos]
Extension Version [e.g. 2.0.0]
VSCode Version [e.g. 1.21.0]

Additional context Add any other context about the problem here.

stdedos commented 1 year ago

I also have output format like:

mode: set
-proj-/1/2/3/4/5.go:32.99,34.26 2 0
...

... and I don't know what lcov format is (I cannot find it online or an example file), and idk if the extension can read multiple formats, and whether my files are parsed and "stored internally" correctly or not :sweat:

ryanluker commented 1 year ago

@stdedos thanks for the issue / question! The default golang code coverage format is not compatible with one of the support coverage format types, afaik. https://github.com/ryanluker/vscode-coverage-gutters/blob/master/src/files/coveragefile.ts#L3-L6

Looks like you should be able to covert to other formats that are supported though (see below). https://stackoverflow.com/a/31440740

stdedos commented 1 year ago

The default golang code coverage format is not compatible with one of the support coverage format types, afaik. master/src/files/coveragefile.ts#L3-L6

Could it be that there's a message in the extension log for reading unsupported format and failing? :confused:

Here are the last lines of the extension's log:

[1668504730779][coverageservice]: INITIALIZING
[1668505343076][coverageservice]: LOADING
[1668505345080][coverageservice]: Loading 2 file(s)
[1668505345080][coverageservice]: -proj-/test_reports/coverage.txt,-proj-/test_reports/cobertura_report.xml
[1668505345093][coverageservice]: Loaded 2 data file(s)
[1668505362017][coverageservice]: LOADING
[1668505364014][coverageservice]: Loading 2 file(s)
[1668505364014][coverageservice]: -proj-/test_reports/coverage.txt,-proj-/cov.xml
[1668505364019][displayCoverageForActiveFile]: ENOENT: no such file or directory, open '-proj-/cov.xml'
[1668505364019][displayCoverageForActiveFile]: Error: ENOENT: no such file or directory, open '-proj-/cov.xml'
[1668505372660][coverageservice]: LOADING
[1668505375112][coverageservice]: Loading 1 file(s)
[1668505375112][coverageservice]: -proj-/test_reports/coverage.txt
[1668505375115][coverageservice]: Loaded 1 data file(s)

"Does this mean it's working, or not?" :thinking:

Looks like you should be able to covert to other formats that are supported though (see below). stackoverflow.com/a/31440740

I have already the look-alike file:

$ gocover-cobertura < "$COVERAGE_FILE" > "$COBERTURA_REPORT"
$ head "$COBERTURA_REPORT"
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE coverage SYSTEM "http://cobertura.sourceforge.net/xml/coverage-04.dtd">
<coverage line-rate="0.5702541" branch-rate="0" version="" timestamp="1668431560233" lines-covered="4578" lines-valid="8028" branches-covered="0" branches-valid="0" complexity="0">
  <sources>
    <source>-path-/-proj-/git-submodule</source>
    <source>-path-/-proj-</source>
  </sources>
  <packages>
    <package name="a/b/git-submodule/d" line-rate="0.50127876" branch-rate="0" complexity="0">
      <classes>
$

So I don't understand why doesn't work :confused:

On a tangent, there is a weird situation that, gocover-cobertura < "$COVERAGE_FILE" > "$COBERTURA_REPORT" and gocov convert go-code-cover | gocov-xml >| coverage.xml do not give byte-for-byte similar inputs (but they look comparable, and they reference the same schema <!DOCTYPE coverage SYSTEM "http://cobertura.sourceforge.net/xml/coverage-04.dtd">) - and hence that re-enforces my question "Does vscode-coverage-gutters successfully parse that file?".

I do see coverage data for the file that interests me, and for some reason, they are not rendered.

ryanluker commented 1 year ago

Thanks for the extra info, normally the debug logs look like this... so there must be an issue with the source paths in the cobertura report file 🤔.

[1668970981951][coverageservice]: INITIALIZING
[1668970998653][coverageservice]: LOADING
[1668970998732][coverageservice]: Loading 6 file(s)
[1668970998732][coverageservice]: c:\Users\ryanl\dev\vscode-coverage-gutters\example\node\lcov.info,c:\Users\ryanl\dev\vscode-coverage-gutters\example\remote-node\lcov.info,c:\Users\ryanl\dev\vscode-coverage-gutters\example\ruby\lcov.info,c:\Users\ryanl\dev\vscode-coverage-gutters\example\java\my-app\cov.xml,c:\Users\ryanl\dev\vscode-coverage-gutters\example\php\cov.xml,c:\Users\ryanl\dev\vscode-coverage-gutters\example\python\cov.xml
[1668970998735][coverageservice]: Loaded 6 data file(s)
[1668970998759][coverageservice]: Caching 17 coverage(s)
[1668970998759][coverageservice]: READY
[1668970998759][coverageservice]: RENDERING
[1668970998759][renderer][section file path]: ./node/test-coverage.js
[1668970998759][renderer][section file path]: ./node/test-coverage.js
[1668970998760][coverageservice]: READY
[1668970998760][coverageservice]: Listening to file system at {c:\Users\ryanl\dev\vscode-coverage-gutters\example}/**/{lcov.info,cov.xml,coverage.xml,jacoco.xml,coverage.cobertura.xml}

The key output here would be the caching line which signifies that coverages were found in the files and now are stored in memory. [1668970998759][coverageservice]: Caching 17 coverage(s)

Make sure that you are opening the project / folder from root or where-ever the cobertura file has its root at, or otherwise the extension might not be able to see the files properly.

stdedos commented 1 year ago

[1668970998759][coverageservice]: Caching 17 coverage(s)

I have managed to see this line "at some point", although I don't remember exactly when.

IMHO, an "Invalid coverage file format" log line would be very worth it. Even if you want to have it on debug/trace level.

Make sure that you are opening the project / folder from root or where-ever the cobertura file has its root at

IIRC, coverage files contain absolute paths 😕 Codium is "normally" opened in project mode for the project we are talking about. Originally, I have opened those projects with codium . (inside project top-level), but nowadays "I just open Codium" (hot exit takes care of re-opening my stuff). I also have 3 projects opened at the same time

stdedos commented 1 year ago

[1668970998759][coverageservice]: Caching 17 coverage(s)
I have managed to see this line "at some point", although I don't remember exactly when.

So basically I have:

[1669028337981][coverageservice]: INITIALIZING
[1669028802913][coverageservice]: LOADING
[1669028804923][coverageservice]: Loading 1 file(s)
[1669028804923][coverageservice]: -proj-/test_reports/cobertura_report.xml
[1669028804925][coverageservice]: Loaded 1 data file(s)
[1669028805348][coverageservice]: Caching 141 coverage(s)
[1669028805348][coverageservice]: READY
[1669028805348][coverageservice]: RENDERING
[1669028805356][coverageservice]: READY

... and still rendering does not happen.

I have re-activated Preview manually (via shortcut), soft (Developer: Reload Codium) and hard reloaded Codium. No change.

ryanluker commented 1 year ago

@stdedos Thanks for the additional logs! I will do a bit of investigation next time I get back to this project, but from what you have posted, it looks like it should be working 🤔.

One other area you could try, is tinkering with these 3 settings that deal with the file matching between the coverages and the actual files you open. https://github.com/ryanluker/vscode-coverage-gutters/blob/master/package.json#L112-L126

ryanluker commented 1 year ago

Closing due to staleness but definitely willing to reopen!

ryanluker / vscode-coverage-gutters

Cannot visualize manual coverage generation #386