microsoft / vscode-java-test

Run and debug Java test cases in Visual Studio Code.
https://marketplace.visualstudio.com/items?itemName=vscjava.vscode-java-test
Other
289 stars 123 forks source link

Code coverage: execution count not correct #1698

Open pcarterubc opened 3 weeks ago

pcarterubc commented 3 weeks ago

Version: 1.90.0 (Universal) Commit: 89de5a8d4d6205e5b11647eb6a74844ca23d2573 Date: 2024-06-04T19:34:44.157Z Electron: 29.4.0 ElectronBuildId: 9593362 Chromium: 122.0.6261.156 Node.js: 20.9.0 V8: 12.2.281.27-electron.0 OS: Darwin x64 23.5.0 Test Runner for Java v0.41.1

Steps to reproduce:

Example: hovering over the "6x" decoration on line 41, the pop-up indicates the statement was executed 6 times:

Screenshot 2024-06-12 at 12 31 47 PM

I believe it is displaying the instruction count for line 41 found in the Jacoco report:

Screenshot 2024-06-11 at 5 28 20 PM

In any case, it would be great to have the option to turn off these decorations.

jdneo commented 3 weeks ago

In any case, it would be great to have the option to turn off these decorations.

Are you using insider vscode or stable?

Yes, it's displaying the ci as the count. Is that wrong in this case?

pcarterubc commented 3 weeks ago

I am using the stable release.

If the intention is to display the ci, the tooltip needs to be updated to reflect this. On line 41 in the example above, the tooltip text is "y = y + DY; was executed 6 times" but it should say something like "Instruction count: 6". I believe the ci is a count of the number of Java byte code instructions that were executed on the corresponding line, not a count of the number of times the Java statement was executed by the tests. (https://www.jacoco.org/jacoco/trunk/doc/counters.html)

jdneo commented 3 weeks ago

In stable, you can hover on the colored line number, wait for 1 second, you will see the toggle action to close the inline decoration.

If the intention is to display the ci, the tooltip needs to be updated to reflect this.

Yes, looks there is some gap here. The API from VS Code only have line/statement coverage, which is not fully equal to the ci.