jest-community / vscode-jest

The optimal flow for Jest based testing in VS Code
MIT License
2.84k stars 295 forks source link

[BUG] Inline coverage using new VSC coverage api doesn't appear #1174

Closed karmeleon closed 2 months ago

karmeleon commented 2 months ago

Describe the bug Using the prerelease extension version (v6.3.0), I can use vscode-jest to run all the tests in a suite, and view the test coverage panel in the Testing view. This data is correct, but when I click on a file with incomplete coverage, VSCode's inline coverage feature doesn't display any coverage info despite clicking on "Show Inline Coverage" in the coverage toolbar.

To Reproduce Steps to reproduce the behavior:

  1. Run a Jest suite with incomplete coverage.
  2. Load a file with incomplete coverage in the main editor view.
  3. Observe that even with inline coverage enabled, no coverage info is visible on the file.

Note: A sample repo will help us identify the bug much faster. 🙏

Expected behavior VSCode's built-in coverage API highlights lines that lack coverage.

Screenshots image Not much to look at, but line 114 is lacking coverage here.

Environment (please complete the following information):

Prerequisite

Additional context Add any other context about the problem here.

If I revert back to the release version of the extension (v6.2.5), vscode-jest's homebrew coverage report works correctly with otherwise identical setup:

image


The fastest (and the most fun) way to resolve the issue is to submit a pull request yourself. If you are interested, please check out the contribution guide, we look forward to seeing your PR...

connectdotz commented 2 months ago

This is interesting, but I wasn't able to reproduce the issue on my Mac. A couple of things stand out from your screenshots:

  1. In your screenshots, each loop has a different color bar. I don't see that behavior in my repository. Are you using any additional coverage-related plugins or specific settings?
  2. The statement number gutter isn't highlighted in your snapshots, whereas it is highlighted in mine, as documented by VS Code:

image

Would you be able to provide a minimal repository that we could use to reproduce and further diagnose the issue?

karmeleon commented 2 months ago
  1. In your screenshots, each loop has a different color bar. I don't see that behavior in my repository. Are you using any additional coverage-related plugins or specific settings?

Nope, just VSC's bracket pair colorization feature and this extension :)

  1. The statement number gutter isn't highlighted in your snapshots, whereas it is highlighted in mine, as documented by VS Code:

Right, VSC doesn't have gutter highlighting either. The native coverage stuff actually did work once or twice, but when I came back to my editor after the holiday weekend it didn't work anymore.

I'm not able to provide a sample repo right now -- lots of proprietary stuff to strip out and I'm really just messing with this extension at this point, but is there some way I could use debug mode and the VSC dev tools to intercept calls to the API and see if something's not working as expected?

Thanks for taking a look!

karmeleon commented 2 months ago

Aha, I actually see a stacktrace in the VSC debugger when this fails to work:

[Extension Host] JestFileCoverage getDetailed failed for /path/to/the/source/file/being/viewed.tsx: Error: Invalid arguments
    at new $ (/nail/home/shawn/.vscode-server/cli/servers/Stable-fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/server/out/vs/workbench/api/node/extensionHostProcess.js:152:6717)
    at r (/nail/home/shawn/.vscode-server/extensions/orta.vscode-jest-6.3.0/out/extension.js:2:463810)
    at /nail/home/shawn/.vscode-server/extensions/orta.vscode-jest-6.3.0/out/extension.js:2:464103
    at Array.forEach (<anonymous>)
    at /nail/home/shawn/.vscode-server/extensions/orta.vscode-jest-6.3.0/out/extension.js:2:464066
    at Array.forEach (<anonymous>)
    at a.loadDetails (/nail/home/shawn/.vscode-server/extensions/orta.vscode-jest-6.3.0/out/extension.js:2:464035)
    at t.JestTestCoverageProvider.<anonymous> (/nail/home/shawn/.vscode-server/extensions/orta.vscode-jest-6.3.0/out/extension.js:2:465110)
    at Generator.next (<anonymous>)
    at /nail/home/shawn/.vscode-server/extensions/orta.vscode-jest-6.3.0/out/extension.js:2:463028
    at new Promise (<anonymous>)
    at r (/nail/home/shawn/.vscode-server/extensions/orta.vscode-jest-6.3.0/out/extension.js:2:462773)
    at t.JestTestCoverageProvider.loadDetailedCoverage (/nail/home/shawn/.vscode-server/extensions/orta.vscode-jest-6.3.0/out/extension.js:2:464942)
    at t.JestTestProvider.<anonymous> (/nail/home/shawn/.vscode-server/extensions/orta.vscode-jest-6.3.0/out/extension.js:2:484204)
    at Generator.next (<anonymous>)
    at /nail/home/shawn/.vscode-server/extensions/orta.vscode-jest-6.3.0/out/extension.js:2:483119
    at new Promise (<anonymous>)
    at r (/nail/home/shawn/.vscode-server/extensions/orta.vscode-jest-6.3.0/out/extension.js:2:482864)
    at W.loadDetailedCoverage (/nail/home/shawn/.vscode-server/extensions/orta.vscode-jest-6.3.0/out/extension.js:2:484141)
    at b.getCoverageDetails (/nail/home/shawn/.vscode-server/cli/servers/Stable-fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/server/out/vs/workbench/api/node/extensionHostProcess.js:156:24477)
    at v.getCoverageDetails (/nail/home/shawn/.vscode-server/cli/servers/Stable-fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/server/out/vs/workbench/api/node/extensionHostProcess.js:156:27280)
    at E.$getCoverageDetails (/nail/home/shawn/.vscode-server/cli/servers/Stable-fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/server/out/vs/workbench/api/node/extensionHostProcess.js:156:20185)
    at y.S (/nail/home/shawn/.vscode-server/cli/servers/Stable-fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/server/out/vs/workbench/api/node/extensionHostProcess.js:151:5980)
    at y.Q (/nail/home/shawn/.vscode-server/cli/servers/Stable-fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/server/out/vs/workbench/api/node/extensionHostProcess.js:151:5746)
    at y.M (/nail/home/shawn/.vscode-server/cli/servers/Stable-fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/server/out/vs/workbench/api/node/extensionHostProcess.js:151:4739)
    at y.L (/nail/home/shawn/.vscode-server/cli/servers/Stable-fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/server/out/vs/workbench/api/node/extensionHostProcess.js:151:3830)
    at i.value (/nail/home/shawn/.vscode-server/cli/servers/Stable-fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/server/out/vs/workbench/api/node/extensionHostProcess.js:151:2297)
    at n.B (/nail/home/shawn/.vscode-server/cli/servers/Stable-fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/server/out/vs/workbench/api/node/extensionHostProcess.js:83:737)
    at n.fire (/nail/home/shawn/.vscode-server/cli/servers/Stable-fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/server/out/vs/workbench/api/node/extensionHostProcess.js:83:954)
    at s.fire (/nail/home/shawn/.vscode-server/cli/servers/Stable-fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/server/out/vs/workbench/api/node/extensionHostProcess.js:97:14453)
    at i.value (/nail/home/shawn/.vscode-server/cli/servers/Stable-fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/server/out/vs/workbench/api/node/extensionHostProcess.js:177:8655)
    at n.B (/nail/home/shawn/.vscode-server/cli/servers/Stable-fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/server/out/vs/workbench/api/node/extensionHostProcess.js:83:737)
    at n.fire (/nail/home/shawn/.vscode-server/cli/servers/Stable-fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/server/out/vs/workbench/api/node/extensionHostProcess.js:83:954)
    at s.fire (/nail/home/shawn/.vscode-server/cli/servers/Stable-fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/server/out/vs/workbench/api/node/extensionHostProcess.js:97:14453)
    at d.A (/nail/home/shawn/.vscode-server/cli/servers/Stable-fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/server/out/vs/workbench/api/node/extensionHostProcess.js:97:17612)
    at i.value (/nail/home/shawn/.vscode-server/cli/servers/Stable-fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/server/out/vs/workbench/api/node/extensionHostProcess.js:97:15970)
    at n.B (/nail/home/shawn/.vscode-server/cli/servers/Stable-fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/server/out/vs/workbench/api/node/extensionHostProcess.js:83:737)
    at n.fire (/nail/home/shawn/.vscode-server/cli/servers/Stable-fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/server/out/vs/workbench/api/node/extensionHostProcess.js:83:954)
    at m.acceptChunk (/nail/home/shawn/.vscode-server/cli/servers/Stable-fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/server/out/vs/workbench/api/node/extensionHostProcess.js:97:12199)
    at /nail/home/shawn/.vscode-server/cli/servers/Stable-fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/server/out/vs/workbench/api/node/extensionHostProcess.js:97:11469
    at Socket.v (/nail/home/shawn/.vscode-server/cli/servers/Stable-fee1edb8d6d72a0ddff41e5f71a671c23ed924b9/server/out/vs/workbench/api/node/extensionHostProcess.js:174:14266)
    at Socket.emit (node:events:519:28)
    at addChunk (node:internal/streams/readable:559:12)
    at readableAddChunkPushByteMode (node:internal/streams/readable:510:3)
    at Socket.Readable.push (node:internal/streams/readable:390:5)
    at Pipe.onStreamRead (node:internal/stream_base_commons:191:23)

I believe that maps to this line in the source.

EDIT: I can see coverage if I use coverageProvider: 'v8' in my Jest config file, but for my setup only babel provides accurate coverage data.

karmeleon commented 2 months ago

Upon further investigation, it appears that some files have working coverage, mostly the simple ones. Here's an example file that fails to render coverage, even though it's simple:

export default function getPaddingClass(side: string, units?: string | number | null): string {
    if (!units) {
        return '';
    }

    const unitsLabel = units === 'half' ? '-half' : String(units);
    const sideLabel = side[0];
    return `u-padding-${sideLabel}${unitsLabel}`;
}
Coverage JSON blob generated with Babel coverage provider (does not display in VSC) ```json "/path/to/get-padding-class.ts": { "path": "/path/to/get-padding-class.ts", "statementMap": { "0": { "start": { "line": 2, "column": 4 }, "end": { "line": 4, "column": 5 } }, "1": { "start": { "line": 3, "column": 8 }, "end": { "line": 3, "column": 18 } }, "2": { "start": { "line": 6, "column": 23 }, "end": { "line": 6, "column": 65 } }, "3": { "start": { "line": 7, "column": 22 }, "end": { "line": 7, "column": 29 } }, "4": { "start": { "line": 8, "column": 4 }, "end": { "line": 8, "column": 49 } } }, "fnMap": { "0": { "name": "getPaddingClass", "decl": { "start": { "line": 1, "column": 24 }, "end": { "line": 1, "column": 39 } }, "loc": { "start": { "line": 1, "column": 94 }, "end": { "line": 9, "column": 1 } }, "line": 1 } }, "branchMap": { "0": { "loc": { "start": { "line": 2, "column": 4 }, "end": { "line": 4, "column": 5 } }, "type": "if", "locations": [ { "start": { "line": 2, "column": 4 }, "end": { "line": 4, "column": 5 } }, { "start": {}, "end": {} } ], "line": 2 }, "1": { "loc": { "start": { "line": 6, "column": 23 }, "end": { "line": 6, "column": 65 } }, "type": "cond-expr", "locations": [ { "start": { "line": 6, "column": 42 }, "end": { "line": 6, "column": 49 } }, { "start": { "line": 6, "column": 52 }, "end": { "line": 6, "column": 65 } } ], "line": 6 } }, "s": { "0": 77, "1": 69, "2": 8, "3": 8, "4": 8 }, "f": { "0": 77 }, "b": { "0": [ 69, 8 ], "1": [ 2, 6 ] }, "_coverageSchema": "1a1c01bbd47fc00a2c39e90264f33305004495a9", "hash": "f60631d78e4a8d06a4ee22ec337399c1016f9f88" }, ```
Coverage JSON blob generated with V8 coverage provider (displays correctly in VSC) ```json "/path/to/get-padding-class.ts": { "path": "/path/to/get-padding-class.ts", "all": false, "statementMap": { "0": { "start": { "line": 1, "column": 0 }, "end": { "line": 1, "column": 95 } }, "1": { "start": { "line": 2, "column": 0 }, "end": { "line": 2, "column": 17 } }, "2": { "start": { "line": 3, "column": 0 }, "end": { "line": 3, "column": 18 } }, "3": { "start": { "line": 4, "column": 0 }, "end": { "line": 4, "column": 5 } }, "4": { "start": { "line": 5, "column": 0 }, "end": { "line": 5, "column": 0 } }, "5": { "start": { "line": 6, "column": 0 }, "end": { "line": 6, "column": 66 } }, "6": { "start": { "line": 7, "column": 0 }, "end": { "line": 7, "column": 30 } }, "7": { "start": { "line": 8, "column": 0 }, "end": { "line": 8, "column": 49 } }, "8": { "start": { "line": 9, "column": 0 }, "end": { "line": 9, "column": 1 } } }, "s": { "0": 1, "1": 77, "2": 69, "3": 69, "4": 8, "5": 77, "6": 77, "7": 77, "8": 77 }, "branchMap": { "0": { "type": "branch", "line": 1, "loc": { "start": { "line": 1, "column": 15 }, "end": { "line": 9, "column": 1 } }, "locations": [ { "start": { "line": 1, "column": 15 }, "end": { "line": 9, "column": 1 } } ] }, "1": { "type": "branch", "line": 2, "loc": { "start": { "line": 2, "column": 16 }, "end": { "line": 4, "column": 5 } }, "locations": [ { "start": { "line": 2, "column": 16 }, "end": { "line": 4, "column": 5 } } ] }, "2": { "type": "branch", "line": 4, "loc": { "start": { "line": 4, "column": 4 }, "end": { "line": 6, "column": 42 } }, "locations": [ { "start": { "line": 4, "column": 4 }, "end": { "line": 6, "column": 42 } } ] }, "3": { "type": "branch", "line": 6, "loc": { "start": { "line": 6, "column": 39 }, "end": { "line": 6, "column": 49 } }, "locations": [ { "start": { "line": 6, "column": 39 }, "end": { "line": 6, "column": 49 } } ] }, "4": { "type": "branch", "line": 6, "loc": { "start": { "line": 6, "column": 49 }, "end": { "line": 6, "column": 65 } }, "locations": [ { "start": { "line": 6, "column": 49 }, "end": { "line": 6, "column": 65 } } ] } }, "b": { "0": [ 77 ], "1": [ 69 ], "2": [ 8 ], "3": [ 2 ], "4": [ 6 ] }, "fnMap": { "0": { "name": "getPaddingClass", "decl": { "start": { "line": 1, "column": 15 }, "end": { "line": 9, "column": 1 } }, "loc": { "start": { "line": 1, "column": 15 }, "end": { "line": 9, "column": 1 } }, "line": 1 } }, "f": { "0": 77 } }, ```

I do see that the Babel JSON has an empty-ish block that V8 doesn't, maybe that's the culprit?

            "type": "if",
                "locations": [
                    {
                        "start": {
                            "line": 2,
                            "column": 4
                        },
                        "end": {
                            "line": 4,
                            "column": 5
                        }
                    },
                    {
                        "start": {},
                        "end": {}
                    }
                ],
                "line": 2
connectdotz commented 2 months ago

@karmeleon, this is tremendously helpful, thanks. I will take a closer look later today.

connectdotz commented 2 months ago

@karmeleon, I think I found the issue. Before I release the fix, could you verify it with the attached vsix file? Thanks. vscode-jest-6.3.1.vsix.zip

karmeleon commented 2 months ago

image

Confirmed, works perfectly both in the above file as well as a few other spot checked files across the repo. Thanks so much for the quick fix!