Feature: Provide instant visual test feedback on dev code

[Gorthog]

Similar to Visual Studio Live Testing feature, where each line of code provides red / green status:

Green if line is covered by a passing test, and red if a test covering this line currently fails.

Also, when user clicks red mark, they can go to failing test(s).

I would like to champion this feature, but would love some feedback from maintainers: Do you see any problems with this feature that may prevent you from accepting it?

Also, if possible, who can I talk to discuss best approach to enhance coverage data - i.e. which test is covering which line?

[connectdotz]

@sinapis I think this would be a great feature.

The UI part should be pretty easy, you can take a look at the Coverage/Formatters to see how the current formatters are implemented.

Click-to-open-test is a powerful feature, however, I am not sure if the current istanbul based coverage map supports such a feature. Worst case you can consider cross-reference with TestResult...

Hopefully, this is sufficient to get you started, we can certainly discuss more detail in the PR...

[Gorthog]

@connectdotz thanks for the quick reply! I'll do my research and let you know if I have any more questions.

[Gorthog]

@connectdotz so I did some research and found that seems you were right about istanbul not supporting this feature. Currently nyc does not populate the TN (test name) field name in lcov file, and there does not seem to be any way to configure it to do so. I looked into TestResult, but IIRC, the position currently reported is of the test case, not which lines it covers in dev code... Am I correct about this? If it is, the best approach I can think of is to run istanbul separately for each test, and save coverage separately. Ugly, I know. Any better ideas?

[connectdotz]

we are already generating coverage map, see src/Coverage, specifically CoverageCodeLensProvider.ts and CoverageMapProvider.ts. Will you be able to use those?

[Gorthog]

This is good but not granular enough, as the coverage map is per file, and I need per test. This is not a problem that can be fixed by examining the current coverage map, because the required info (which test stepped where) is missing from the coverage map. I'm looking for a way to add it. Hence my proposal - run a single test each time, and store its coverage separately, by test.

Btw, I'm having trouble setting the project up so I'm analyzing the code without debugging. I don't have the folder jest/packages/jest-editor-support and therefore this part of the setup fails:

# link jest-editor-support
cd packages/jest-editor-support
yarn link

Are the instructions updated?

[connectdotz]

run a single test each time, and store its coverage separately, by test.

This will most likely cause a significant structural change for this extension, not to mention potential performance hit. I think we should look for other alternatives.

I wonder how Istanbul generates the coverage map? Is it possible to inject test info when it is instrumenting the code? Maybe it's worthy to post these questions there...

I don't have the folder jest/packages/jest-editor-support and therefore this part of the setup fails:

You don't need to link with jest-editor-support unless you want to change the code there.

[Gorthog]

I wonder how Istanbul generates the coverage map? Is it possible to inject test info when it is instrumenting the code? Maybe it's worthy to post these questions there...

I asked. They said it can't be done. You can read answer here: https://github.com/istanbuljs/nyc/issues/633

[Gorthog]

Also checked node built in code coverage feature, I added a feature request. Additional thought is to use findRelatedTests. It's not as sexy, since we will not be able to provide per line status, but at least we can show the user which tests are failing that are related to the current file opened in the editor.