Show test results as soon as they occur

[ffMathy]

It would be awesome if the extension would not wait until all tests have run to show the status of individual tests.

This especially happens if the tests are in the same file.

[connectdotz]

This extension processes test results from a json file generated by jest after it runs all the tests in the given run. Changing it would mean the extension needs to parse the output for test results, errors, coverage reporting, etc., which is not trivial and would be fragile whenever jest changes its output messaging. Jest created the stable json file precisely to spare the tooling systems from such pain...

Having said that, the delay should not be too noticeable... unless there is a bug? Does this happen more noticeably in certain files or situations?

[ffMathy]

It's not the extension's fault. I am running some integration tests that spin up a lot of stuff, which just takes time.

However, would it be possible somehow to hook into the reporters feature of Jest or something similar? There must be some way to get the status while it's running.

Maybe this can help? https://github.com/jestjs/jest/issues/5292

[ffMathy]

I just confirmed that Jest reporters can get the status after each test. So it should be doable for the reporter to communicate back.

[ffMathy]

Here's a sample reporter that can get the results of tests as they occur.

https://github.com/ryparker/jest-reporter-template/blob/master/src/reporter.ts

I really think this could be useful. Also for Playwright tests etc which may also take some time. Currently when you have hundreds of tests, it is super useful.

[ffMathy]

Using this approach, it wouldn't be fragile. In fact, it would be officially supported.

I'm even willing to do a PR on this if you can point me to where and how you'd approach this 🙏

[connectdotz]

Indeed, this is promising! We already have a reporter that was included in all test run; you can start from there.

The most challenging part, I think, is to match the assertion to the source code blocks (parsed by @babel/parser in jest-editor-support), because the location info do not always match, and the title match also will not match for dynamic title like test.each. So we used something we called "context match" (match-by-context.ts) that combines with multiple match mechanisms to provide a more robust match. In processing one test block at a time, we might not have the full context, which will most likely fail the match logic and require some non-trivial fixes...

The way we notify the testing system (for visual update) is done in test-item-date.ts, you can take a look at the event processing there to get a sense how things wire together.

I think full feature parity with the current process might be pretty involved... But do feel free to take a look of the code and let me know what you think and we can take from there.

[ffMathy]

That's great. I'll see what I can do. But either way, let's leave this issue open until it is fixed. Not sure I'll get the time.

[ffMathy]

I got it working! Just need to clean it up now. Pull request coming soon! 🥳

[ffMathy]

May I ask... Is there a reason that Runner.js, Settings.js, Snapshot.js and test_reconciler.js are not in TypeScript?

[ffMathy]

@connectdotz the PR is over at #1075!

Let me know what you think. It currently breaks some tests, and I could use some assistance figuring out why. But the feature works.

[lonevvolf]

This would be a really great feature to have - can be a big time saver for large suites of tests.