Failed test error messages should be printed at the end of the output

[Nick-Mazuk]

What is the problem this feature would solve?

When a test fails, I want to be able to identify the failure as quickly as possible to fix it. Currently when a test errors, the error message is printed inline with the output whenever the test runner decides to run the test.

As a result, in a codebase with hundreds of tests (and growing), I can quickly see when a test fails but not which test fails. To find which test that fails, I must scroll upwards.

I've worked in codebases with tens of thousands of tests, which makes this workflow unpracticle.

What is the feature you are proposing to solve the problem?

By printing all test errors at the bottom of the output, I can quickly view which tests are failing and why they're failing regardless of the number of tests in the repository.

What alternatives have you considered?

No response

[simylein]

Any progress on this? This would be super helpful when working on larger test suites.

[jakeboone02]

Copying https://github.com/oven-sh/bun/issues/3342#issuecomment-1602990605 here as it's worth seeing the difference:

Jest shows the pass/fail result for each test inline, but batches the logging of all the verbose stuff (Expected/Received/etc.) at the end of the output. bun test puts the verbose failure explanations interspersed with the one-line pass/fail logs. Comparison of the same test suite as above, with a couple of intentional failures:

With Jest's format it's much easier to see everything that needs to be addressed in one place without much scrolling, if any.

[simylein]

Any updates on this?

[jakeboone02]

@simylein This is partially implemented already, but in a weird way. It lists the names of any .skip'd, .todo'd, and failed tests at the end of all the results, but only if more than 20 tests pass. Relevant code:

https://github.com/oven-sh/bun/blob/c2a5451e93bae70d25012bba3aeac51f0756df63/src/cli/test_command.zig#L926-L958

So, for example, if you have 20 tests where 15 pass and 5 fail, the names won't be printed because less than 20 passed. But if you have 22 tests and only 1 fails, the failing test will be listed at the end because more than 20 passed.

[jakeboone02]

Another thing I just noticed: the changes in #12189 (which I assume will be included in v1.1.19) make the output less useful because the describe scope names are not printed in front of the test names. This can probably be addressed at the same time as #12378.

[simylein]

@simylein This is partially implemented already, but in a weird way. It lists the names of any .skip'd, .todo'd, and failed tests at the end of all the results, but only if more than 20 tests pass. Relevant code:

https://github.com/oven-sh/bun/blob/c2a5451e93bae70d25012bba3aeac51f0756df63/src/cli/test_command.zig#L926-L958

So, for example, if you have 20 tests where 15 pass and 5 fail, the names won't be printed because less than 20 passed. But if you have 22 tests and only 1 fails, the failing test will be listed at the end because more than 20 passed.

Yes, I noticed that failed tests get printed last as I am often working in a code base with 400+ unit tests. It's more the scrolling up to find the toEqual diffs which is annoying, since the test output of failed tests itself does not get printed last.