Failed test output difficult to read when ExUnit running with `async: true`

[lpil]

Hello!

It would be nice if we could run tests asynchronously while keeping the debug information from failed tests readable.

Could we use a custom ExUnit formatter for this?

Cheers, Louis

[parroty]

Thanks for the comment. I'm not so familiar with the formatter you mentioned. Do you have any reference or examples? One not is that it basically uses the outputs of triq, so not sure if the formatting (capturing and reformatting) is feasible.

[lpil]

I must admit I'm not overly familiar with them either. The only one I've stumbled across in the wild is the one from the shouldi testing library.

[parroty]

Thanks. I don't have much time during the weekdays, but may try to check over the weekend. PRs are welcomed too :smile:.

[lpil]

Thank you. It's probably a bit beyond my Elixir understanding at the moment, but I'll also take a look if I have some free time.

[luc-tielen]

Might also be worth looking into changing group leader process to manipulate the output from triq since it uses io:format to output all information? Will try to look into this when I find some time.

[parroty]

I was wondering that having some hook (ex. ExUnit.CaptureIO) and printing them at appropriate timing could be one option. Also, having a leader process might make it easier to control the overall display too.

[luc-tielen]

I almost have a temporary solution ready, I created a new process that is set to the group leader process before every test. The process receives all test output and can manipulate it before forwarding to the original group leader.. I haven't implemented "the output per test behavior" yet but shouldn't be too difficult (store errors in an associative container in the new process and print in a on_exit callback / :test_suite_finished in custom formatter module?) Should also be possible to filter out the "Ran xxx tests" and add the number to the ExUnit total by combining this solution with the formatter solution proposed above.

Here's a quick preview of a part of what my test output looks like currently when I run mix test in this test project (note the coloring):

Thoughts?

[parroty]

Thanks for the PR! I understood that this PR gathers the outputs from triq by changing process_leader of each test execution processes, and then parse and formats the result nicely (coloring and counting number of property tests).

@lpil Does this address your original request?

[lpil]

Looks good! :)

[parroty]

Thanks. I've pushed as v0.3.0. If there's any remaining issues, please reopen or open another one.