avajs / ava

Node.js test runner that lets you develop with confidence 🚀
MIT License
20.74k stars 1.41k forks source link

Make it easier to find failed tests in the default reporter #2641

Closed stavalfi closed 3 years ago

stavalfi commented 3 years ago

Background

I'm migrating a Jest project to Ava and I find it hard to search the failed tests in Ava reporter.

With jest, you can search in the terminal "fail " (with a space at the end) and you will find all the failed tests files which failed. (it won't help you to find the actual tests which failed).

With Ava, I need to search for: "✖" symbol to find the actual tests which failed (but it won't help you to find the failed test files).


Problem

In a project with 20+- tests it can be ok. but in my project I have 150+ tests and each one has a lot of logging (with t.log). so the test output is too long to find what I'm looking for.


Feature Request

At the end of the in the default reporter, list the following:

  1. a list of all failed test files.
  2. for each failed test-file, list all the failed tests titles.

Possible workaround

working locally with custom tap reporters is annoying because I need to pipe every command with them every time.


What to you think?

novemberborn commented 3 years ago

Sure. Would you mind adding this to https://github.com/avajs/ava/issues/2500?

stavalfi commented 3 years ago

Thanks @novemberborn.

from my understanding, this request won't be done in the next months. is there any workaround I can do to make it happen? different Ava reporter maybe?

It's unproductive to not knowing which tests are failing and which files are failing immediately.

thanks again for taking the time to answer me.

novemberborn commented 3 years ago

Nah, all we have is TAP and our default reporter, sorry.

tymfear commented 3 years ago

@novemberborn at this point I would like to ask again about this one #2583 . It should be easy to implement (I guess at least easier than #2608 ) and this will allow users to create their own reporters in the whatever way they want to.

novemberborn commented 3 years ago

@tymfear I think our energy is better spent elsewhere.

tymfear commented 3 years ago

...Just checked State of JS 2020 and I'm sad seeing people leave AVA or don't like it. I'm sure they just don't understand all the power of AVA and super used to Mocha/Jasmine/Jest (the same I see in my org with some people). But some people want good reporting, and what currently AVA provides is not that good and that makes people leave as well. And I really believe that improving reporting will make more people chose AVA, as otherwise it is hard to build good integration or E2E harness around it.

stavalfi commented 3 years ago

...Just checked State of JS 2020 and I'm sad seeing people leave AVA or don't like it. I'm sure they just don't understand all the power of AVA and super used to Mocha/Jasmine/Jest (the same I see in my org with some people). But some people want good reporting, and what currently AVA provides is not that good and that makes people leave as well. And I really believe that improving reporting will make more people chose AVA, as otherwise it is hard to build good integration or E2E harness around it.

I agree, I can't suggest my Org to use Ava as well yet. Until there is a better reporting.

tymfear commented 3 years ago

I do use AVA in my organisation, and advertise it, and show all the benefits, but every time someone new sees AVA they are confused by macros and TAP reports, so that needs clarifications. But because of that I assume the same reactions across JS community.

Actually TAP is ok for analysis, but in Allure you can attach screenshots, for example, or videos. So overall Allure of course is much better than TAP, so if we just have hooks for injecting Allure, that would be extremely beneficial.

novemberborn commented 3 years ago

Would be great to have a Discussion about this!

tymfear commented 3 years ago

I'm not sure what do you mean by this. But I'll create discussion about hooks.

novemberborn commented 3 years ago

About the points you're raising. Just suggesting we move it out of an issue thread and into the Discussion space 😄

tymfear commented 3 years ago

Done