search

LironEr / cypress-mochawesome-reporter

Zero config Mochawesome reporter for Cypress with screenshots and videos
MIT License
159 stars 49 forks source link

Add video only on failed Tests #141

Closed irapths closed 1 year ago

irapths commented 1 year ago

The last feature addition on v3.4.0 -adding video on tests- is very helpfull, even if the video is about the entire spec file.

A feature of Mockawesome Report is to add a marker appears on the right next to the elapsed time if any test, PASSed or FAILed, has Additional Text Content. A feature of Cypress Mockawesome Reporter is to mark the Additional Text Content entry of each try of a test with its "retry" index number.

In my company we have set retries: 1 and fix/improve any flaky test (test that rerun on fail and pass the second time) if is possible. In order to find the flaky ones in the report we "use" the "Additional Text Content" marker as "flaky marker".

With the last update the captured video is added to all tests. This makes the identification of "flaky/able-to-be-improved" tests a lot harder.

I don't think there is any need for the video to be added to the PASSed tests. I think it is better to be added only on FAILed ones (or FAILed tries of flaky ones). If needed and is possible, and since the video is the same for all tests, it can be added only on the 1st tests of the spec file.

joakim-sch commented 1 year ago

Hi @irapths and thank you for your feedback. I am glad you found the video attachments helpful 😄

The video is as you say the entire spec-file, but this is how Cypress designed their video recording feature and thus not something we can control. This fact is also why I deliberately made the video feature add the recording to both Passed and Failed tests as the theory was that it would be more confusing to have a full-spec video recording only added to some tests. By adding the same video to all tests within the spec, it clearly implied that the recording was off ALL tests in the spec, not just the failed one.

However, that being said I do see your point in having the video on Passed tests might not be desired for all setups.

I have made a PR (#142) that allows you to set the reporterOption videoOnFailOnly = true to only add the video recordings on the failed tests. I hope this will help 😃

irapths commented 1 year ago

Hi @joakim-sch

I know that there is no way to locate the timestamp each test case starts in order to start the video to that. And this is the reason I am OK even if the same video is on each test of the same scpec file.

As for the impruvement you did, I think the [de]activation on PASSed via flag is very good, since it is simple but alos meets any need.

LironEr commented 1 year ago

Released in v3.5.0