cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
47.03k stars 3.19k forks source link

Feature Request: Retry failed tests after the entire test suite has ran instead of retrying instantly #15585

Open manovotny opened 3 years ago

manovotny commented 3 years ago

What would you like?

It'd be awesome if the retrying of failed tests would happen after the entire test suite has ran instead of retrying failed tests instantly.

Why is this needed?

If there is a connection / network blip or flakiness with an internal or external API or some other integrated dependency causing a test to fail, retrying the test immediately will likely result in the test failing again as it tries again so quickly.

It would be nice to delay or postpone the retries until the end after the rest of the test suite has had a chance to run. This would allow more time in between test run attempts, giving more time for the blips and kinks to work themselves out instead of expecting those things to work themselves out nearly immediately.

abhidp commented 3 years ago

Workaround: hi @manovotny, I also needed something like this. However, there is workaround for this if you are using cypress.run() to run your tests. You can get a list of all the failures after the suite has ran and then re-run them. I had raised a bug few days back and @jennifer-shehane had come up with a solution on how to get a list of the failed specs. It is also mentioned in the Cypress module API documentation.

https://github.com/cypress-io/cypress/issues/15443

vaclavGabriel commented 2 years ago

Hello @abhidp, may you be able to share your workaround, please?

Pverma96 commented 1 year ago

@abhidp did the workaround work for you? Can you please share some details?