enhancement request: option to re-run failing tests <n> number of times

[mam-p]

Many GUI test frameworks support an option to re-run failing tests a specified number of times. For example, I'd like to see this command:

nightwatch -s FAILING -a regression --rerun 3

give every failing test TWO more tries to pass.

[lacymorrow]

+1 for this. A huge number of users need this due to browser instability (especially IE).

I'm working on a hacky way of accomplishing this with the current version of nightwatch, I'll post my results if I have success.

[beatfactor]

interesting...

On Wed, Oct 1, 2014 at 2:55 AM, Lacy Morrow notifications@github.com wrote:

+1 for this. A huge number of users need this due to browser instability (especially IE).

I'm working on a hacky way of accomplishing this with the current version of nightwatch, I'll post my results if I have success.

— Reply to this email directly or view it on GitHub https://github.com/beatfactor/nightwatch/issues/284#issuecomment-57405456 .

[chan-dra]

+1 for this. We use nightwatch to run our end2end tests, but it is starting to become a nightmare to see so many false positives, which would go away after a rerun.

[maxbeatty]

I work with Chandra and would love to have automatic retries. If anyone has ideas on how to make it work, we could probably take a stab at it. =

[rcherny]

+1 Yeah ok, this would be very useful. Could also resolve issues such as #252

[didemacet]

+1.

[didemacet]

@lacymorrow do you have any updates? If not, can you tell me how can I do that. I need this because I am totally agree with chan-dra's comment.

[yn5]

+1

[codemonkeybot]

+1

[andymost]

+1

[ghost]

:+1:

[ghost]

Here's similar example from other JS framework: http://galenframework.com/docs/reference-galen-javascript-api/#retry

[sknopf]

+1

[sknopf]

I have a solution here (https://github.com/beatfactor/nightwatch/pull/436) if anyone wants to give it a try.

If testing in an existing project, just reference this branch in your package.json file:

"nightwatch": "git://github.com/sknopf/nightwatch.git#add_flag_for_n_retries"

To use, add --retries <n> as a command line arg.

[sknopf]

As of 0.7 there is a retries CL option to rerun failed testcases: http://nightwatchjs.org/guide#command-line-options

[IndraniBiswas]

is there going to be retry option that can be specified in nightwatch.json config file?

[jenni]

+1

[quantuminformation]

@IndraniBiswas thats a good idea. Any news?

[AlfredoMoretta]

+1

[samtbabu]

+1

[mari8i]

+1 also for me!

This would save me at the moment, actually :-)

[HereticalCrown]

I don't know if people know this or not but you can add "--retries " at the end of the tests in your package files. This will then re-rerun your failed tests immediately after they fail.

I am currently struggling with this specifically because we are using the workers config in conjunction with these changes to speed up our run time. Unfortunately due to the amount of workers we (4 currently) it gets itself into a bit of a tiz and fails some of the tests. Once the run has finished though and we re-run them manually they pass. Is there a setting or config for the retries that dictates when to re-run the tests?

[islacruse]

Is there going to be retry option that can be specified in nightwatch.json config file?

It would be great!