search

anotherchrisberry / spock-retry

Spock extension to allow test features and specifications to be retried before failed
MIT License
34 stars 15 forks source link

Specs with @Stepwise fail as retry all tests in spec, only executes failed tests #19

Open deepaksanem opened 8 years ago

deepaksanem commented 8 years ago

I like this extension a lot and we use it in our daily testing but we had a strange situation where we had to execute tests sequentially so, we used @Stepwise to accomplish that.

We had one test which was failing inside spec with @Stepwise and retry executed that specific test. In this situation I would expect @RetryOnFailure to execute all tests inside that spec instead of only failed tests as it is annotated with @Stepwise

anotherchrisberry commented 8 years ago

That makes sense. I am on vacation but will take a look at this next week.

anotherchrisberry commented 8 years ago

Unfortunately, I don't see an easy way to re-execution previous specs. I'll leave this open; maybe someone can figure out a way to handle this.

david-w-millar commented 8 years ago

Hi guys, I'm a fan of this extension as well. I'm trying to conceptualize the problem, and I might be missing something :)

The semantics and behavior of the annotation seem pretty well spelled out - superclass, spec, or feature, when @Stepwise is not in the picture.

How @Stepwise would change the semantics is not clear to me. eg. - if I have a feature annotated with @RetryOnFailure, I would expect that feature to retry, not the whole spec. Annotating the spec with @RetryOnFailure should take care of that situation.

If the particular feature in question requires the entire spec to run again, wouldn't it make sense, and solve the problem to annotate the spec itself with @RetryOnFailure?

If things need to be more complex, I have some ideas, but I hate complexity ;)

I think that some failing tests / specs might be really valuable for this.

Klaas-Jan commented 7 years ago

Did someone find a solution for this already during the last months?

david-w-millar commented 7 years ago

I think that you may be able to use Spock's built in Polling Conditions to accomplish what you're looking for.

More info here

ivanomilo commented 6 years ago

Hi, I hope this isn't too out of place. I noticed that you guys had the extension running. I am running into this issue: "unable to resolve class RetryOnFailure , unable to find class for annotation"

Is there an import that you guys are using to be able to use the annotation? I have a gradle dependency in my build file for spock-retry. Thanks so much! Ivano