search

cloudfoundry / cf-deployment-concourse-tasks

Apache License 2.0
23 stars 76 forks source link

[Feature request]Ability to specify flakeAttempts in run-cats #62

Closed norshtein closed 6 years ago

norshtein commented 6 years ago

Sometimes the environment might be fragile, it would be useful to be able to manually set flakeAttempts in run-cats to make ginkgo try more times.

cf-gitbot commented 6 years ago

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/156794196

The labels on this github issue will be updated when the story is started.

dsabeti commented 6 years ago

Hi @norshtein. We thought about exposing this feature exactly as you've described. Ultimately, we decided against it because we don't want to "train" people running CATs to simply increase the flakeAttempts whenever there's a CATs failure.

We're hoping to address CATs flakiness in a more direct way sometime soon. If you look in our backlog, we have some stories to start measuring CATs flakiness so we can diagnose the root cause.

norshtein commented 6 years ago

Hi @dsabeti , thanks for explanation, I'm looking forward to the improvement.