search

tarantool / test-run

Tarantool functional testing framework
14 stars 15 forks source link

Option to disable test auto-rerun #334

Closed Gerold103 closed 2 years ago

Gerold103 commented 2 years ago

It complicates investigation of failure artifacts - rerun just creates more of them making it hard to find needed info.

NickVolynkin commented 2 years ago

Should this option disable auto-rerun for all tests, including those marked as fragile?

There's also a request for setting the amount of re-runs with an env variable and/or command line option. A solution for both request could look something like this.

# disable auto-retry
./test-run.py --retry 0
# disable auto-retry for fragile tests as well
./test-run.py --retry-force 0
# retry all tests like it's Christmas
./rest-run.py --retry 10

Follow-up to #328

NickVolynkin commented 2 years ago

@Gerold103 besides the CLI option, would you find it useful to set a system-wide config with "don't ever rerun tests on my machine"? Say, in the ~/.testrunrc file:

retry=0
Gerold103 commented 2 years ago

For me it would be enough to have a command line option which disables all auto-reruns, including fragile tests.