search

UT-SE-Research / iDFlakies

29 stars 33 forks source link

Added configurability of iDFlakies' behavior when the initial run of … #10

Closed s-winter closed 5 years ago

s-winter commented 5 years ago

…'mvn test' (e.g., during container creation) has errors or failures. The standard behavior of iDFlakies is to throw an exception in this case. For NOD flaky tests, this implied requirement of a passing mvn test run is overly restrictive, as NOD flaky tests can fail even during the initial run. To accommodate for this possibility, a property 'dt.mvn_test.must_pass' is introduced, which is set to true by default. This resembles iDFlakies' original behavior. If set to false, the check for mvn test errors or failures is simply skipped and no exception is thrown.

winglam commented 5 years ago

LGTM! But since you've been adding more configuration options to iDFlakies perhaps it's about time that we started documentation for all such options :) You can start by simply documenting the ones you added and at some point, one of us can document the other existing options as well!

s-winter commented 5 years ago

since you've been adding more configuration options to iDFlakies perhaps it's about time that we started documentation for all such options

Good point. What's a good way to document this project a bit more? I've seen some projects using wikis for 'user' documentation.