Early termination of test execution

[GiovaniGuizzo]

I was talking to Justyna and she raised up the idea of early termination of test executions when a single test fails. Thus, instead of executing the test suite until the end, we can stop the execution at the first failing test case and return the maximum fitness value.

We can alternatively set it as default (since I believe it is going to speed up considerably the optimisation), and add a command line option such as "-ft" (full testing) to run all test cases even when one fails, just in case this functionality is needed.

We have to also think how to do it in case of multiple repetitions of test case executions.

[MaximoOliveira]

Hi @GiovaniGuizzo and @justynapt !

Would you like it to have the run configurations like this?

(1) : java -jar build/gin.jar -f examples/triangle/Triangle.java -ft -m "classifyTriangle(int,int,int)"

(2): java -jar build/gin.jar -f examples/triangle/Triangle.java -m "classifyTriangle(int,int,int)"

Where (1) will run all tests, even when one fails (i.e, do the full testing) and (2) will do an early termination of test executions when a single test fails.

Let me know!

Thank you!

[GiovaniGuizzo]

Hey @MaximoOliveira,

I have implemented this in my personal branch. However, I am waiting for a result of a paper to merge my commits :)

Anyway, I think a better option would be to have a -ff (fail fast) option.

[justynapt]

Pull request providing this functionality accepted. Closing.