STAMP-project / test-runner

test driver to run JUnit tests isolated in a new JVM
GNU General Public License v3.0
16 stars 15 forks source link

[FEAT] System exit #122

Closed danglotb closed 2 years ago

danglotb commented 2 years ago

We call now a System.exit() at the end of the main methods of the runner classes. I recall that these classes should never be used directly, but only called by the EntryPoint, which will use a new java command line to run the main of the runner classes.

@martinezmatias @andre15silva This should solve the issue on Math2, since the process was not ending properly. I guess that the System.exit() is a bit rough but IMO, it is a valid workaround.

The tests is executing directly the main methods of the runner classes, this is why we must now use a special flag when running the test: mvn clean test -DnoExitForTesting=true.

danglotb commented 2 years ago

Merging and releasing.

martinezmatias commented 2 years ago

Thanks a lot @danglotb ! Which is the version of the release? 3.1.1? In that case, we will update the dependency from Flacoco (it uses 3.1.0) Regards Matias

danglotb commented 2 years ago

Hum, I have issue releasing, since the release goal of maven launches an inside maven command, without the option for testing and avoiding the System.exit().

I'll investigate to find a solution but might have revert this commit

martinezmatias commented 2 years ago

ok!

andre15silva commented 2 years ago

Hum, I have issue releasing, since the release goal of maven launches an inside maven command, without the option for testing and avoiding the System.exit().

You can use https://github.com/stefanbirkner/system-rules.

https://stefanbirkner.github.io/system-rules/apidocs/org/junit/contrib/java/lang/system/ExpectedSystemExit.html

We use it in Flacoco:

https://github.com/SpoonLabs/flacoco/blob/6f15f8162ce59c66af9f378691a5c9dcc603af7a/src/test/java/fr/spoonlabs/flacoco/cli/FlacocoMainTest.java#L56-L76

danglotb commented 2 years ago

Hi @martinezmatias

In the meantime, you can try on the SNAPSHOT, which is deployed on Nexus Repository.

martinezmatias commented 2 years ago

great, thanks, I will try 3.1.1-SNAPSHOT.

danglotb commented 2 years ago

@andre15silva Awesome! Thank you very much for the resources. I'll fix the test and release, thank you! :smile:

danglotb commented 2 years ago

test-runner-3.1.1 is out :confetti_ball:

andre15silva commented 2 years ago

Thanks @danglotb !

I'll merge https://github.com/SpoonLabs/flacoco/pull/117 once it is green @martinezmatias

andre15silva commented 2 years ago

Hi @danglotb @martinezmatias

I confirm Math5 generates fault localization results on my local setup! Should work form Math2 and Math8 too. Good job @danglotb !

martinezmatias commented 2 years ago

Great job @danglotb @andre15silva !!

I confirm that the issue is solved on the CI for Math-5, Math-8 and Math-28!! see here. (for M2 we don't have the memory issue but still no result, to be discussed in other issue suspicions logging from TestRunner)

monperrus commented 2 years ago

nice!