search

GZoltar / gzoltar

GZoltar - Java Library for Automatic Debugging http://www.gzoltar.com
Other
77 stars 34 forks source link

--timeout argument is useless ? #24

Closed lllwwwbbb closed 4 years ago

lllwwwbbb commented 4 years ago

I use runTestMethods command to run junit tests, and I want the gzoltar terminate some tests that cannot complete in 10 s. It seems that --timeout argument is not work?

jose commented 4 years ago

Hi @lllwwwbbb, thanks for reporting this. I've just checked and you're right, --timeout is not implemented. I thought it was.

I will implement it. (Ping me in a week if you don't hear any news from me.)

lllwwwbbb commented 4 years ago

Hi @jose , thanks for your reply. How is the implementation?

jose commented 4 years ago

Hi @lllwwwbbb,

The feature has been implemented. By default, the timeout is disabled. But using the option --timeout 10, forces GZoltar to cancel a test method if its runtime goes over 10 seconds.

Please let me know if it does not work as expected.

-- Best, Jose

lllwwwbbb commented 4 years ago

I use GZoltar as a JAR file. Will you release a new JAR file? Or how can I pack a JAR by some commands?

jose commented 4 years ago

There is no plan to release a new version any time soon. However, you can generate the jar files yourself by running mvn clean package in the root directory. Assuming the compilation and package runs without any issue, you can either find the jar file of each module in the target directory or find a zip file under com.gzoltar/target with all jar files.

lllwwwbbb commented 4 years ago

@jose Thanks so much. If one test method is killed due to TIMEOUT, will it appear in the result (matrix, spectral, tests) files? Is it treated as FAILED test?

lllwwwbbb commented 4 years ago

@jose I want to use GZoltar on Java1.7, so I run 'mvn package' with Java1.7. However it failed with exception 'javassist/CtBehavior : Unsupported major.minor version 52.0'.

jose commented 4 years ago

@jose Thanks so much. If one test method is killed due to TIMEOUT, will it appear in the result (matrix, spectral, tests) files? Is it treated as FAILED test?

Actually, I've just killed the timeout parameter. I thought that setting a time limit on the thread that runs each test case individually would do it, but it does not. The timeout I've implemented terminates the thread that collects coverage when it runs out of time which also terminates GZoltar. Long story short, unless the JUnit team implements (at some point) an API to set the timeout of each test case, GZoltar will never ever have an option/parameter to force a test case to run in X seconds.

How should you proceed then? There is no easy answer. I would suggest you go to every single test case you want to run under X seconds (e.g., 10 seconds) and annotated them as

@Test(timeout=10000)

(More information in here).

@jose I want to use GZoltar on Java1.7, so I run 'mvn package' with Java1.7. However it failed with exception 'javassist/CtBehavior : Unsupported major.minor version 52.0'.

The latest version of GZoltar requires Java 8 because one of its dependencies (i.e., javassist) requires Java 8. If you really need to run GZoltar under Java 7, I suggest you use GZoltar v1.7.2.

lllwwwbbb commented 4 years ago

@jose OK. Thank you so much.