Unable to run testrunner to create the folds

[AlexMViver]

When executing the example runAll.sh data/test.csv the following output is produced for each test:

***IFIXPLUS**** Finished run_debugging_tools.sh seg 15 abr 2024 20:08:13 WEST Traceback (most recent call last): File "/home/alex/Repositories/ODRepair/experiments/run.py", line 834, in runExp() File "/home/alex/Repositories/ODRepair/experiments/run.py", line 769, in runExp all_cases = extractCasesInfo(EXP_CSV) File "/home/alex/Repositories/ODRepair/experiments/run.py", line 111, in extractCasesInfo case['field_fqn'] = row[13] IndexError: list index out of range seg 15 abr 2024 20:08:13 WEST ===> Running ktuukkan/marine-api net.sf.marineapi.ais.event.AbstractAISMessageListenerTest.testConstructor find: ‘gen_tests/marine-api’: No such file or directory NO GEN TESTS FOR net.sf.marineapi.ais.event.AbstractAISMessageListenerTest.testConstructor

This seems to occur due to the folds expected to be generated by running the testrunner in run_debugging_tools_new.sh (line 104) not being generated. Since the next step would be executing outputResults.sh and it needs those folds, the temporary result.csv file is created with several null columns impeding the rest of the process.

The generated log:

[INFO] Scanning for projects... [INFO] [INFO] ---------------------< net.sf.marineapi:marineapi >--------------------- [INFO] Building Java Marine API 0.11.0-SNAPSHOT [INFO] -------------------------------[ bundle ]------------------------------- [INFO] [INFO] >>> testrunner-maven-plugin:1.2-SNAPSHOT:testplugin (default-cli) > test-compile @ marineapi >>> [INFO] [INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-maven) @ marineapi --- [INFO] Skipping Rule Enforcement. [INFO] [INFO] --- maven-resources-plugin:3.3.1:resources (default-resources) @ marineapi --- [INFO] Copying 3 resources from src/main/resources to target/classes [INFO] [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ marineapi --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- maven-resources-plugin:3.3.1:testResources (default-testResources) @ marineapi --- [INFO] Copying 10 resources from src/test/resources to target/test-classes [INFO] [INFO] --- maven-compiler-plugin:3.8.0:testCompile (default-testCompile) @ marineapi --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] <<< testrunner-maven-plugin:1.2-SNAPSHOT:testplugin (default-cli) < test-compile @ marineapi <<< [INFO] [INFO] [INFO] --- testrunner-maven-plugin:1.2-SNAPSHOT:testplugin (default-cli) @ marineapi --- xmlFileName: 0 [INFO] Module is not using a supported test framework (probably not JUnit). [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 1.741 s [INFO] Finished at: 2024-04-15T20:08:13+01:00 [INFO] ------------------------------------------------------------------------

The result of executing mvn testrunner:testplugin on the projectsinstall/ktuukan/marine-api project directory: testrunner.txt. The complete execution log for runAll.sh data/test.csv: ExecutionLog.txt

I was also unable of obtaining any test or patch generation for any other projects I tried besides ktuukan/marine-api. I am unaware of how to resolve the issue with the testrunner, is there a way to resolve it?

[ChopinLi-cp]

Hi,

Thank you for reaching out!

I think the problem can be the java version. We use a old customized version for testrunner that sometimes can not work on higher java version (like java 17). You can try java 8 instead.

And mvn testrunner:testplugin needs to run with some configuration options that can be seen in the line 104 of run_debugging_tools_new.sh.

There are some steps in the runAll.sh. You can try to run each line one by one to check if the tool outputs the static fields. The results can be found in output/result.csv. If the tool succeeds at the first step, you should see the static fields at the 14th column in the file output/result.csv.

You may also want to check if in the repos directory, idflakies4iFixPlus/iDFlakies and idflakies4iFixPlus/testrunner compiles successfully because they run together to find the static fields.

[AlexMViver]

Hello, thank you for the answer.

I'm using Java 8 for this.

I've verified the steps from runAll.sh until it reaches the testrunner execution. Until there everything seems to work correctly, however, once it attempts to execute the testrunner it says the Module is not using a supported test framework (probably not JUnit). while also printing xmlFileName: 0.

This can be observed in the second log I posted previously. Since the testrunner ends up not actually executing, it is unable to generate the necessary data for output/result.csv to be complete, resulting in the null columns.

Relating to idflakies4iFixPlus/iDFlakies and idflakies4iFixPlus/testrunner both compile successfully, although idflakies4iFixPlus/testrunner has a few warnings testrunnerbuild.txt.

I am unsure of what could be causing the testrunner to not recognize ktuukan/marine-api as JUnit tests.

[AlexMViver]

I have identified the cause for this issue.

In my setup I have a folder for repositories at /home/user/Repositories . Meanwhile I have a shortcut for the repositories folder in my desktop, this results in the /home/user/Desktop/Repositories directory.

ODRepair works correctly when executed in the /home/user/Repositories directory but when executed for the shortcut /home/user/Desktop/Repositories it fails when it reaches the testrunner. Something similar happens while running the tool iDFlakies and I suspect it would be reproducible for any use of the testrunner.

Hope this helps anyone who finds themselves in a similar situation.