ASSERT-KTH / flacoco

FLACOCO: Fault Localization for Java based on Industry-grade Coverage http://arxiv.org/pdf/2111.12513
MIT License
29 stars 15 forks source link

Flacoco TestMethodFilter gives NoClassDefFoundError #251

Closed SleeplessMonths closed 4 months ago

SleeplessMonths commented 4 months ago

I'm trying to run Flacoco on a project containing multiple packages, each containing a class and a test, but I only get NoClassDefFoundErrors on all the found classes. When debugging I see that no tests are picked up, even though the directories do seem to be correct.

Does Flacoco perhaps not support packages? All the classes and test classes have been compiled before I try to run Flacoco. The tests are written in JUnit4, and not JUnit5 if that matters.

andre15silva commented 4 months ago

Hi @SleeplessMonths,

Could you post the command you are using the run flacoco?

SleeplessMonths commented 4 months ago

Hi @SleeplessMonths,

Could you post the command you are using the run flacoco?

Hi! I'm running it through the API, based on the example usage given in the wiki. Would it be preferable to run it through the CLI?

andre15silva commented 4 months ago

No, the API is fine!

Could you post your usage and the folder structure (e.g. using tree) of your project?

SleeplessMonths commented 4 months ago

It's a bit messy, but:

This is my usage:

public class FlacocoRunner {
    public static void main(String[] args) {
        FlacocoConfig config = new FlacocoConfig();
        config.setProjectPath("C4BJ/");
        config.setClasspath("target/classes/");

        config.setFamily(FlacocoConfig.FaultLocalizationFamily.SPECTRUM_BASED);
        config.setSpectrumFormula(SpectrumFormula.OCHIAI);

        Flacoco flacoco = new Flacoco(config);
        FlacocoResult result = flacoco.run();

        Set<TestMethod> failingTests = result.getFailingTests();
        Map<Location, Suspiciousness> mapping = result.getDefaultSuspiciousnessMap();
    }
}

This is the structure of my project:

.
├── C4BDumper
│   ├── README.md
│   ├── pom.xml
│   ├── src
│   │   ├── main
│   │   │   ├── java
│   │   │   └── resources
│   │   │       ├── exclude
│   │   │       │   └── subIdExclusionList
│   │   │       ├── queries
│   │   │       │   ├── sources.sql
│   │   │       │   └── tests.sql
│   │   │       └── templates
│   │   │           ├── testTemplate-experimental.txt
│   │   │           ├── testTemplate-old.txt
│   │   │           └── testTemplate.txt
│   │   └── test
│   │       └── java
│   └── target
│       ├── classes
│       │   ├── exclude
│       │   │   └── subIdExclusionList
│       │   ├── queries
│       │   │   ├── sources.sql
│       │   │   └── tests.sql
│       │   └── templates
│       │       ├── testTemplate-experimental.txt
│       │       ├── testTemplate-old.txt
│       │       └── testTemplate.txt
│       └── maven-status
│           └── maven-compiler-plugin
│               ├── compile
│               │   └── default-compile
│               │       └── inputFiles.lst
│               └── testCompile
│                   └── default-testCompile
│                       └── inputFiles.lst
├── C4BJ
│   ├── filter_non_usable.py
│   ├── no_failures
│   │   └── src
│   │       ├── main
│   │       │   └── java
│   │       │       └── java_6
│   │       │           └── problem_49A
│   │       │               └── subId_5686830
│   │       │                   └── Sleuth.java
│   │       └── test
│   │           └── java
│   │               └── java_6
│   │                   └── problem_49A
│   │                       └── subId_5686830
│   │                           └── Sleuth_Test.java
│   ├── pom.xml
│   ├── src
│   │   ├── main
│   │   │   └── java
│   │   │       └── java_6
│   │   │           ├── problem_10C
│   │   │           │   └── subId_8996077
│   │   │           │       └── Digit.java
│   │   │           ├── problem_17A
│   │   │           │   └── subId_9149865
│   │   │           │       └── NoldbachProblem.java
│   │   │           ├── problem_23A
│   │   │           │   ├── subId_1992033
│   │   │           │   │   └── A.java
│   │   │           │   └── subId_5687663
│   │   │           │       └── Main.java
│   │   │           ├── problem_40C
│   │   │           │   └── subId_1979122
│   │   │           │       └── Main.java
│   │   │           └── problem_42A
│   │   │               ├── subId_199916
│   │   │               │   └── Guilty.java
│   │   │               ├── subId_199917
│   │   │               │   └── Guilty.java
│   │   │               ├── subId_199918
│   │   │               │   └── Guilty.java
│   │   │               ├── subId_199919
│   │   │               │   └── Guilty.java
│   │   │               └── subId_199920
│   │   │                   └── Guilty.java
│   │   └── test
│   │       └── java
│   │           └── java_6
│   │               ├── problem_10C
│   │               │   └── subId_8996077
│   │               │       └── Digit_Test.java
│   │               ├── problem_17A
│   │               │   └── subId_9149865
│   │               │       └── NoldbachProblem_Test.java
│   │               ├── problem_23A
│   │               │   ├── subId_1992033
│   │               │   │   └── A_Test.java
│   │               │   └── subId_5687663
│   │               │       └── Main_Test.java
│   │               ├── problem_40C
│   │               │   └── subId_1979122
│   │               │       └── Main_Test.java
│   │               └── problem_42A
│   │                   ├── subId_199916
│   │                   │   └── Guilty_Test.java
│   │                   ├── subId_199917
│   │                   │   └── Guilty_Test.java
│   │                   ├── subId_199918
│   │                   │   └── Guilty_Test.java
│   │                   ├── subId_199919
│   │                   │   └── Guilty_Test.java
│   │                   └── subId_199920
│   │                       └── Guilty_Test.java
│   ├── src.zip
│   ├── target
│   │   ├── classes
│   │   │   └── java_6
│   │   │       ├── problem_10C
│   │   │       │   └── subId_8996077
│   │   │       │       └── Digit.class
│   │   │       ├── problem_17A
│   │   │       │   └── subId_9149865
│   │   │       │       └── NoldbachProblem.class
│   │   │       ├── problem_23A
│   │   │       │   ├── subId_1992033
│   │   │       │   │   └── A.class
│   │   │       │   └── subId_5687663
│   │   │       │       └── Main.class
│   │   │       ├── problem_40C
│   │   │       │   └── subId_1979122
│   │   │       │       └── Main.class
│   │   │       └── problem_42A
│   │   │           ├── subId_199916
│   │   │           │   └── Guilty.class
│   │   │           ├── subId_199917
│   │   │           │   └── Guilty.class
│   │   │           ├── subId_199918
│   │   │           │   └── Guilty.class
│   │   │           ├── subId_199919
│   │   │           │   └── Guilty.class
│   │   │           └── subId_199920
│   │   │               └── Guilty.class
│   │   ├── generated-sources
│   │   │   └── annotations
│   │   ├── generated-test-sources
│   │   │   └── test-annotations
│   │   ├── maven-status
│   │   │   └── maven-compiler-plugin
│   │   │       ├── compile
│   │   │       │   └── default-compile
│   │   │       │       ├── createdFiles.lst
│   │   │       │       └── inputFiles.lst
│   │   │       └── testCompile
│   │   │           └── default-testCompile
│   │   │               ├── createdFiles.lst
│   │   │               └── inputFiles.lst
│   │   ├── surefire-reports
│   │   │   ├── TEST-java_6.problem_10C.subId_8996077.Digit_Test.xml
│   │   │   ├── TEST-java_6.problem_17A.subId_9149865.NoldbachProblem_Test.xml
│   │   │   ├── TEST-java_6.problem_23A.subId_1992033.A_Test.xml
│   │   │   ├── TEST-java_6.problem_23A.subId_5687663.Main_Test.xml
│   │   │   ├── TEST-java_6.problem_40C.subId_1979122.Main_Test.xml
│   │   │   ├── TEST-java_6.problem_42A.subId_199916.Guilty_Test.xml
│   │   │   ├── TEST-java_6.problem_42A.subId_199917.Guilty_Test.xml
│   │   │   ├── TEST-java_6.problem_42A.subId_199918.Guilty_Test.xml
│   │   │   ├── TEST-java_6.problem_42A.subId_199919.Guilty_Test.xml
│   │   │   ├── TEST-java_6.problem_42A.subId_199920.Guilty_Test.xml
│   │   │   ├── java_6.problem_10C.subId_8996077.Digit_Test.txt
│   │   │   ├── java_6.problem_17A.subId_9149865.NoldbachProblem_Test.txt
│   │   │   ├── java_6.problem_23A.subId_1992033.A_Test.txt
│   │   │   ├── java_6.problem_23A.subId_5687663.Main_Test.txt
│   │   │   ├── java_6.problem_40C.subId_1979122.Main_Test.txt
│   │   │   ├── java_6.problem_42A.subId_199916.Guilty_Test.txt
│   │   │   ├── java_6.problem_42A.subId_199917.Guilty_Test.txt
│   │   │   ├── java_6.problem_42A.subId_199918.Guilty_Test.txt
│   │   │   ├── java_6.problem_42A.subId_199919.Guilty_Test.txt
│   │   │   └── java_6.problem_42A.subId_199920.Guilty_Test.txt
│   │   └── test-classes
│   │       └── java_6
│   │           ├── problem_10C
│   │           │   └── subId_8996077
│   │           │       ├── Digit_Test.class
│   │           │       └── TestUtils.class
│   │           ├── problem_17A
│   │           │   └── subId_9149865
│   │           │       ├── NoldbachProblem_Test.class
│   │           │       └── TestUtils.class
│   │           ├── problem_23A
│   │           │   ├── subId_1992033
│   │           │   │   ├── A_Test.class
│   │           │   │   └── TestUtils.class
│   │           │   └── subId_5687663
│   │           │       ├── Main_Test.class
│   │           │       └── TestUtils.class
│   │           ├── problem_40C
│   │           │   └── subId_1979122
│   │           │       ├── Main_Test.class
│   │           │       └── TestUtils.class
│   │           └── problem_42A
│   │               ├── subId_199916
│   │               │   ├── Guilty_Test.class
│   │               │   └── TestUtils.class
│   │               ├── subId_199917
│   │               │   ├── Guilty_Test.class
│   │               │   └── TestUtils.class
│   │               ├── subId_199918
│   │               │   ├── Guilty_Test.class
│   │               │   └── TestUtils.class
│   │               ├── subId_199919
│   │               │   ├── Guilty_Test.class
│   │               │   └── TestUtils.class
│   │               └── subId_199920
│   │                   ├── Guilty_Test.class
│   │                   └── TestUtils.class
│   └── throws_errors
│       └── src
│           ├── main
│           │   └── java
│           │       └── java_6
│           │           └── problem_36A
│           │               ├── subId_524615
│           │               │   └── Main.java
│           │               ├── subId_564148
│           │               │   └── p36a.java
│           │               ├── subId_565814
│           │               │   └── A.java
│           │               ├── subId_565816
│           │               │   └── A.java
│           │               ├── subId_565825
│           │               │   └── A.java
│           │               ├── subId_610002
│           │               │   └── P036A.java
│           │               ├── subId_822489
│           │               │   └── A.java
│           │               └── subId_989083
│           │                   └── A036.java
│           └── test
│               └── java
│                   └── java_6
│                       └── problem_36A
│                           ├── subId_524615
│                           │   └── Main_Test.java
│                           ├── subId_564148
│                           │   └── p36a_Test.java
│                           ├── subId_565814
│                           │   └── A_Test.java
│                           ├── subId_565816
│                           │   └── A_Test.java
│                           ├── subId_565825
│                           │   └── A_Test.java
│                           ├── subId_610002
│                           │   └── P036A_Test.java
│                           ├── subId_822489
│                           │   └── A_Test.java
│                           └── subId_989083
│                               └── A036_Test.java
├── README.md
├── jfixsuite.iml
├── out
│   └── production
│       └── jfixsuite
│           ├── FlacocoRunner.class
│           └── Main.class
├── pom.xml
├── scripts
│   ├── count_errors_and_failures.py
│   ├── filter_non_usable.py
│   ├── move_already_processed.py
│   ├── test.py
│   └── test.sh
├── src
│   ├── main
│   │   ├── java
│   │   │   ├── FlacocoRunner.java
│   │   │   └── Main.java
│   │   └── resources
│   └── test
│       ├── java
│       │   └── Main_Test.java
│       └── resources
└── target
    └── CoveredTestResultPerTest.dat
andre15silva commented 4 months ago
        config.setProjectPath("C4BJ/");
        config.setClasspath("target/classes/");

Here you are setting the classpath to be target/classes. Shouldn't it be C4BJ/target/classes:C4BJ/target/test-classes instead?

This would fix two problems: 1) not pointing to the right sub-directory 2) not including test classes in the classpath

SleeplessMonths commented 4 months ago

When I debugged and checked the FlacocoConfig, both ways seem to give the same results in the src and bin paths. I still get the same error even with the classpath that you suggested.

andre15silva commented 4 months ago

Could you upload this project somewhere so that I can test on my side?

SleeplessMonths commented 4 months ago

Could you upload this project somewhere so that I can test on my side?

I've pushed my project and some src files I was testing with, to this repo:

https://github.com/SleeplessMonths/JFixSuite

andre15silva commented 4 months ago

Hi @SleeplessMonths ,

I am not being able to run your FlacocoRunner class. Could you provide instructions on how to do this? I think you might have to change your pom file.

However, I tried running flacoco through the cli using a pre-compiled jar and it worked. Since the C4BJ directory is a maven project too, you can simply pass the argument projectpath or p for short and the classpath/directories will be computed automatically.

See the logs:

base ❯ pwd                                                                                                                                                                                                                                                                           (base)
/home/andre/Repos/tmp/JFixSuite

~/Repos/tmp/JFixSuite main*
base ❯ java -jar /home/andre/Repos/flacoco/target/flacoco-1.0.7-SNAPSHOT-jar-with-dependencies.jar -p C4BJ/                                                                                                                                                                          (base)
[0] INFO Flacoco - Running Flacoco...
[4375] INFO CoverageRunner - Tests found: 638
[4375] INFO CoverageRunner - Tests executed: 638
java_6.problem_42A.subId_199916.TestUtils,2137,0.3913730528802255
java_6.problem_42A.subId_199917.TestUtils,2143,0.3913730528802255
java_6.problem_42A.subId_199918.TestUtils,2143,0.3913730528802255
java_6.problem_42A.subId_199919.TestUtils,2143,0.3913730528802255
java_6.problem_42A.subId_199920.TestUtils,2143,0.3913730528802255
java_6.problem_42A.subId_199916.Guilty,13,0.366095968573391
java_6.problem_42A.subId_199917.Guilty,13,0.366095968573391
java_6.problem_42A.subId_199918.Guilty,13,0.366095968573391
java_6.problem_42A.subId_199919.Guilty,13,0.366095968573391
java_6.problem_42A.subId_199920.Guilty,13,0.366095968573391
java_6.problem_42A.subId_199916.Guilty,14,0.366095968573391
java_6.problem_42A.subId_199917.Guilty,14,0.366095968573391
java_6.problem_42A.subId_199918.Guilty,14,0.366095968573391
java_6.problem_42A.subId_199919.Guilty,14,0.366095968573391
java_6.problem_42A.subId_199920.Guilty,14,0.366095968573391
java_6.problem_42A.subId_199916.Guilty,15,0.366095968573391
java_6.problem_42A.subId_199917.Guilty,15,0.366095968573391
java_6.problem_42A.subId_199918.Guilty,15,0.366095968573391
java_6.problem_42A.subId_199919.Guilty,15,0.366095968573391
java_6.problem_42A.subId_199920.Guilty,15,0.366095968573391
java_6.problem_42A.subId_199916.Guilty,16,0.366095968573391
java_6.problem_42A.subId_199917.Guilty,16,0.366095968573391
java_6.problem_42A.subId_199918.Guilty,16,0.366095968573391
java_6.problem_42A.subId_199919.Guilty,16,0.366095968573391
java_6.problem_42A.subId_199920.Guilty,16,0.366095968573391
java_6.problem_42A.subId_199916.Guilty,17,0.366095968573391
java_6.problem_42A.subId_199917.Guilty,17,0.366095968573391
java_6.problem_42A.subId_199918.Guilty,17,0.366095968573391
java_6.problem_42A.subId_199919.Guilty,17,0.366095968573391
java_6.problem_42A.subId_199920.Guilty,17,0.366095968573391
java_6.problem_42A.subId_199916.Guilty,18,0.366095968573391
java_6.problem_42A.subId_199917.Guilty,18,0.366095968573391
java_6.problem_42A.subId_199918.Guilty,18,0.366095968573391
java_6.problem_42A.subId_199919.Guilty,18,0.366095968573391
java_6.problem_42A.subId_199920.Guilty,18,0.366095968573391
java_6.problem_42A.subId_199916.Guilty,19,0.366095968573391
java_6.problem_42A.subId_199917.Guilty,19,0.366095968573391
java_6.problem_42A.subId_199918.Guilty,19,0.366095968573391
java_6.problem_42A.subId_199919.Guilty,19,0.366095968573391
java_6.problem_42A.subId_199920.Guilty,19,0.366095968573391
java_6.problem_42A.subId_199916.Guilty,20,0.366095968573391
java_6.problem_42A.subId_199917.Guilty,20,0.366095968573391
java_6.problem_42A.subId_199918.Guilty,20,0.366095968573391
java_6.problem_42A.subId_199919.Guilty,20,0.366095968573391
java_6.problem_42A.subId_199920.Guilty,20,0.366095968573391
java_6.problem_42A.subId_199916.Guilty,21,0.366095968573391
java_6.problem_42A.subId_199917.Guilty,21,0.366095968573391
java_6.problem_42A.subId_199918.Guilty,21,0.366095968573391
java_6.problem_42A.subId_199919.Guilty,21,0.366095968573391
java_6.problem_42A.subId_199920.Guilty,21,0.366095968573391
java_6.problem_42A.subId_199916.Guilty,23,0.366095968573391
java_6.problem_42A.subId_199917.Guilty,23,0.366095968573391
java_6.problem_42A.subId_199918.Guilty,23,0.366095968573391
java_6.problem_42A.subId_199919.Guilty,23,0.366095968573391
java_6.problem_42A.subId_199920.Guilty,23,0.366095968573391
java_6.problem_42A.subId_199916.Guilty,24,0.366095968573391
java_6.problem_42A.subId_199917.Guilty,24,0.366095968573391
java_6.problem_42A.subId_199918.Guilty,24,0.366095968573391
java_6.problem_42A.subId_199919.Guilty,24,0.366095968573391
java_6.problem_42A.subId_199920.Guilty,24,0.366095968573391
java_6.problem_42A.subId_199916.Guilty,25,0.366095968573391
java_6.problem_42A.subId_199917.Guilty,25,0.366095968573391
java_6.problem_42A.subId_199918.Guilty,25,0.366095968573391
java_6.problem_42A.subId_199919.Guilty,25,0.366095968573391
java_6.problem_42A.subId_199920.Guilty,25,0.366095968573391
java_6.problem_42A.subId_199916.Guilty,26,0.366095968573391
java_6.problem_42A.subId_199917.Guilty,26,0.366095968573391
java_6.problem_42A.subId_199918.Guilty,26,0.366095968573391
java_6.problem_42A.subId_199919.Guilty,26,0.366095968573391
java_6.problem_42A.subId_199920.Guilty,26,0.366095968573391
java_6.problem_42A.subId_199916.Guilty,27,0.366095968573391
java_6.problem_42A.subId_199917.Guilty,27,0.366095968573391
java_6.problem_42A.subId_199918.Guilty,27,0.366095968573391
java_6.problem_42A.subId_199919.Guilty,27,0.366095968573391
java_6.problem_42A.subId_199920.Guilty,27,0.366095968573391
java_6.problem_42A.subId_199916.Guilty,28,0.366095968573391
java_6.problem_42A.subId_199917.Guilty,28,0.366095968573391
java_6.problem_42A.subId_199918.Guilty,28,0.366095968573391
java_6.problem_42A.subId_199919.Guilty,28,0.366095968573391
java_6.problem_42A.subId_199920.Guilty,28,0.366095968573391
java_6.problem_42A.subId_199916.Guilty,29,0.366095968573391
java_6.problem_42A.subId_199917.Guilty,29,0.366095968573391
java_6.problem_42A.subId_199918.Guilty,29,0.366095968573391
java_6.problem_42A.subId_199919.Guilty,29,0.366095968573391
java_6.problem_42A.subId_199920.Guilty,29,0.366095968573391
java_6.problem_40C.subId_1979122.TestUtils,1961,0.2604490315409803
java_6.problem_23A.subId_1992033.TestUtils,1233,0.24752605238300693
java_6.problem_23A.subId_5687663.TestUtils,1233,0.23852207234014097
java_6.problem_40C.subId_1979122.Main,61,0.2093066251016711
java_6.problem_10C.subId_8996077.TestUtils,1103,0.20390069235134212
java_6.problem_40C.subId_1979122.Main,73,0.2010953007402246
java_6.problem_23A.subId_1992033.A,12,0.19745748132842209
java_6.problem_40C.subId_1979122.Main,60,0.1955339839771952
java_6.problem_40C.subId_1979122.Main,68,0.1955339839771952
java_6.problem_40C.subId_1979122.Main,72,0.1955339839771952
java_6.problem_40C.subId_1979122.Main,75,0.1955339839771952
java_6.problem_23A.subId_1992033.A,5,0.19525118323914187
java_6.problem_23A.subId_1992033.A,8,0.19525118323914187
java_6.problem_23A.subId_1992033.A,9,0.19525118323914187
java_6.problem_23A.subId_1992033.A,10,0.19525118323914187
java_6.problem_23A.subId_1992033.A,11,0.19525118323914187
java_6.problem_23A.subId_1992033.A,16,0.19525118323914187
java_6.problem_23A.subId_1992033.A,19,0.19525118323914187
java_6.problem_23A.subId_1992033.A,20,0.19525118323914187
java_6.problem_23A.subId_5687663.Main,19,0.18994301159656732
java_6.problem_23A.subId_5687663.Main,20,0.18994301159656732
java_6.problem_23A.subId_5687663.Main,24,0.18994301159656732
java_6.problem_23A.subId_1992033.A,13,0.1897099503292974
java_6.problem_40C.subId_1979122.Main,55,0.18878939207081794
java_6.problem_40C.subId_1979122.Main,56,0.18878939207081794
java_6.problem_40C.subId_1979122.Main,57,0.18878939207081794
java_6.problem_40C.subId_1979122.Main,58,0.18878939207081794
java_6.problem_40C.subId_1979122.Main,59,0.18878939207081794
java_6.problem_40C.subId_1979122.Main,78,0.18878939207081794
java_6.problem_40C.subId_1979122.Main,79,0.18878939207081794
java_6.problem_40C.subId_1979122.Main,65,0.18456624629811574
java_6.problem_40C.subId_1979122.Main,69,0.18456624629811574
java_6.problem_23A.subId_5687663.Main,17,0.1833533755192491
java_6.problem_23A.subId_5687663.Main,18,0.1833533755192491
java_6.problem_23A.subId_5687663.Main,9,0.18130467015063173
java_6.problem_23A.subId_5687663.Main,12,0.18130467015063173
java_6.problem_23A.subId_5687663.Main,13,0.18130467015063173
java_6.problem_23A.subId_5687663.Main,14,0.18130467015063173
java_6.problem_23A.subId_5687663.Main,15,0.18130467015063173
java_6.problem_23A.subId_5687663.Main,16,0.18130467015063173
java_6.problem_23A.subId_5687663.Main,23,0.18130467015063173
java_6.problem_23A.subId_5687663.Main,27,0.18130467015063173
java_6.problem_23A.subId_5687663.Main,28,0.18130467015063173
java_6.problem_40C.subId_1979122.Main,62,0.1775101316182642
java_6.problem_40C.subId_1979122.Main,63,0.1775101316182642
java_6.problem_40C.subId_1979122.Main,9,0.16972357215341674
java_6.problem_40C.subId_1979122.Main,12,0.16972357215341674
java_6.problem_40C.subId_1979122.Main,13,0.16972357215341674
java_6.problem_40C.subId_1979122.Main,15,0.16972357215341674
java_6.problem_40C.subId_1979122.Main,16,0.16972357215341674
java_6.problem_40C.subId_1979122.Main,20,0.16972357215341674
java_6.problem_40C.subId_1979122.Main,21,0.16972357215341674
java_6.problem_40C.subId_1979122.Main,22,0.16972357215341674
java_6.problem_40C.subId_1979122.Main,23,0.16972357215341674
java_6.problem_40C.subId_1979122.Main,24,0.16972357215341674
java_6.problem_40C.subId_1979122.Main,25,0.16972357215341674
java_6.problem_40C.subId_1979122.Main,26,0.16972357215341674
java_6.problem_40C.subId_1979122.Main,27,0.16972357215341674
java_6.problem_40C.subId_1979122.Main,28,0.16972357215341674
java_6.problem_40C.subId_1979122.Main,29,0.16972357215341674
java_6.problem_40C.subId_1979122.Main,31,0.16972357215341674
java_6.problem_40C.subId_1979122.Main,36,0.16972357215341674
java_6.problem_40C.subId_1979122.Main,38,0.16972357215341674
java_6.problem_40C.subId_1979122.Main,50,0.16972357215341674
java_6.problem_40C.subId_1979122.Main,81,0.16972357215341674
java_6.problem_40C.subId_1979122.Main,82,0.16972357215341674
java_6.problem_10C.subId_8996077.Digit,4,0.14052885419779249
java_6.problem_10C.subId_8996077.Digit,6,0.14052885419779249
java_6.problem_10C.subId_8996077.Digit,7,0.14052885419779249
java_6.problem_10C.subId_8996077.Digit,8,0.14052885419779249
java_6.problem_10C.subId_8996077.Digit,9,0.14052885419779249
java_6.problem_10C.subId_8996077.Digit,10,0.14052885419779249
java_6.problem_10C.subId_8996077.Digit,11,0.14052885419779249
java_6.problem_10C.subId_8996077.Digit,13,0.14052885419779249
java_6.problem_10C.subId_8996077.Digit,14,0.14052885419779249
java_6.problem_10C.subId_8996077.Digit,15,0.14052885419779249
java_6.problem_10C.subId_8996077.Digit,16,0.14052885419779249
java_6.problem_10C.subId_8996077.Digit,19,0.14052885419779249
java_6.problem_10C.subId_8996077.Digit,20,0.14052885419779249
java_6.problem_10C.subId_8996077.Digit,21,0.14052885419779249
java_6.problem_10C.subId_8996077.Digit,22,0.14052885419779249
java_6.problem_40C.subId_1979122.Main,43,0.12474140525999677
java_6.problem_40C.subId_1979122.Main,44,0.12474140525999677
java_6.problem_40C.subId_1979122.Main,45,0.12474140525999677
java_6.problem_40C.subId_1979122.Main,39,0.11535390393632235
java_6.problem_40C.subId_1979122.Main,40,0.11535390393632235
java_6.problem_17A.subId_9149865.TestUtils,975,0.08102191939419527
java_6.problem_40C.subId_1979122.Main,41,0.0441027467770415
java_6.problem_40C.subId_1979122.Main,46,0.03307706008278112
java_6.problem_17A.subId_9149865.NoldbachProblem,33,0.024807795062085844
java_6.problem_17A.subId_9149865.NoldbachProblem,7,0.0237207891141676
java_6.problem_17A.subId_9149865.NoldbachProblem,11,0.0237207891141676
java_6.problem_17A.subId_9149865.NoldbachProblem,12,0.0237207891141676
java_6.problem_17A.subId_9149865.NoldbachProblem,14,0.0237207891141676
java_6.problem_17A.subId_9149865.NoldbachProblem,15,0.0237207891141676
java_6.problem_17A.subId_9149865.NoldbachProblem,16,0.0237207891141676
java_6.problem_17A.subId_9149865.NoldbachProblem,17,0.0237207891141676
java_6.problem_17A.subId_9149865.NoldbachProblem,18,0.0237207891141676
java_6.problem_17A.subId_9149865.NoldbachProblem,20,0.0237207891141676
java_6.problem_17A.subId_9149865.NoldbachProblem,21,0.0237207891141676
java_6.problem_17A.subId_9149865.NoldbachProblem,24,0.0237207891141676
java_6.problem_17A.subId_9149865.NoldbachProblem,25,0.0237207891141676
java_6.problem_17A.subId_9149865.NoldbachProblem,26,0.0237207891141676
java_6.problem_17A.subId_9149865.NoldbachProblem,27,0.0237207891141676
java_6.problem_17A.subId_9149865.NoldbachProblem,30,0.0237207891141676
java_6.problem_17A.subId_9149865.NoldbachProblem,31,0.0237207891141676
java_6.problem_17A.subId_9149865.NoldbachProblem,32,0.0237207891141676
java_6.problem_17A.subId_9149865.NoldbachProblem,37,0.0237207891141676
java_6.problem_17A.subId_9149865.NoldbachProblem,38,0.0237207891141676
SleeplessMonths commented 4 months ago

Thank you for the help @andre15silva, using the CLI seems to be much more convenient, so I'll just go with that moving forward.