Replication Package for SBST 2020 Competition?

Hello,

As SBST Unit-Test Competition utilizes Docker for the benchmarking I wonder if there is any plan in the future to publish the replication package of Evosuite in SBST competitions too?

Actually, I want to reproduce the result of Evosuite in SBST 2020 competition, but I don't know why I keep getting different results (i.e. almost all targets get 0% branch cov.) from Evosuite's Result Paper.

Context

I want to reproduce the result of Evosuite in SBST 2020 as published in the Result Paper

Steps to Reproduce

Followed the procedure here
Write a custom Evosuite runtool (following the Randoop example) evosuite_runtool.zip
contest_generate_tests.sh evosuite 1 1 60
contest_compute_metrics.sh results_evosuite_60 > stat_log.txt 2> error_log.txt

EvoSuite Arguments

cmdLine.append(String.format("%s -jar %s ", javaCommand, evosuiteJarFile.getAbsolutePath()));
cmdLine.append(String.format("-class %s ", testClass));
cmdLine.append(String.format("-Dsearch_budget=%s ", timeLimit));
cmdLine.append(String.format("-projectCP %s ", classPath));
cmdLine.append(String.format("-Dtest_dir=%s ", junitOutputDirName));
cmdLine.append("-Duse_separate_classloader=false -Dassertions=false -Dminimize_skip_coincidental=false ");

Current Result

Although there lies some test cases successfully generated, (almost) all branch coverage is 0%.

Here is a sample of GUAVA-128_1 GUAVA-128_1.zip p.s. the separateClassLoader = false and public Throwables_ESTest() { super(); } in Throwables_ESTest.java were manually added, but it didn't work.

I also found this error message in the metrics/log_detailed.txt GUAVA-128_1-metrics.zip

Running the tests: [SBSTDummyForCoverageAndMutationCalculation, com.google.common.base.Throwables_ESTest]
Failure: [initializationError(com.google.common.base.Throwables_ESTest): java.lang.IllegalStateException: Error during attachment using: org.evosuite.shaded.net.bytebuddy.agent.ByteBuddyAgent$AttachmentProvider$Compound@3f991331]
Failing Tests: initializationError(com.google.common.base.Throwables_ESTest)
java.lang.RuntimeException: java.lang.IllegalStateException: Error during attachment using: org.evosuite.shaded.net.bytebuddy.agent.ByteBuddyAgent$AttachmentProvider$Compound@3f991331
    at org.evosuite.runtime.agent.AgentLoader.loadAgent(AgentLoader.java:107)
    at org.evosuite.runtime.agent.InstrumentingAgent.initialize(InstrumentingAgent.java:101)
    at org.evosuite.runtime.EvoRunner.getClass(EvoRunner.java:102)
    at org.evosuite.runtime.EvoRunner.<init>(EvoRunner.java:77)
...

Expected result

Additional info

It's quite a dilemma for me, whether to post an issue here or in the SBST docker infrastructure's Github repo

But reflecting on my initial question, is there any plan to publish the replication package for Evosuite?

Thank you. :)

EvoSuite / evosuite