Open DehengYang opened 3 years ago
Dear Sirs,
Hi @DehengYang !
I am very interested in this project and I am trying to package it on my pc for fault localization.
Nice!
I cloned this project and then ran
mvn package -DskipTests
, but it reports the following two types of compilation errors:[ERROR] /home/apr/tools/flacoco/src/main/java/fr/spoonlabs/flacoco/core/test/strategies/testrunner/TestRunnerStrategy.java:[11,24] package org.apache.log4j does not exist [ERROR] /home/apr/tools/flacoco/src/main/java/fr/spoonlabs/flacoco/core/test/strategies/testrunner/TestRunnerStrategy.java:[29,13] cannot find symbol [ERROR] symbol: class Logger
[ERROR] /home/apr/tools/flacoco/src/main/java/fr/spoonlabs/flacoco/core/coverage/CoverageMatrix.java:[15,31] package org.jacoco.core.runtime does not exist
That is weird. I'm not able to reproduce such error.
Could you post the entire log when running mvn package -DskipTests -X
?
I think this could be resolved by adding the following dependencies into
pom.xml
:<dependency> <groupId>log4j</groupId> <artifactId>log4j</artifactId> <version>1.2.17</version> </dependency> <dependency> <groupId>org.jacoco</groupId> <artifactId>org.jacoco.core</artifactId> <version>0.8.7</version> </dependency>
You should not need to add the org.jacoco.core
dependency as it already comes from test-runner
.
The logging also works on my setup and in CI, so I don't really know what could be the issue.
With this workaround, I have used the
flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar
to localize a bug in alibaba/fastjson@96d0ee5 with the following command:flacocoPath=/home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar projPath=/home/apr/fastjson java -jar $flacocoPath --projectpath $projPath
It failed with the following error:
[1522] WARN TestMethodFilter - NoClassDefFoundError: class com.alibaba.json.bvt.issue_1700.Issue1701 [1526] WARN TestMethodFilter - NoClassDefFoundError: class com.alibaba.json.bvt.cglib.TestCglib [1531] WARN TestMethodFilter - NoClassDefFoundError: class com.alibaba.json.test.JsonIteratorImageTest [1531] WARN TestMethodFilter - NoClassDefFoundError: class com.alibaba.json.test.Issue3805 [1533] WARN TestMethodFilter - NoClassDefFoundError: class com.alibaba.json.test.codec.Jackson2AfterBurnCodec [1538] WARN TestMethodFilter - NoClassDefFoundError: class com.alibaba.json.test.benchmark.jdk10.generated.StringBenchmark_testParseJsonComp_jmhTest [1539] WARN TestMethodFilter - NoClassDefFoundError: class com.alibaba.json.test.benchmark.jdk10.generated.StringBenchmark_jackson_testParseJson_jmhTest [1539] WARN TestMethodFilter - NoClassDefFoundError: class com.alibaba.json.test.benchmark.jdk10.generated.StringBenchmark_testParseJson_jmhTest [1539] WARN TestMethodFilter - NoClassDefFoundError: class com.alibaba.json.test.benchmark.jdk10.generated.StringBenchmark_jackson_testParseJsonComp_jmhTest [1541] WARN TestMethodFilter - NoClassDefFoundError: class com.alibaba.json.test.JSONLibXmlTest SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. Exception in thread "main" java.lang.NoClassDefFoundError: org/jacoco/agent/rt/RT at eu.stamp_project.testrunner.EntryPoint.<clinit>(EntryPoint.java:868) at fr.spoonlabs.flacoco.core.coverage.framework.TestFrameworkStrategy.setupTestRunnerEntryPoint(TestFrameworkStrategy.java:32) at fr.spoonlabs.flacoco.core.coverage.framework.JUnit4Strategy.execute(JUnit4Strategy.java:25) at fr.spoonlabs.flacoco.core.coverage.CoverageRunner.getCoverageMatrix(CoverageRunner.java:49) at fr.spoonlabs.flacoco.localization.spectrum.SpectrumRunner.computeCoverageMatrix(SpectrumRunner.java:56) at fr.spoonlabs.flacoco.localization.spectrum.SpectrumRunner.run(SpectrumRunner.java:32) at fr.spoonlabs.flacoco.api.Flacoco.run(Flacoco.java:31) at fr.spoonlabs.flacoco.cli.FlacocoMain.call(FlacocoMain.java:157) at fr.spoonlabs.flacoco.cli.FlacocoMain.call(FlacocoMain.java:27) at picocli.CommandLine.executeUserObject(CommandLine.java:1953) at picocli.CommandLine.access$1300(CommandLine.java:145) at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352) at picocli.CommandLine$RunLast.handle(CommandLine.java:2346) at picocli.CommandLine$RunLast.handle(CommandLine.java:2311) at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179) at picocli.CommandLine.execute(CommandLine.java:2078) at fr.spoonlabs.flacoco.cli.FlacocoMain.main(FlacocoMain.java:148) Caused by: java.lang.ClassNotFoundException: org.jacoco.agent.rt.RT at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 17 more
Then I used the jar from the maven repo:
projPath=/home/apr/fastjson flacocoPath=/home/apr/.m2/repository/com/github/spoonlabs/flacoco/1.0.0/flacoco-1.0.0.jar java -cp $flacocoPath:/home/apr/tools/flacoco/target/dependency/* fr.spoonlabs.flacoco.cli.FlacocoMain --projectpath $projPath
It also failed with the same error.
This could be related to the way you add the dependencies to the classpath. How did you generate them?
Notes:
* the CLI usage of flacoco is obtained from [SpoonLabs/flacoco/wiki/CLI](https://github.com/SpoonLabs/flacoco/wiki/CLI) and [SpoonLabs/flacoco/wiki#usage-through-the-cli](https://github.com/SpoonLabs/flacoco/wiki#usage-through-the-cli), where it claims to use `fr.spoonlabs.flacoco.FlacocoMain` but actually we should use `fr.spoonlabs.flacoco.cli.FlacocoMain` seemingly.
Thanks. I've updated the information there.
Hi @andre15silva ,
Thank you very much for your prompt reply!
That is weird. I'm not able to reproduce such error.
I could no longer reproduce this error, either. That's a weird build. Anyway, I can mvn clean package -DskipTest now without any error.
This could be related to the way you add the dependencies to the classpath. How did you generate them?
I have added the classpath now, but a new error arises:
[0] INFO Flacoco - Running Flacoco...
[1538] INFO EntryPoint - Path to runner Classes: /home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar
[1546] INFO EntryPoint - /home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar
[1546] INFO EntryPoint - /home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar
[1546] INFO EntryPoint - /home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar
[1546] INFO EntryPoint - /home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar
[1547] INFO EntryPoint - /home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar
[1547] INFO EntryPoint - /home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar
[1547] INFO EntryPoint - /home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar
[1547] INFO EntryPoint - /home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar
[1548] INFO EntryPoint - /home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar
[1548] INFO EntryPoint - /home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar
[1548] INFO EntryPoint - /home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar
java.lang.RuntimeException: Forked process did not finish correctly. Timeout set was 1000000 ms, process took 1000000 ms before ending.
at eu.stamp_project.testrunner.EntryPoint.runGivenCommandLine(EntryPoint.java:764)
at eu.stamp_project.testrunner.EntryPoint.runOnlineCoveredTestResultPerTestMethods(EntryPoint.java:656)
at fr.spoonlabs.flacoco.core.coverage.framework.JUnit4Strategy.execute(JUnit4Strategy.java:27)
at fr.spoonlabs.flacoco.core.coverage.CoverageRunner.getCoverageMatrix(CoverageRunner.java:47)
at fr.spoonlabs.flacoco.localization.spectrum.SpectrumRunner.computeCoverageMatrix(SpectrumRunner.java:56)
at fr.spoonlabs.flacoco.localization.spectrum.SpectrumRunner.run(SpectrumRunner.java:32)
at fr.spoonlabs.flacoco.api.Flacoco.run(Flacoco.java:31)
at fr.spoonlabs.flacoco.cli.FlacocoMain.call(FlacocoMain.java:157)
at fr.spoonlabs.flacoco.cli.FlacocoMain.call(FlacocoMain.java:27)
at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
at picocli.CommandLine.access$1300(CommandLine.java:145)
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
at picocli.CommandLine.execute(CommandLine.java:2078)
at fr.spoonlabs.flacoco.cli.FlacocoMain.main(FlacocoMain.java:148)
My full command is as follows:
flacocoPath=/home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar
bug_dir="/home/apr/tests/fastjson"
src_java=${bug_dir}/"src/main/java/"
bin_java=${bug_dir}/"target/classes/"
bin_test=${bug_dir}/"target/test-classes/"
classpath=""
cd $bug_dir;
mvn dependency:copy-dependencies
cd -;
for file in ${bug_dir}/target/dependency/*
do
if [[ $file == *".jar" ]]; then
classpath="${classpath}:${file}"
fi
done
echo -e "java -jar $flacocoPath --projectpath $bug_dir --classpath ${bin_java}:${bin_test}$classpath \n\n"
java -jar $flacocoPath --projectpath $bug_dir --classpath ${bin_java}:${bin_test}$classpath
#flacocoPath=/home/apr/.m2/repository/com/github/spoonlabs/flacoco/1.0.0/flacoco-1.0.0.jar
#java -cp $flacocoPath:/home/apr/tools/flacoco/target/dependency/* fr.spoonlabs.flacoco.cli.FlacocoMain --projectpath $projPath
Much appreciated if any advice could be offered. Thank you!
I could no longer reproduce this error, either. That's a weird build. Anyway, I can mvn clean package -DskipTest now without any error.
Great!
This could be related to the way you add the dependencies to the classpath. How did you generate them?
I have added the classpath now, but a new error arises:
[0] INFO Flacoco - Running Flacoco... [1538] INFO EntryPoint - Path to runner Classes: /home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar [1546] INFO EntryPoint - /home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar [1546] INFO EntryPoint - /home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar [1546] INFO EntryPoint - /home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar [1546] INFO EntryPoint - /home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar [1547] INFO EntryPoint - /home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar [1547] INFO EntryPoint - /home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar [1547] INFO EntryPoint - /home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar [1547] INFO EntryPoint - /home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar [1548] INFO EntryPoint - /home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar [1548] INFO EntryPoint - /home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar [1548] INFO EntryPoint - /home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar java.lang.RuntimeException: Forked process did not finish correctly. Timeout set was 1000000 ms, process took 1000000 ms before ending. at eu.stamp_project.testrunner.EntryPoint.runGivenCommandLine(EntryPoint.java:764) at eu.stamp_project.testrunner.EntryPoint.runOnlineCoveredTestResultPerTestMethods(EntryPoint.java:656) at fr.spoonlabs.flacoco.core.coverage.framework.JUnit4Strategy.execute(JUnit4Strategy.java:27) at fr.spoonlabs.flacoco.core.coverage.CoverageRunner.getCoverageMatrix(CoverageRunner.java:47) at fr.spoonlabs.flacoco.localization.spectrum.SpectrumRunner.computeCoverageMatrix(SpectrumRunner.java:56) at fr.spoonlabs.flacoco.localization.spectrum.SpectrumRunner.run(SpectrumRunner.java:32) at fr.spoonlabs.flacoco.api.Flacoco.run(Flacoco.java:31) at fr.spoonlabs.flacoco.cli.FlacocoMain.call(FlacocoMain.java:157) at fr.spoonlabs.flacoco.cli.FlacocoMain.call(FlacocoMain.java:27) at picocli.CommandLine.executeUserObject(CommandLine.java:1953) at picocli.CommandLine.access$1300(CommandLine.java:145) at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352) at picocli.CommandLine$RunLast.handle(CommandLine.java:2346) at picocli.CommandLine$RunLast.handle(CommandLine.java:2311) at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179) at picocli.CommandLine.execute(CommandLine.java:2078) at fr.spoonlabs.flacoco.cli.FlacocoMain.main(FlacocoMain.java:148)
My full command is as follows:
flacocoPath=/home/apr/tools/flacoco/target/flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar bug_dir="/home/apr/tests/fastjson" src_java=${bug_dir}/"src/main/java/" bin_java=${bug_dir}/"target/classes/" bin_test=${bug_dir}/"target/test-classes/" classpath="" cd $bug_dir; mvn dependency:copy-dependencies cd -; for file in ${bug_dir}/target/dependency/* do if [[ $file == *".jar" ]]; then classpath="${classpath}:${file}" fi done echo -e "java -jar $flacocoPath --projectpath $bug_dir --classpath ${bin_java}:${bin_test}$classpath \n\n" java -jar $flacocoPath --projectpath $bug_dir --classpath ${bin_java}:${bin_test}$classpath #flacocoPath=/home/apr/.m2/repository/com/github/spoonlabs/flacoco/1.0.0/flacoco-1.0.0.jar #java -cp $flacocoPath:/home/apr/tools/flacoco/target/dependency/* fr.spoonlabs.flacoco.cli.FlacocoMain --projectpath $projPath
Much appreciated if any advice could be offered. Thank you!
You are getting this message because the the test execution takes longer than the default timeout value (1000000 ms).
To increase the timeout, you can use the following option:
--testRunnerTimeoutInMs=<testRunnerTimeoutInMs>
Timeout for each test execution with test-runner.
Must be greater than 0. Default value is 1000000
Thank you for the helpful guidance. But it's kinda weird that while mvn test
only takes about two minutes, the localization process by flacoco takes over 1000s (about 16 minutes). What are the reasons for such time difference? Seemingly It is understandable, to a certain extent, that collecting the matrix might bring extra overhead.
I wonder if there is any approach or workaround to reduce the time cost of flacoco, especially for real-world large-scale projects? Please kindly advise. Thank you!
Hi @DehengYang very good to meet you again and welcome to flacoco!
FYI, we're finishing a paper about flacoco, we'll let you know as soon as it's ready.
Hi Prof. @monperrus , thank you for the information and looking forward to that paper!
Hi @DehengYang ,
Sorry for the delayed response.
Thank you for the helpful guidance. But it's kinda weird that while
mvn test
only takes about two minutes, the localization process by flacoco takes over 1000s (about 16 minutes). What are the reasons for such time difference? Seemingly It is understandable, to a certain extent, that collecting the matrix might bring extra overhead.
Although there is some overhead, I'm surprised to see such a big difference. I will do some profiling efforts to see what is happening. Thanks for the information!
I wonder if there is any approach or workaround to reduce the time cost of flacoco, especially for real-world large-scale projects? Please kindly advise. Thank you!
There are a couple of ways of reducing the overhead, although it shouldn't be required for an usual case:
--junit4tests
and --junit5tests
options to manually set which tests to run)--jacocoExcludes
and --jacocoIncludes
). This would override the automatic logic in place to select which classes are interesting, so you could further limit the scope of JaCoCo's, and therefore flacoco's, action.Hi @andre15silva , thank you for the great help and useful guidance. I will try that soon.
Dear Sirs,
I am very interested in this project and I am trying to package it on my pc for fault localization. I cloned this project and then ran
mvn package -DskipTests
, but it reports the following two types of compilation errors:I think this could be resolved by adding the following dependencies into
pom.xml
:With this workaround, I have used the
flacoco-1.0.1-SNAPSHOT-jar-with-dependencies.jar
to localize a bug in https://github.com/alibaba/fastjson/commit/96d0ee532a37351a126ebef2d5071964c33a1ec7 with the following command:It failed with the following error:
Then I used the jar from the maven repo:
It also failed with the same error.
Notes:
fr.spoonlabs.flacoco.FlacocoMain
but actually we should usefr.spoonlabs.flacoco.cli.FlacocoMain
seemingly.It would be much appreciated if any advice could be provided. Thank you.