search

EvoSuite / evosuite

EvoSuite - automated generation of JUnit test suites for Java classes
http://www.evosuite.org
GNU Lesser General Public License v3.0
832 stars 341 forks source link

When I use the docker image ; I get an error during compiling and checking tests #360

Closed JopSchaap closed 3 years ago

JopSchaap commented 3 years ago

Context

I tried running the docker image evosuite/evosuite:1.1.0-java -11 from dockerhub. It seems to complete every step as it should except for compiling and checking tests. When I tried running Evosuite through the jar file I did not receive an error and the entire test suite was generated.

Steps to Reproduce

Project I am using. level0.zip

  1. I compiled the source code gradle build.
  2. Then I run docker run -v "${PWD}:/evosuite" evosuite/evosuite:1.1.0-java-11 -class Level -target build/classes/java/main

EvoSuite Arguments

docker run -v "${PWD}:/evosuite" evosuite/evosuite:1.1.0-java-11 -class Level -target build/classes/java/main

Current Result

Compiling and checking tests
[MASTER] 12:13:59.503 [logback-2] ERROR JUnitAnalyzer - Issue in scaffolding of the test suite: /usr/local/openjdk-11/lib/libfontmanager.so: libfreetype.so.6: cannot open shared object file: No such file or directory
Stack trace:
java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method)
java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2442)
java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2498)
java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2694)
java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2648)
java.base/java.lang.Runtime.loadLibrary0(Runtime.java:830)
java.base/java.lang.System.loadLibrary(System.java:1870)
java.desktop/sun.font.FontManagerNativeLibrary$1.run(FontManagerNativeLibrary.java:57)
java.base/java.security.AccessController.doPrivileged(Native Method)
java.desktop/sun.font.FontManagerNativeLibrary.<clinit>(FontManagerNativeLibrary.java:32)
java.desktop/sun.font.SunFontManager$1.run(SunFontManager.java:270)
java.base/java.security.AccessController.doPrivileged(Native Method)
java.desktop/sun.font.SunFontManager.<clinit>(SunFontManager.java:266)
java.desktop/sun.font.FontDesignMetrics.getMetrics(FontDesignMetrics.java:265)
java.desktop/sun.swing.SwingUtilities2.getFontMetrics(SwingUtilities2.java:1243)
java.desktop/javax.swing.JComponent.getFontMetrics(JComponent.java:1646)
org.evosuite.runtime.GuiSupport.initialize(GuiSupport.java:96)
Level_4_tmp__ESTest_scaffolding.initEvoSuiteFramework(Level_4_tmp__ESTest_scaffolding.java:30)
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.base/java.lang.reflect.Method.invoke(Method.java:566)
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33)
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
org.junit.runners.ParentRunner.run(ParentRunner.java:413)
org.junit.runners.Suite.runChild(Suite.java:128)
org.junit.runners.Suite.runChild(Suite.java:27)
org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
org.junit.runners.ParentRunner.run(ParentRunner.java:413)
org.junit.runner.JUnitCore.run(JUnitCore.java:137)
org.junit.runner.JUnitCore.run(JUnitCore.java:115)
org.junit.runner.JUnitCore.run(JUnitCore.java:105)
org.junit.runner.JUnitCore.run(JUnitCore.java:94)
org.evosuite.junit.JUnitAnalyzer.runJUnitOnCurrentProcess(JUnitAnalyzer.java:306)
org.evosuite.junit.JUnitAnalyzer.runTests(JUnitAnalyzer.java:276)
org.evosuite.junit.JUnitAnalyzer.handleTestsThatAreUnstable(JUnitAnalyzer.java:178)
org.evosuite.TestSuiteGenerator.compileAndCheckTests(TestSuiteGenerator.java:571)
org.evosuite.TestSuiteGenerator.postProcessTests(TestSuiteGenerator.java:516)
org.evosuite.TestSuiteGenerator.generateTestSuite(TestSuiteGenerator.java:215)
org.evosuite.rmi.service.ClientNodeImpl.lambda$startNewSearch$0(ClientNodeImpl.java:150)
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base/java.lang.Thread.run(Thread.java:834)

[MASTER] 12:13:59.637 [logback-2] ERROR JUnitAnalyzer - Issue in scaffolding of the test suite: Could not initialize class sun.font.SunFontManager
Stack trace:
java.desktop/sun.font.FontDesignMetrics.getMetrics(FontDesignMetrics.java:265)
java.desktop/sun.swing.SwingUtilities2.getFontMetrics(SwingUtilities2.java:1243)
java.desktop/javax.swing.JComponent.getFontMetrics(JComponent.java:1646)
org.evosuite.runtime.GuiSupport.initialize(GuiSupport.java:96)
Level_5_tmp__ESTest_scaffolding.initEvoSuiteFramework(Level_5_tmp__ESTest_scaffolding.java:30)
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.base/java.lang.reflect.Method.invoke(Method.java:566)
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33)
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
org.junit.runners.ParentRunner.run(ParentRunner.java:413)
org.junit.runners.Suite.runChild(Suite.java:128)
org.junit.runners.Suite.runChild(Suite.java:27)
org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
org.junit.runners.ParentRunner.run(ParentRunner.java:413)
org.junit.runner.JUnitCore.run(JUnitCore.java:137)
org.junit.runner.JUnitCore.run(JUnitCore.java:115)
org.junit.runner.JUnitCore.run(JUnitCore.java:105)
org.junit.runner.JUnitCore.run(JUnitCore.java:94)
org.evosuite.junit.JUnitAnalyzer.runJUnitOnCurrentProcess(JUnitAnalyzer.java:306)
org.evosuite.junit.JUnitAnalyzer.runTests(JUnitAnalyzer.java:276)
org.evosuite.junit.JUnitAnalyzer.handleTestsThatAreUnstable(JUnitAnalyzer.java:178)
org.evosuite.TestSuiteGenerator.compileAndCheckTests(TestSuiteGenerator.java:571)
org.evosuite.TestSuiteGenerator.postProcessTests(TestSuiteGenerator.java:516)
org.evosuite.TestSuiteGenerator.generateTestSuite(TestSuiteGenerator.java:215)
org.evosuite.rmi.service.ClientNodeImpl.lambda$startNewSearch$0(ClientNodeImpl.java:150)
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base/java.lang.Thread.run(Thread.java:834)

[MASTER] 12:13:59.752 [logback-2] ERROR JUnitAnalyzer - Issue in scaffolding of the test suite: Could not initialize class sun.font.SunFontManager
Stack trace:
java.desktop/sun.font.FontDesignMetrics.getMetrics(FontDesignMetrics.java:265)
java.desktop/sun.swing.SwingUtilities2.getFontMetrics(SwingUtilities2.java:1243)
java.desktop/javax.swing.JComponent.getFontMetrics(JComponent.java:1646)
org.evosuite.runtime.GuiSupport.initialize(GuiSupport.java:96)
Level_6_tmp__ESTest_scaffolding.initEvoSuiteFramework(Level_6_tmp__ESTest_scaffolding.java:30)
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.base/java.lang.reflect.Method.invoke(Method.java:566)
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33)
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
org.junit.runners.ParentRunner.run(ParentRunner.java:413)
org.junit.runners.Suite.runChild(Suite.java:128)
org.junit.runners.Suite.runChild(Suite.java:27)
org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
org.junit.runners.ParentRunner.run(ParentRunner.java:413)
org.junit.runner.JUnitCore.run(JUnitCore.java:137)
org.junit.runner.JUnitCore.run(JUnitCore.java:115)
org.junit.runner.JUnitCore.run(JUnitCore.java:105)
org.junit.runner.JUnitCore.run(JUnitCore.java:94)
org.evosuite.junit.JUnitAnalyzer.runJUnitOnCurrentProcess(JUnitAnalyzer.java:306)
org.evosuite.junit.JUnitAnalyzer.runTests(JUnitAnalyzer.java:276)
org.evosuite.junit.JUnitAnalyzer.handleTestsThatAreUnstable(JUnitAnalyzer.java:178)
org.evosuite.TestSuiteGenerator.compileAndCheckTests(TestSuiteGenerator.java:571)
org.evosuite.TestSuiteGenerator.postProcessTests(TestSuiteGenerator.java:516)
org.evosuite.TestSuiteGenerator.generateTestSuite(TestSuiteGenerator.java:215)
org.evosuite.rmi.service.ClientNodeImpl.lambda$startNewSearch$0(ClientNodeImpl.java:150)
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base/java.lang.Thread.run(Thread.java:834)

[MASTER] 12:13:59.850 [logback-2] ERROR JUnitAnalyzer - Issue in scaffolding of the test suite: Could not initialize class sun.font.SunFontManager
Stack trace:
java.desktop/sun.font.FontDesignMetrics.getMetrics(FontDesignMetrics.java:265)
java.desktop/sun.swing.SwingUtilities2.getFontMetrics(SwingUtilities2.java:1243)
java.desktop/javax.swing.JComponent.getFontMetrics(JComponent.java:1646)
org.evosuite.runtime.GuiSupport.initialize(GuiSupport.java:96)
Level_7_tmp__ESTest_scaffolding.initEvoSuiteFramework(Level_7_tmp__ESTest_scaffolding.java:30)
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.base/java.lang.reflect.Method.invoke(Method.java:566)
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33)
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
org.junit.runners.ParentRunner.run(ParentRunner.java:413)
org.junit.runners.Suite.runChild(Suite.java:128)
org.junit.runners.Suite.runChild(Suite.java:27)
org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
org.junit.runners.ParentRunner.run(ParentRunner.java:413)
org.junit.runner.JUnitCore.run(JUnitCore.java:137)
org.junit.runner.JUnitCore.run(JUnitCore.java:115)
org.junit.runner.JUnitCore.run(JUnitCore.java:105)
org.junit.runner.JUnitCore.run(JUnitCore.java:94)
org.evosuite.junit.JUnitAnalyzer.runJUnitOnCurrentProcess(JUnitAnalyzer.java:306)
org.evosuite.junit.JUnitAnalyzer.runTests(JUnitAnalyzer.java:276)
org.evosuite.junit.JUnitAnalyzer.handleTestsThatAreUnstable(JUnitAnalyzer.java:178)
org.evosuite.TestSuiteGenerator.compileAndCheckTests(TestSuiteGenerator.java:571)
org.evosuite.TestSuiteGenerator.postProcessTests(TestSuiteGenerator.java:516)
org.evosuite.TestSuiteGenerator.generateTestSuite(TestSuiteGenerator.java:215)
org.evosuite.rmi.service.ClientNodeImpl.lambda$startNewSearch$0(ClientNodeImpl.java:150)
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base/java.lang.Thread.run(Thread.java:834)

* Writing tests to file
* Writing JUnit test case 'Level_ESTest' to evosuite-tests
* Done!

* Computation finished
jop@jops-linux-komputer:/mnt/Windows/Users/jopsc/Documents/uni/second year/SP/tower-defense-for-fuzzing-and-security/server/games/TESTS$ sudo docker run -v "${PWD}:/evosuite" evosuite/evosuite:1.1.0-java-8 -class Level -target build/classes/java/main
Unable to find image 'evosuite/evosuite:1.1.0-java-8' locally
1.1.0-java-8: Pulling from evosuite/evosuite
afb6ec6fdc1c: Already exists 
ee19e84e8bd1: Already exists 
832173019a27: Already exists 
9ffa71fb0db8: Already exists 
329a9e2db8f8: Already exists 
9b57233a932b: Already exists 
Digest: sha256:6293610b4cfa56ca68f5960cd031d5987063a2efcafc47ce769dbfb78b58fb12
Status: Downloaded newer image for evosuite/evosuite:1.1.0-java-8
* EvoSuite 1.1.0
* Going to generate test cases for class: Level
* Starting Client-0
* Connecting to master process on port 16953
* Analyzing classpath: 
  - build/classes/java/main
* Error while initializing target class: Level has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0
* Computation finished
[MASTER] 12:20:08.519 [logback-2] ERROR TestSuiteGenerator - Problem for Level. Full stack:
java.lang.ClassNotFoundException: Level has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0
        at org.evosuite.instrumentation.InstrumentingClassLoader.instrumentClass(InstrumentingClassLoader.java:164)
        at org.evosuite.instrumentation.InstrumentingClassLoader.loadClass(InstrumentingClassLoader.java:132)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:348)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.evosuite.testcase.statements.MethodStatement$1.execute(MethodStatement.java:261)
        at org.evosuite.testcase.statements.AbstractStatement.exceptionHandler(AbstractStatement.java:172)
        at org.evosuite.testcase.statements.MethodStatement.execute(MethodStatement.java:224)
        at org.evosuite.testcase.execution.TestRunnable.executeStatements(TestRunnable.java:295)
        at org.evosuite.testcase.execution.TestRunnable.call(TestRunnable.java:201)
        at org.evosuite.testcase.execution.TestRunnable.call(TestRunnable.java:54)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.UnsupportedClassVersionError: Level has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:635)
        at org.evosuite.instrumentation.InstrumentingClassLoader.instrumentClass(InstrumentingClassLoader.java:157)
        ... 17 common frames omitted
[MASTER] 12:20:08.651 [main] ERROR SearchStatistics - No statistics has been saved because EvoSuite failed to generate any test case
[MASTER] 12:20:08.752 [main] ERROR TestGeneration - failed to write statistics data

Additional info

gofraser commented 3 years ago

@mitchellolsthoorn any idea?

mitchellolsthoorn commented 3 years ago

@gofraser from searching online (https://github.com/docker-library/openjdk/issues/335, https://github.com/docker-library/openjdk/issues/333, https://github.com/docker-library/openjdk/pull/322, https://github.com/docker-library/openjdk/issues/320), the openjdk docker base image removed the libfontmanager.so from the slim image. Apparently, this library is used within the sandbox of EvoSuite. Like @JopSchaap already indicated, the linked issues indicate that we have to switch to the non-slim variants. The only thing is that the images will become 90MB bigger.

Shall I make a quick PR?

gofraser commented 3 years ago

I don't think the issue is in the sandbox but in the GuiManager which tries to force-load the fonts once initially (the sandbox just explicitly allows this to happen), but in any case if using the non-slim variant solves the issue then fine! If it's just a matter of changing to FROM openjdk:11 then I can do this, if it needs more testing a MR would be welcome!

mitchellolsthoorn commented 3 years ago

@gofraser I tested it locally on some projects and the code used in the issue. The linked PR (https://github.com/EvoSuite/evosuite/pull/374) fixes the problem. Since we don't have the automatic image set up, I will update the latest images on Docker Hub and back port it to the older versions.