ClassNotFoundException error in continuous mode / maven plugin with Java 9+(JDK 11)

[micskeiz]

Context

• I tried the use EvoSuite 1.1.1-SNAPSHOT on JDK 11 with Maven on the files from the EvoSuite Maven Tutorial
• Test generation is not started because of a ClassNotFoundException for the CUTs of the tutorial
• As far as I could debug the problem is that continuous mode (which is called from the EvoSuite Maven plugin) uses ProjectAnalyzer to check classes.
• The target parameter is correctly passed to Continuous, it tries to use ClassPathHacker to add files in the target classes to the actual classpath (which is currently EvoSuite's classpath and not the target's), but it failes because in Java 9+ the default classloader is not an URLClassLoader.
• Therefore the classpath remains the original one, and ProjectAnalyzer will fail during Class<?> clazz = Class.forName(className);.

Relevant part of the log:

[INFO] [MASTER] 22:59:12.444 [main] ERROR ClassPathHacker - Trying to add URL to class path:file:/C:/code/Tutorial_Maven/target/test-classes/
[INFO] [MASTER] 22:59:12.445 [main] ERROR ClassPathHacker - Did not add file:/C:/code/Tutorial_Maven/target/test-classes/, because system class loader is no URLClassLoader
...
[INFO] [MASTER] 21:52:09.636 [main] ERROR ProjectAnalyzer - java.lang.ClassNotFoundException: tutorial.LinkedList
[INFO] java.lang.ClassNotFoundException: tutorial.LinkedList
[INFO]  at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583)
[INFO]  at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
[INFO]  at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
[INFO]  at java.base/java.lang.Class.forName0(Native Method)
[INFO]  at java.base/java.lang.Class.forName(Class.java:332)
[INFO]  at org.evosuite.continuous.project.ProjectAnalyzer.getCutsToAnalyze(ProjectAnalyzer.java:179)
[INFO]  at org.evosuite.continuous.project.ProjectAnalyzer.analyze(ProjectAnalyzer.java:210)
[INFO]  at org.evosuite.continuous.ContinuousTestGeneration.execute(ContinuousTestGeneration.java:157)
[INFO]  at org.evosuite.executionmode.Continuous.execute(Continuous.java:138)
[INFO]  at org.evosuite.EvoSuite.parseCommandLine(EvoSuite.java:351)
[INFO]  at org.evosuite.EvoSuite.main(EvoSuite.java:115)

(I have added a few log messages during debug to the source, thus some of the line numbers changed.)

• If I start the test generation from the CLI without continuous mode, then tests are generated. As far as I can see this is because the other mode (TestGeneration.executeTestGeneration) skips ProjectAnalyzer. It also tries to use ClassPathHacker (which fails also in this mode), but uses ResourceList to get the class. This suceed, because it uses the SUT class loader from TestGenerationContext.

Maybe ProjectAnalyzer could also rely on ResourceList to get the classes and check whether they are tests. (For this, a Class instance is required, but perhaphs the output of getClassAsStream can be converted (like in InstrumentingClassLoader).

Steps to Reproduce

1. Use openJDK 11 (but I guess it is the same for any 9+ JDK)
2. Get the source of EvoSuite, build and install it locally with mvn install (as currently 1.1.0 is not yet on Maven Central)
3. Get the maven tutorial project, add plugin to pom with 1.1.1-SNAPSHOT version
4. Try to generate tests: mvn evosuite:generate
5. Test generation will fail for all 4 classes in the tutorial

EvoSuite Arguments

Please provide the whole EvoSuite commmand you executed (if relevant)

mvn evosuite:generate

it translates to:

-continuous execute -target C:\code\Tutorial_Maven\target\classes -Dcriterion=LINE:BRANCH:EXCEPTION:WEAKMUTATION:OUTPUT:METHOD:METHODNOEXCEPTION:CBRANCH
-Dctg_schedule=BUDGET -Dctg_memory=800 -Dctg_cores=1 -Dspawn_process_manager_port=50872 -Dctg_time_per_class=1
-DCP_file_path=C:\Users\user\AppData\Local\Temp\EvoSuite_classpathFile14299949377027317047.txt

Current Result

Please describe here below the current result you got (if relevant)

• Exception: [main] ERROR ProjectAnalyzer - java.lang.ClassNotFoundException: tutorial.LinkedListIterator
• See relevant log above

Expected result

Please describe here below what should be the expected behaviour (if relevant)

• Test generation starts

Additional info

Please add any information of interest here below

• OS: Windows 10 (but I tried it also on Ubuntu Linux)

[licongchao]

I had the same problem. Any solution?

[VoglSebastian]

I think this issue was fixed with aa606bb8bf4924a5696bbce4cf332f7d89e5bd60, but i am not sure.

If this is still relevant i was not able to reproduce the Bug. Can anyone confirm that this is fixed?