Unsupported fitness function error in several SystemTests

fc51111 commented 2 years ago

Context

I Tried to run the NullStringSystemTest system test (as described in the part 4 of the EvoSuite tutorial), but it failed (see the error in the "Current Result" section). Meanwhile, I found several other tests with the same problem, and they were all system tests.

Steps to Reproduce

Run the NullStringSystemTest (./master/src/test/java/org/evosuite/basic/NullStringSystemTest.java) system test using EvoSuite v 1.2.0.

Current Result

WARN  evo_logger - Originally, MOSA was implemented with a 'RANK_CROWD_DISTANCE_TOURNAMENT' selection function. You may want to consider using it.
ERROR org.evosuite.rmi.service.ClientNodeImpl - Error when generating tests for: com.examples.with.different.packagename.NullString with seed 8. Configuration id : null
java.lang.IllegalArgumentException: Unsupported type of fitness function: class org.evosuite.coverage.branch.BranchCoverageSuiteFitness
    at org.evosuite.ga.metaheuristics.TestSuiteAdapter.mapFitnessFunctionToTestCaseLevel(TestSuiteAdapter.java:651)
    at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
    at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
    at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
    at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
    at org.evosuite.ga.metaheuristics.TestSuiteAdapter.addFitnessFunctions(TestSuiteAdapter.java:658)
    at org.evosuite.strategy.WholeTestSuiteStrategy.generateTests(WholeTestSuiteStrategy.java:66)
    at org.evosuite.TestSuiteGenerator.generateTests(TestSuiteGenerator.java:630)
    at org.evosuite.TestSuiteGenerator.generateTestSuite(TestSuiteGenerator.java:208)
    at org.evosuite.rmi.service.ClientNodeImpl.lambda$startNewSearch$0(ClientNodeImpl.java:140)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
ERROR org.evosuite.statistics.SearchStatistics - No statistics has been saved because EvoSuite failed to generate any test case
ERROR org.evosuite.executionmode.TestGeneration - failed to write statistics data

java.lang.AssertionError
    at org.evosuite.SystemTestBase.getGAFromResult(SystemTestBase.java:353)
    at org.evosuite.basic.NullStringSystemTest.testNullString(NullStringSystemTest.java:46)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
    at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
    at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
    at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
    at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
    at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
    at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
    at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
    at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:221)
    at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)

Expected result

All tests should have passed.

Additional info

Adding the following line of code to these tests appears to fix this problem (though there are still warning messages related to “an illegal reflective access”): Properties.ALGORITHM = Properties.Algorithm.MONOTONIC_GA;
I am currently using EvoSuite v 1.2.0. on Windows 10.
jdk 1.8.0_212

saiema commented 2 years ago

Hi, I also have this issue, running everything from maven (from a terminal) seems to end without any issue. But as soon as I try any test withing IntelliJ (after importing the project as a maven project).

Echo009 commented 2 years ago

mark

zzctmac commented 2 years ago

This is because the default algorithm now is DynaMOSA. But these old system tests use the WholeSuiteGeneration strategy, which doesn't support DynaMOSA. So it can be fixed in tests by adding the parameter "-Dalgorithm=MONOTONIC_GA".

saiema commented 2 years ago

This is because the default algorithm now is DynaMOSA. But these old system tests use the WholeSuiteGeneration strategy, which doesn't support DynaMOSA. So it can be fixed in tests by adding the parameter "-Dalgorithm=MONOTONIC_GA".

Then this is an issue about test configuration that must be updated right?

EhsanMashhadi commented 2 years ago

Any new update?

saiema commented 2 years ago

Any new update?

I tried with @zzctmac suggestion and it works. But this feels like a workaround that shouldn't be necessary.

EhsanMashhadi commented 2 years ago

@saiema It works, but in this case, the EvoSuite strategy will be used not the MOSuiteStrategy.

EvoSuite / evosuite