EvoSuite / evosuite

EvoSuite - automated generation of JUnit test suites for Java classes
http://www.evosuite.org
GNU Lesser General Public License v3.0
823 stars 341 forks source link

Inconsistent Test Generation for Custom Methods in EvoSuite #464

Open Juvemanu opened 7 months ago

Juvemanu commented 7 months ago

Context

Please provide below a detailed introduction to the issue itself, and describe what you were doing when the issue happened. Or, what do you want to achieve?

Context I am encountering an issue with EvoSuite 1.0.6 while generating tests for the tutorial.Stack class. The source code is based on the EvoSuite tutorial part 1, with the addition of two methods: is9 and is10. When running the command java -jar .\evosuite-1.0.6.jar -class tutorial.Stack -projectCP=target/classes -Dtest_comments=true -criterion branch, EvoSuite generates tests that include unrelated methods (is10 and isEmpty) along with the target method is9. This is problematic because it makes it challenging to discern which method is being tested in each generated test case.

Steps to Reproduce

Please break down here below all the needed steps to reproduce the issue. [If possible, please upload an example of the project you are generating tests for.]

Utilize the source code from the EvoSuite tutorial part 1 for the tutorial.Stack class. Add the following methods (is9 and is10) to the existing code:

public boolean is10() {
    return capacity == 9;
}

public boolean is9(int a, int b) {
    return a + b == 9;
}

Navigate to the project directory. Execute the following EvoSuite command: java -jar .\evosuite-1.0.6.jar -class tutorial.Stack -projectCP=target/classes -Dtest_comments=true -criterion branch

EvoSuite Arguments

Please provide the whole EvoSuite commmand you executed (if relevant)

java -jar .\evosuite-1.0.6.jar -class tutorial.Stack -projectCP=target/classes -Dtest_comments=true -criterion branch

Current Result

Please describe here below the current result you got (if relevant) [if relevant, include a screenshot]

EvoSuite generates a test case (test0) that covers goals for methods is10and isEmptyin addition to the target method is9. This is causing confusion as the focus is on testing is9, and the inclusion of unrelated methods is not expected.

//Test case number: 0
  /*
   * 2 covered goals:
   * Goal 1. tutorial.Stack.<init>()V: root-Branch
   * Goal 2. tutorial.Stack.is9(II)Z: I6 Branch 5 IF_ICMPNE L28 - false
   */

  @Test(timeout = 4000)
  public void test0()  throws Throwable  {
      Stack<Object> stack0 = new Stack<Object>();
      boolean boolean0 = stack0.is9(9, 0);
      assertTrue(boolean0);
      assertFalse(stack0.is10());
      assertTrue(stack0.isEmpty());
  }

Expected result

Please describe here below what should be the expected behaviour (if relevant)

The generated test cases should only focus on testing the target method is9and not include unrelated methods such as is10and isEmpty.

Additional info

Please add any information of interest here below It is observed that the -Dtarget_method option does not seem to work in this version, making it challenging to specify the target method for testing. #455