WebFuzzing / EvoMaster

The first open-source AI-driven tool for automatically generating system-level test cases (also known as fuzzing) for web/enterprise applications. Currently targeting whitebox and blackbox testing of Web APIs, like REST, GraphQL and RPC (e.g., gRPC and Thrift).
GNU Lesser General Public License v3.0
515 stars 85 forks source link

StackOverFlowError when I try to run the tests generated by evomaster #959

Closed victorramalhocognizant closed 1 week ago

victorramalhocognizant commented 6 months ago

StackOverFlowError when I try to run the tests generated by evomaster executing (same for mvn test)

apparently a recurrence of tests with the EMDriver class

EMVersion: 2.0.0

java.lang.StackOverflowError at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCFieldAccess.accept(JCTree.java:2450) at jdk.compiler/com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:49) at jdk.compiler/com.sun.tools.javac.tree.TreeScanner.visitApply(TreeScanner.java:247) at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCMethodInvocation.accept(JCTree.java:1797) at jdk.compiler/com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:49) at jdk.compiler/com.sun.tools.javac.tree.TreeScanner.visitSelect(TreeScanner.java:331) at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCFieldAccess.accept(JCTree.java:2450) at jdk.compiler/com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:49) at jdk.compiler/com.sun.tools.javac.tree.TreeScanner.visitApply(TreeScanner.java:247) at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCMethodInvocation.accept(JCTree.java:1797) at jdk.compiler/com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:49) at jdk.compiler/com.sun.tools.javac.tree.TreeScanner.visitSelect(TreeScanner.java:331) at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCFieldAccess.accept(JCTree.java:2450) at jdk.compiler/com.sun.tools.javac.tree.TreeScanner.scan(TreeScanner.java:49) at jdk.compiler/com.sun.tools.javac.tree.TreeScanner.visitApply(TreeScanner.java:247) ..... ......

victorramalhocognizant commented 6 months ago

some informations that should be util:

EMDriver:

`import java.util.Arrays; import java.util.List; import java.util.Map;

import org.evomaster.client.java.controller.AuthUtils; import org.evomaster.client.java.controller.EmbeddedSutController; import org.evomaster.client.java.controller.InstrumentedSutStarter; import org.evomaster.client.java.controller.api.dto.AuthenticationDto; import org.evomaster.client.java.controller.api.dto.SutInfoDto; import org.evomaster.client.java.controller.internal.db.DbSpecification; import org.evomaster.client.java.controller.problem.ProblemInfo; import org.evomaster.client.java.controller.problem.RestProblem; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.boot.autoconfigure.domain.EntityScan; import org.springframework.context.ConfigurableApplicationContext; import org.springframework.data.jpa.repository.config.EnableJpaRepositories;

import io.swagger.annotations.Api; import springfox.documentation.swagger2.annotations.EnableSwagger2;

@Api(value = "API REST system") @EnableSwagger2 @SpringBootApplication(scanBasePackages="br.system") @EntityScan("br.system") @EnableJpaRepositories("br.system") public class EMDriver extends EmbeddedSutController{ protected static String[] argumentos;

 public static void main(String[] args){
     argumentos=args;
        int port = 40100;

        EMDriver controller = new EMDriver(port);
        InstrumentedSutStarter starter = new InstrumentedSutStarter(controller);

        starter.start();
    }

 public EMDriver() {
        this(40100);
    }

    public EMDriver(int port) {
        setControllerPort(port);
    }
    protected int getSutPort() {
        return (Integer) ((Map) ctx.getEnvironment()
                .getPropertySources().get("server.ports").getSource())
                .get("local.server.port");
    }
@Override
public String startSut() {
    ctx = SpringApplication.run(SystemApplication.class, argumentos);

    return "http://localhost:" + getSutPort();
}

@Override
public void stopSut() {
     ctx.stop();
}

@Override
public void resetStateOfSUT() {
    // TODO Auto-generated method stub

}

private ConfigurableApplicationContext ctx;

@Override
public boolean isSutRunning() {
    return ctx!=null && ctx.isRunning();
}

@Override
public String getPackagePrefixesToCover() {
    return "br.ufrn.system.";
}

@Override
public List<AuthenticationDto> getInfoForAuthentication() {
    return Arrays.asList(
            AuthUtils.getForBasic("admin","system","$2a$10$system.Zcsystem3KirsystemGB/FPq")
    );
}

@Override
 public ProblemInfo getProblemInfo() {
    return new RestProblem("http://localhost:8080/system/v2/api-docs", null);
}

@Override
public SutInfoDto.OutputFormat getPreferredOutputFormat() {
    return SutInfoDto.OutputFormat.JAVA_JUNIT_5;
}

@Override
public List<DbSpecification> getDbSpecifications() {
    // TODO Auto-generated method stub
    return null;
}

} `

Example of test class `import org.junit.jupiter.api.AfterAll; import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.Timeout; import static org.junit.jupiter.api.Assertions.; import java.util.Map; import java.util.List; import static org.evomaster.client.java.controller.api.EMTestUtils.; import org.evomaster.client.java.controller.SutHandler; import io.restassured.RestAssured; import static io.restassured.RestAssured.given; import io.restassured.response.ValidatableResponse; import static org.hamcrest.Matchers.; import io.restassured.config.JsonConfig; import io.restassured.path.json.config.JsonPathConfig; import static io.restassured.config.RedirectConfig.redirectConfig; import static org.evomaster.client.java.controller.contentMatchers.NumberMatcher.; import static org.evomaster.client.java.controller.contentMatchers.StringMatcher.; import static org.evomaster.client.java.controller.contentMatchers.SubStringMatcher.; import static org.evomaster.client.java.controller.expect.ExpectationHandler.expectationHandler; import org.evomaster.client.java.controller.expect.ExpectationHandler; import io.restassured.path.json.JsonPath; import java.util.Arrays;

public class EvoMaster_successes_Test {

private static final SutHandler controller = new EMDriver();
private static String baseUrlOfSut;
/** [ems] - expectations master switch - is the variable that activates/deactivates expectations individual test cases
* by default, expectations are turned off. The variable needs to be set to [true] to enable expectations
*/
private static boolean ems = false;
/**
* sco - supported code oracle - checking that the response status code is among those supported according to the schema
*/
private static boolean sco = false;
/**
* rso - response structure oracle - checking that the response objects match the responses defined in the schema
*/
private static boolean rso = false;

@BeforeAll
public static void initClass() {
    controller.setupForGeneratedTest();
    baseUrlOfSut = controller.startSut();
    controller.registerOrExecuteInitSqlCommandsIfNeeded();
    assertNotNull(baseUrlOfSut);
    RestAssured.enableLoggingOfRequestAndResponseIfValidationFails();
    RestAssured.useRelaxedHTTPSValidation();
    RestAssured.urlEncodingEnabled = false;
    RestAssured.config = RestAssured.config()
        .jsonConfig(JsonConfig.jsonConfig().numberReturnType(JsonPathConfig.NumberReturnType.DOUBLE))
        .redirect(redirectConfig().followRedirects(false));
}

@AfterAll
public static void tearDown() {
    controller.stopSut();
}

@BeforeEach
public void initTest() {
    controller.resetStateOfSUT();
}

@Test @Timeout(60)
public void test_55() throws Exception {

    given().accept("*/*")
            .header("x-EMextraHeader123", "")
            .get(baseUrlOfSut + "/endpoint?EMextraParam123=_EM_205_XYZ_")
            .then()
            .statusCode(200)
            .assertThat()
            .contentType("application/json")
            .body("size()", equalTo(0));

}
`
arcuri82 commented 6 months ago

Hi,

technically, this is not a bug in EvoMaster, but rather an issue in the JDK. But we have experienced this before, so likely we should write some documentation for it.

You will need to use something like -Xss10m option (or higher value).

For IntelliJ, you can look here

For Maven, you could add it to MAVEN_OPTS environment variable.

Let me know if this fix your issues. If so, I will write some documentation for it.

victorramalhocognizant commented 6 months ago

I tried but I'm facing the same problem

Apparently it's a recursion problem

arcuri82 commented 6 months ago

just to be on safe side, when you said "tried", did you re-start the shell/process (eg a Bash terminal, or an IDE like IntelliJ or Eclipse) you ran Maven from? changing/setting environment variables has not effect on running processes

victorramalhocognizant commented 6 months ago

I changed the IntelliJ configuration inserting -Xss10m and then I restarted everything (IntelliJ, windows,etc) but I'm facing the same problem

arcuri82 commented 6 months ago

hi, just to be sure, in IntelliJ you have something like the following? (Note the -Xss8m there) It was a while ago, but I think I had the same issue, and that configuration fixed it. Maybe you might want to try something bigger like -Xss50m

image

victorramalhocognizant commented 6 months ago

my cfg image

arcuri82 commented 6 months ago

to be honest... I don't know... what are full logs you get when compiling? can you upload the full text of a test case that lead to such error?

also, why do you have the following in the test driver? it seems unnecessary

@Api(value = "API REST system")
@EnableSwagger2
@SpringBootApplication(scanBasePackages="br.system")
@EntityScan("br.system")
@EnableJpaRepositories("br.system")
arcuri82 commented 1 week ago

closed as no further info provided. cannot reproduce.