Error in EvoMaster 3.1.0: Black-Box Testing Initialization Failure with SSLException

[sathiyajith]

• EvoMaster version: 3.1.0
• EM is run as a Linux Installer - evomaster_3.1.0_amd64.deb
• JDK - 11.0.24
• Log

-(venv) # evomaster --blackBox true --bbSwaggerUrl file:/restler_server.yaml --bbTargetUrl https://localhost:8888/docs --outputFormat JAVA_JUNIT_4 --maxTime 30s --ratePerMinute 60

• EvoMaster version: 3.1.0
• Loading configuration file from: /em.yaml
• WARNING: You are doing Black-Box testing, but you did not specify the 'problemType'. The system will default to RESTful API testing.
• Initializing... 14:18:27.542 [main] WARN o.t.u.TestcontainersConfiguration - Attempted to read Testcontainers configuration file at file:/root/.testcontainers.properties but the file was not found. Exception message: FileNotFoundException: /root/.testcontainers.properties (No such file or directory)
• There are 6 usable RESTful API endpoints defined in the schema configuration 14:18:27.795 [main] WARN o.e.c.p.rest.RestActionBuilderV3 - No fields for object definition: loc_item
• Starting to generate test cases
• [ERROR] EvoMaster process terminated abruptly. This is likely a bug in EvoMaster. Please copy&paste the following stacktrace, and create a new issue on https://github.com/EMResearch/EvoMaster/issues javax.ws.rs.ProcessingException: javax.net.ssl.SSLException: Unsupported or unrecognized SSL message at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:267) at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:297) at org.glassfish.jersey.client.JerseyInvocation.lambda$invoke$0(JerseyInvocation.java:630) at org.glassfish.jersey.client.JerseyInvocation.call(JerseyInvocation.java:665) at org.glassfish.jersey.client.JerseyInvocation.lambda$runInScope$3(JerseyInvocation.java:659) at org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.glassfish.jersey.internal.Errors.process(Errors.java:205) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:390) at org.glassfish.jersey.client.JerseyInvocation.runInScope(JerseyInvocation.java:659) at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:629) at org.evomaster.core.problem.rest.service.AbstractRestFitness.handleRestCall(AbstractRestFitness.kt:448) at org.evomaster.core.problem.rest.service.BlackBoxRestFitness.doCalculateCoverage(BlackBoxRestFitness.kt:60) at org.evomaster.core.problem.rest.service.BlackBoxRestFitness.doCalculateCoverage(BlackBoxRestFitness.kt:17) at org.evomaster.core.search.service.FitnessFunction.calculateIndividualCoverageWithStats(FitnessFunction.kt:136) at org.evomaster.core.search.service.FitnessFunction.calculateCoverage(FitnessFunction.kt:61) at org.evomaster.core.search.service.FitnessFunction.calculateCoverage$default(FitnessFunction.kt:50) at org.evomaster.core.search.algorithms.RandomAlgorithm.searchOnce(RandomAlgorithm.kt:22) at org.evomaster.core.search.service.SearchAlgorithm.search(SearchAlgorithm.kt:73) at org.evomaster.core.Main$Companion.run(Main.kt:558) at org.evomaster.core.Main$Companion.initAndRun(Main.kt:174) at org.evomaster.core.Main$Companion.main(Main.kt:85) at org.evomaster.core.Main.main(Main.kt) Caused by: javax.net.ssl.SSLException: Unsupported or unrecognized SSL message at java.base/sun.security.ssl.SSLSocketInputRecord.handleUnknownRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(Unknown Source) at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source) at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source) at java.base/java.net.HttpURLConnection.getResponseCode(Unknown Source) at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(Unknown Source) at org.glassfish.jersey.client.internal.HttpUrlConnector.handleException(HttpUrlConnector.java:543) at org.glassfish.jersey.client.internal.HttpUrlConnector._apply(HttpUrlConnector.java:373) at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:265) ... 22 common frames omitted (venv) #

[arcuri82]

Hi @sathiyajith , thanks for reporting this issue. You are trying to test https://localhost:8888/docs. Are you sure that you want to use HTTPS to connect to localhost? shouldn't rather be http://localhost:8888/docs? Regardless, we should provide a better, more descriptive error message.

[sathiyajith]

Hey @arcuri82 , Thanks for pointing this out. It's a mistake from my end. I'm able to run the blackbox test and I got the test suites java files. Now, are these suites standalone? I dont have much experience on Maven. How should I create the pom.xml file and how should I run these suites using the evomaster executable file? Could you please explain?

[arcuri82]

hi @sathiyajith , if you are unfamiliar with Maven, you can use Gradle. Setting up third-party libraries is explained in the documentation (see https://github.com/WebFuzzing/EvoMaster/blob/master/docs/library_dependencies.md). Note that, if you prefer Python or JavaScript, you can use a different --outputFormat, like PYTHON_UNITTEST or JS_JEST

[sathiyajith]

@arcuri82 Thanks for your explanation. I can now execute the test suites. But, the test suites have only limited number of test cases. I thought Evomaster is kind of a fuzzing tool and I wanted it to run in the loop endlessly. Can we do that (maybe in white box testing)? Or does it give only specific number of test case everytime and stops after that?

[arcuri82]

if you want to run it for longer, you can increase the --maxTime value. However, what you get at the end is only a minimized test suite, with the best test cases.

There are more advanced coverage criteria, which would lead to larger and more thorough test suites, which can be activated with --advancedBlackBoxCoverage true. But that is an experimental feature (so, not on by default yet). i d actually like to hear what impact option that would give to you