EvoMaster process terminated when I run EMB

[codingsoo]

Hi, thank you for this amazing tool. I really like this tool.

I want to report some bugs with running EMB (https://github.com/EMResearch/EMB) for 5 hours each (only had --maxTime 5h option).

I found that some of them have issues with unexpected exit error, but they did not give me meaningful error message (just said that it is likely EvoMaster's bug, please report it). Specifically, I did not have any issue with languagetool, ncs, ocvn, restcountries, and scs. However, I had issues with catwatch, features-service, news, proxyprint, and scout-api.

I just re-ran them when the service is terminated, and it worked for catwatch, features-service, proxyprint, and scout-api.

However, for News project, I tried 10 times but never success to get the result (No SQL search option gave me the result in this case).

[arcuri82]

Hi, there might be sporadic issues, eg, due to TCP connections, or weird bugs that manifest theirselves only in some specific case (eg, due to randomness of the algorithms). but should happen seldom (eg, out of 10 runs, 1 our 2 failing could happen).

Regarding News, that is weird (ie, failing all the 10 times). what was the error log on the evomaster.jar console? what about the logs on the SUT?

Haven't run EM on News for so long (ie 5 hours), so might be some memory leak... need to check out. However, which version of EM are you using? (eg, a release, or SNAPSHOT from master branch?)

[arcuri82]

also, did you use Embedded or External drivers?

[arcuri82]

regarding News, it was an interesting bug... I could reproduce it. it crashed at around 2 hours.

The problem when we make millions of HTTP calls is that the OS could fail some due to TCP issues. in those cases, EM tries to restart the SUT before re-evaluatiing a test case.

The problem was in the driver of News, which crashed when trying to re-start it, due to some JMX problem related to database. might be an actual bug in Spring (but didn't investigate it in details) or misconfiguration. anyway, JMX is unnecessary here, so a simple --spring.jmx.enabled=false fixed the issue in the driver.

The fix is pushed in the EMB repository.

[arcuri82]

$ java -jar core/target/evomaster.jar --maxTime 5h
*
 _____          ___  ___          _
|  ___|         |  \/  |         | |
| |____   _____ | .  . | __ _ ___| |_ ___ _ __
|  __\ \ / / _ \| |\/| |/ _` / __| __/ _ \ '__|
| |___\ V / (_) | |  | | (_| \__ \ ||  __/ |
\____/ \_/ \___/\_|  |_/\__,_|___/\__\___|_|

* EvoMaster version: 1.1.1-SNAPSHOT
* There are 7 usable RESTful API endpoints defined in the schema configuration
* Starting to generate test cases
* Consumed search budget: 43.955%; covered targets: 345; time per test: 4.5ms (2.2 actions)
12:37:04.961 [main] WARN  o.e.c.r.service.RemoteController - Failed to parse dto
javax.ws.rs.ProcessingException: Error reading entity from input stream.
        at org.glassfish.jersey.message.internal.InboundMessageContext.readEntity(InboundMessageContext.java:900)
        at org.glassfish.jersey.message.internal.InboundMessageContext.readEntity(InboundMessageContext.java:845)
        at org.glassfish.jersey.client.ClientResponse.readEntity(ClientResponse.java:340)
        at org.glassfish.jersey.client.InboundJaxrsResponse$2.call(InboundJaxrsResponse.java:104)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
        at org.glassfish.jersey.internal.Errors.process(Errors.java:205)
        at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:365)
        at org.glassfish.jersey.client.InboundJaxrsResponse.runInScopeIfPossible(InboundJaxrsResponse.java:244)
        at org.glassfish.jersey.client.InboundJaxrsResponse.readEntity(InboundJaxrsResponse.java:101)
        at org.evomaster.core.remote.service.RemoteController.getDtoFromResponse(RemoteController.kt:174)
        at org.evomaster.core.remote.service.RemoteController.getTestResults(RemoteController.kt:293)
        at org.evomaster.core.problem.httpws.service.HttpWsFitness.updateFitnessAfterEvaluation(HttpWsFtiness.kt:363)
        at org.evomaster.core.problem.rest.service.AbstractRestFitness.restActionResultHandling(AbstractRestFitness.kt:591)
        at org.evomaster.core.problem.rest.service.RestFitness.doCalculateCoverage(RestFitness.kt:89)
        at org.evomaster.core.problem.rest.service.RestFitness.doCalculateCoverage(RestFitness.kt:17)
        at org.evomaster.core.search.service.FitnessFunction.calculateCoverage(FitnessFunction.kt:59)
        at org.evomaster.core.search.service.mutator.Mutator.mutateAndSave(Mutator.kt:143)
        at org.evomaster.core.search.algorithms.MioAlgorithm.search(MioAlgorithm.kt:51)
        at org.evomaster.core.Main$Companion.run(Main.kt:346)
        at org.evomaster.core.Main$Companion.initAndRun(Main.kt:154)
        at org.evomaster.core.Main$Companion.main(Main.kt:76)
        at org.evomaster.core.Main.main(Main.kt)
Caused by: java.io.IOException: Stream closed.
        at java.net.AbstractPlainSocketImpl.available(AbstractPlainSocketImpl.java:470)
        at java.net.SocketInputStream.available(SocketInputStream.java:259)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:353)
        at sun.net.www.MeteredStream.read(MeteredStream.java:134)
        at java.io.FilterInputStream.read(FilterInputStream.java:133)
        at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(HttpURLConnection.java:3454)
        at org.glassfish.jersey.client.internal.HttpUrlConnector$2.read(HttpUrlConnector.java:208)
        at org.glassfish.jersey.message.internal.EntityInputStream.read(EntityInputStream.java:79)
        at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$UnCloseableInputStream.read(ReaderInterceptorExecutor.java:273)
        at com.fasterxml.jackson.core.json.ByteSourceJsonBootstrapper.ensureLoaded(ByteSourceJsonBootstrapper.java:524)
        at com.fasterxml.jackson.core.json.ByteSourceJsonBootstrapper.detectEncoding(ByteSourceJsonBootstrapper.java:129)
        at com.fasterxml.jackson.core.json.ByteSourceJsonBootstrapper.constructParser(ByteSourceJsonBootstrapper.java:247)
        at com.fasterxml.jackson.core.JsonFactory._createParser(JsonFactory.java:1485)
        at com.fasterxml.jackson.core.JsonFactory.createParser(JsonFactory.java:972)
        at org.glassfish.jersey.jackson.internal.jackson.jaxrs.base.ProviderBase._createParser(ProviderBase.java:851)
        at org.glassfish.jersey.jackson.internal.jackson.jaxrs.base.ProviderBase.readFrom(ProviderBase.java:792)
        at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$TerminalReaderInterceptor.invokeReadFrom(ReaderInterceptorExecutor.java:233)
        at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$TerminalReaderInterceptor.aroundReadFrom(ReaderInterceptorExecutor.java:212)
        at org.glassfish.jersey.message.internal.ReaderInterceptorExecutor.proceed(ReaderInterceptorExecutor.java:132)
        at org.glassfish.jersey.message.internal.MessageBodyFactory.readFrom(MessageBodyFactory.java:1072)
        at org.glassfish.jersey.message.internal.InboundMessageContext.readEntity(InboundMessageContext.java:885)
        ... 22 common frames omitted
12:37:04.964 [main] WARN  o.e.c.r.service.RemoteController - Missing DTO
12:37:04.964 [main] WARN  o.e.c.p.r.s.AbstractRestFitness - Cannot retrieve coverage
* Consumed search budget: 99.997%; covered targets: 345; time per test: 2.3ms (1.0 actions)
* Going to save 33 tests to src/em
* Evaluated tests: 5890701
* Evaluated actions: 6861609
* Needed budget: 43%
* Passed time (seconds): 18000
* Execution time per test (ms): Avg=2.88 , min=0.00 , max=9123.00
* Computation overhead between tests (ms): Avg=0.18 , min=0.00 , max=21.00
* Covered targets (lines, branches, faults, etc.): 345
* Potential faults: 7
* Bytecode line coverage: 50% (98 out of 195 in 14 units/classes)
* EvoMaster process has completed successfully
* Use --help and visit http://www.evomaster.org to learn more about available options

[arcuri82]

note that in those logs there is one call failed (out of 6.8M), which ends in the logs, but it does not crash the whole application now

[codingsoo]

I am sorry for my late response. It seems you fixed it! Thank you very much!

[arcuri82]

;)

I ll close this issue now. if you find other bugs, please create a new issue ;) thanks