WebFuzzing / EvoMaster

The first open-source AI-driven tool for automatically generating system-level test cases (also known as fuzzing) for web/enterprise applications. Currently targeting whitebox and blackbox testing of Web APIs, like REST, GraphQL and RPC (e.g., gRPC and Thrift).
GNU Lesser General Public License v3.0
498 stars 83 forks source link

No proper info/warning when mismatch of version between Core and Driver processes #793

Open italktothewind opened 1 year ago

italktothewind commented 1 year ago

Hi, I'm having the following error when EvoMaster tries to call the driver:

* EvoMaster version: 1.6.2-SNAPSHOT
* Initializing...
02:41:25.343 [main] ERROR o.e.c.r.service.RemoteController - Driver error. HTTP status 400. Error: Unrecognized field "resetCustomizedMethodForMockObject" (class org.evomaster.client.java.controller.api.dto.SutRunDto), not marked as ignorable (6 known properties: "calculateSqlHeuristics", "resetState", "resetMockedExternalServicesWithCustomizedMethod", "run", "methodReplacementCategories", "extractSqlExecutionInfo"])
 at [Source: (shaded.org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$UnCloseableInputStream); line: 1, column: 73] (through reference chain: org.evomaster.client.java.controller.api.dto.SutRunDto["resetCustomizedMethodForMockObject"])
02:41:25.344 [main] WARN  o.e.c.r.service.RemoteController - Failed to change running state of the SUT. HTTP status 400. Error: 'null
* [ERROR] ERROR related to the system under test: Failed to start the SUT
  For white-box testing, look at the logs of the EvoMaster Driver to help debugging this problem.

Steps to reproduce it:

1) Run src/main/java/em/embedded/org/javiermf/features/EmbeddedEvoMasterController.java from the latest version of EMB 2) Run src/main/kotlin/org/evomaster/core/Main.kt from the latest version of EvoMaster 3) I got that error on EvoMaster. No errors reported in the driver (at DEBUG level). Same error occurs in EvoMaster if instead of step 2) I generate core/target/evomaster.jar and then I run that jar using java -jar.

In both IDE (IDEA) projects I'm using Java 8 as SDK

Any advice?

Thanks!

italktothewind commented 1 year ago

The issue was that EMB was using 1.6.1 version of EvoMaster and my version of EvoMaster was 1.6.2-SNAPSHOT and SutRunDto has different fields in each versions.

arcuri82 commented 1 year ago

hi,

when you tried with core/target/evomaster.jar, wasn't there a warning stating the versions are different? (such warning is not there when running directly from IDE)

italktothewind commented 1 year ago

Hi,

I don't see a warning, this is the full log when I tried with the jar:

 _____          ___  ___          _
|  ___|         |  \/  |         | |
| |____   _____ | .  . | __ _ ___| |_ ___ _ __
|  __\ \ / / _ \| |\/| |/ _` / __| __/ _ \ '__|
| |___\ V / (_) | |  | | (_| \__ \ ||  __/ |
\____/ \_/ \___/\_|  |_/\__,_|___/\__\___|_|

* EvoMaster version: 1.6.2-SNAPSHOT
* Initializing...
04:34:54.651 [main] ERROR o.e.c.r.service.RemoteController - Driver error. HTTP status 400. Error: Unrecognized field "resetCustomizedMethodForMockObject" (class org.evomaster.client.java.controller.api.dto.SutRunDto), not marked as ignorable (6 known properties: "calculateSqlHeuristics", "resetState", "resetMockedExternalServicesWithCustomizedMethod", "run", "methodReplacementCategories", "extractSqlExecutionInfo"])
 at [Source: (shaded.org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$UnCloseableInputStream); line: 1, column: 73] (through reference chain: org.evomaster.client.java.controller.api.dto.SutRunDto["resetCustomizedMethodForMockObject"])
04:34:54.652 [main] WARN  o.e.c.r.service.RemoteController - Failed to change running state of the SUT. HTTP status 400. Error: 'null
* [ERROR] ERROR related to the system under test: Failed to start the SUT
  For white-box testing, look at the logs of the EvoMaster Driver to help debugging this problem.
arcuri82 commented 1 year ago

hi, thanks. then this is a bug, because there should be a clear warning there, and possibly better error message