search

Endava / cats

CATS is a REST API Fuzzer and negative testing tool for OpenAPI endpoints. CATS automatically generates, runs and reports tests with minimum configuration and no coding effort. Tests are self-healing and do not require maintenance.
Apache License 2.0
1.11k stars 76 forks source link

CATS runner crashing with java.lang.IllegalStateException #94

Closed ganeshnikam18 closed 5 months ago

ganeshnikam18 commented 5 months ago

Describe the bug I have locally built the cats-runner.jar using the mvwn command. Trying to run the tool on one of the custom perstore.json file. In this run cats-runner.jar is crashing with " java.lang.NullPointerException". Here are the complete details:

CATS version: 10.3.0 Petstore Speci File: petstore.json

For the server side, I am running Spotlight Prism Mock Server:

Stacktrace of the crash: [******][*******] ◯ key application/json contentType application\/.*\+?json;?.* java.lang.IllegalStateException: Not a JSON Object: 3.366020007382855 at com.google.gson.JsonElement.getAsJsonObject(JsonElement.java:101) at com.endava.cats.factory.FuzzingDataFactory.getAnyOrOneOffElements(FuzzingDataFactory.java:503) at com.endava.cats.factory.FuzzingDataFactory.getAnyOrOneOffElements(FuzzingDataFactory.java:509) at com.endava.cats.factory.FuzzingDataFactory.addNewCombination(FuzzingDataFactory.java:457) at com.endava.cats.factory.FuzzingDataFactory.getPayloadCombinationsBasedOnOneOfAndAnyOf(FuzzingDataFactory.java:429) at com.endava.cats.factory.FuzzingDataFactory.generateSample(FuzzingDataFactory.java:411) at com.endava.cats.factory.FuzzingDataFactory.getResponsePayloads(FuzzingDataFactory.java:659) at com.endava.cats.factory.FuzzingDataFactory.getFuzzDataForHttpMethod(FuzzingDataFactory.java:209) at com.endava.cats.factory.FuzzingDataFactory.getFuzzDataForPost(FuzzingDataFactory.java:157) at com.endava.cats.factory.FuzzingDataFactory.fromPathItem(FuzzingDataFactory.java:88) at com.endava.cats.factory.FuzzingDataFactory_ClientProxy.fromPathItem(Unknown Source) at com.endava.cats.command.CatsCommand.fuzzPath(CatsCommand.java:321) at com.endava.cats.command.CatsCommand.startFuzzing(CatsCommand.java:261) at com.endava.cats.command.CatsCommand.doLogic(CatsCommand.java:204) at com.endava.cats.command.CatsCommand.run(CatsCommand.java:156) at picocli.CommandLine.executeUserObject(CommandLine.java:2026) at picocli.CommandLine.access$1500(CommandLine.java:148) at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461) at picocli.CommandLine$RunLast.handle(CommandLine.java:2453) at picocli.CommandLine$RunLast.handle(CommandLine.java:2415) at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273) at picocli.CommandLine$RunLast.execute(CommandLine.java:2417) at picocli.CommandLine.execute(CommandLine.java:2170) at com.endava.cats.CatsMain.run(CatsMain.java:39) at com.endava.cats.CatsMain_ClientProxy.run(Unknown Source) at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:132) at io.quarkus.runtime.Quarkus.run(Quarkus.java:71) at io.quarkus.runtime.Quarkus.run(Quarkus.java:44) at io.quarkus.runner.GeneratedMain.main(Unknown Source)

To Reproduce Steps to reproduce the behaviour:

  1. Run the prism mock server container with the above petstore contract file docker run --init --rm -v $(pwd):/tmp -p 4010:4010 stoplight/prism:4 mock -h 0.0.0.0 "/tmp/petstore.json
  2. From the other host where cats tool is built and binary is available run the below command: java -jar target/cats-runner.jar --server=http://:4010 --contract=petstore.json --fuzzers=HappyPathFuzzer --log=com.endava.cats:debug --verbosity=DETAILED

It will crash with above stack trace

Expected behaviour The cats tool should not crash

Below mentioned "Error" object in the contract causing this crash: "properties": { "code": { "type": "array", "items": { "type": "array", "items": { "type": "number" } } },

What is the root cause of the issue ? FuzzingDataFactory.java: getAnyOrOneOffElements() causing this crash. Here as per the current code we are assuming that all the elements in jsonElement are either JsonArray or JsonObject. But that may not be true. Elements in the array can be primitive type like in this case "number". So it crashes saying not a Json Object.

What is the fix ? I have added fix as per my understanding of the code. Here is patch of my fix. You can review it and apply. Patch: cats-crash-fix.txt

en-milie commented 5 months ago

Hi @ganeshnikam18. Thank you for reporting this and for the fix. It will be available in the next release.