search

Endava / cats

CATS is a REST API Fuzzer and negative testing tool for OpenAPI endpoints. CATS automatically generates, runs and reports tests with minimum configuration and no coding effort. Tests are self-healing and do not require maintenance.
Apache License 2.0
1.11k stars 76 forks source link

`java.lang.NullPointerException` when fuzzing #98

Closed qarmin closed 4 months ago

qarmin commented 4 months ago

Describe the bug When fuzzing with command

./cats --server=http://127.0.0.1:5002 --contract=$DOWNLOAD_PATH -b  -k --skipFuzzers=NonRestHttpMethodsFuzzer,LargeNumberOfRandomAlphanumericHeadersFuzzer,LargeNumberOfRandomHeadersFuzzer,VeryLargeUnicodeStringsInFieldsFuzzer,VeryLargeUnicodeStringsInHeaders,DefaultValuesInFieldsFuzzer

after ~15 endpoints, program showed this error and stopped to work

java.lang.NullPointerException
    at com.endava.cats.model.FuzzingData.getFields(FuzzingData.java:135)
    at com.endava.cats.model.FuzzingData.getFields(FuzzingData.java:146)
    at com.endava.cats.model.FuzzingData.getAllFieldsAsCatsFields(FuzzingData.java:219)
    at com.endava.cats.model.FuzzingData.getAllReadOnlyFields(FuzzingData.java:174)
    at com.endava.cats.model.FuzzingData.removeReadWrite(FuzzingData.java:112)
    at com.endava.cats.model.FuzzingData.getPayload(FuzzingData.java:104)
    at com.endava.cats.command.CatsCommand.lambda$fuzzPath$3(CatsCommand.java:357)
    at java.base@20.0.2/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:178)
    at java.base@20.0.2/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
    at java.base@20.0.2/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
    at java.base@20.0.2/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
    at java.base@20.0.2/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
    at java.base@20.0.2/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:575)
    at java.base@20.0.2/java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
    at java.base@20.0.2/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:616)
    at java.base@20.0.2/java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:622)
    at java.base@20.0.2/java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:627)
    at com.endava.cats.command.CatsCommand.fuzzPath(CatsCommand.java:358)
    at com.endava.cats.command.CatsCommand.startFuzzing(CatsCommand.java:286)
    at com.endava.cats.command.CatsCommand.doLogic(CatsCommand.java:226)
    at com.endava.cats.command.CatsCommand.run(CatsCommand.java:177)
    at picocli.CommandLine.executeUserObject(CommandLine.java:2026)
    at picocli.CommandLine.access$1500(CommandLine.java:148)
    at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2453)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2415)
    at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273)
    at picocli.CommandLine$RunLast.execute(CommandLine.java:2417)
    at picocli.CommandLine.execute(CommandLine.java:2170)
    at com.endava.cats.CatsMain.run(CatsMain.java:42)
    at com.endava.cats.CatsMain_ClientProxy.run(Unknown Source)
    at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:132)
    at io.quarkus.runtime.Quarkus.run(Quarkus.java:71)
    at io.quarkus.runtime.Quarkus.run(Quarkus.java:44)
    at io.quarkus.runner.GeneratedMain.main(Unknown Source)

Sadly I cannot provide openapi file because is private, but will try minimize it and publish

Environment:

Key           | Value
------------- | --------------------
OS Name       | Linux
OS Version    | 6.5.0-17-generic
OS Arch       | amd64
Binary Type   | native
Cats Version  | 10.5.0
Cats Build    | 2024-02-05T07:53:16Z
Term Width    | 213
Term Type     | xterm-256color
Shell         | /bin/bash
qarmin commented 4 months ago

When creating minimal file I got 

java.lang.IllegalArgumentException: Scheme is not declared: ReportRequestQuery
    at com.endava.cats.factory.FuzzingDataFactory.generateSample(FuzzingDataFactory.java:426)
    at com.endava.cats.factory.FuzzingDataFactory.getRequestPayloadsSamples(FuzzingDataFactory.java:410)
    at com.endava.cats.factory.FuzzingDataFactory.getFuzzDataForHttpMethod(FuzzingDataFactory.java:231)
    at com.endava.cats.factory.FuzzingDataFactory.getFuzzDataForPost(FuzzingDataFactory.java:172)
    at com.endava.cats.factory.FuzzingDataFactory.fromPathItem(FuzzingDataFactory.java:98)
    at com.endava.cats.factory.FuzzingDataFactory_ClientProxy.fromPathItem(Unknown Source)
    at com.endava.cats.command.CatsCommand.fuzzPath(CatsCommand.java:346)
    at com.endava.cats.command.CatsCommand.startFuzzing(CatsCommand.java:286)
    at com.endava.cats.command.CatsCommand.doLogic(CatsCommand.java:226)
    at com.endava.cats.command.CatsCommand.run(CatsCommand.java:177)
    at picocli.CommandLine.executeUserObject(CommandLine.java:2026)
    at picocli.CommandLine.access$1500(CommandLine.java:148)
    at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2453)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2415)
    at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273)
    at picocli.CommandLine$RunLast.execute(CommandLine.java:2417)
    at picocli.CommandLine.execute(CommandLine.java:2170)
    at com.endava.cats.CatsMain.run(CatsMain.java:42)
    at com.endava.cats.CatsMain_ClientProxy.run(Unknown Source)
    at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:132)
    at io.quarkus.runtime.Quarkus.run(Quarkus.java:71)
    at io.quarkus.runtime.Quarkus.run(Quarkus.java:44)
    at io.quarkus.runner.GeneratedMain.main(Unknown Source)

is this expected? Openapi json

{"openapi":"3.0.3","info":{"title":"ff","description":"fffffff","license":{"name":""},"version":"1.1.0"},"paths":{"/rest/1/pdf/":{"post":{"tags":["crate::pdf_creator::routes"],"operationId":"create_pdf","requestBody":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/ReportRequestQuery"}}},"required":true},"responses":{"200":{"description":"Create PDF","content":{"text/plain":{"schema":{"type":"string"}}}}}}}},"components":{"schemas":{"Category":{"type":"string","enum":["STATIC","KINETIC"]}}}}
qarmin commented 4 months ago

Looks that this file causes this

{"openapi":"3.0.3","info":{"title":"fasf","description":"ASFAFSAF","license":{"name":""},"version":"1.1.0"},"paths":{"/rest/1/pdf/":{"post":{"tags":["crate::pdf_creator::routes"],"operationId":"create_pdf","requestBody":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/ReportRequestQuery2"}}},"required":true},"responses":{"200":{"description":"Create PDF","content":{"text/plain":{"schema":{"type":"string"}}}}}}}},"components":{"schemas":{"Category":{"type":"string","enum":["STATIC","KINETIC"]},"ReportRequestQuery2":{"type":"object","required":["report_type","report_uuid","language","data"],"properties":{"data":{"$ref":"#/components/schemas/ReportData2"},"language":{"$ref":"#/components/schemas/Languages"},"report_type":{"$ref":"#/components/schemas/ReportSymbol"},"report_uuid":{"type":"string"}}}}}}
en-milie commented 4 months ago

@qarmin Thank you for raising this. I'll take a look to see what's happening.

en-milie commented 4 months ago

This is fixed now and will be available in next release.

en-milie commented 4 months ago

Fix is available in latest release: https://github.com/Endava/cats/releases/tag/cats-11.0.0