Endava / cats

CATS is a REST API Fuzzer and negative testing tool for OpenAPI endpoints. CATS automatically generates, runs and reports tests with minimum configuration and no coding effort. Tests are self-healing and do not require maintenance.
Apache License 2.0
1.19k stars 73 forks source link

Fuzzing won't be executed on POST method without request body #37

Closed Mato816 closed 2 years ago

Mato816 commented 2 years ago

OS: Windows Action: Fuzz endpoint with POST method without request body Swagger: 2.0 CATS version: 7.0.6 Server: https://petstore.swagger.io/ API doc: https://petstore.swagger.io/v2/swagger.json

Hi,

we tried to launch test for fuzzing endpoint with POST method without request body. We used "swagger":"2.0" and after run command below we have got error: The following HTTP methods won't be executed for path /pet/{petId}: [GET, DELETE]

Command: java -jar cats-uber.jar --contract=https://petstore.swagger.io/v2/swagger.json --server=https://petstore.swagger.io --paths=/pet/{petId} --urlParams=petId:PET-123 --httpMethods=POST

Log from CMD: Powered by Quarkus 2.7.1.Final [**][*] ? start Starting CATS, version 7.0.6, build-time 2022-03-14T06:40:15Z UTC [**][*] ? note Processing configuration... [**][*] ? info No security custom Fuzzer file. SecurityFuzzer will be skipped! [**][*] ? info No custom Fuzzer file. FunctionalFuzzer will be skipped! [**][*] ? info No reference data file was supplied! Payloads supplied by Fuzzers will remain unchanged! [**][*] ? info URL parameters: [petId:PET-123] [**][*] ? info No headers file was supplied! No additional header will be added! [**][*] ? complete Finished parsing the contract in 1865 ms [**][*] ? start Start cleaning up cats-report folder ... [**][*] ? complete Cleanup complete! [**][*] ? skipping Skipping path /pet [**][*] ? skipping Skipping path /pet/findByStatus [**][*] ? skipping Skipping path /pet/findByTags [**][*] ? info [**][*] ? start Start fuzzing path /pet/{petId} [**][*] ? info The following HTTP methods won't be executed for path /pet/{petId}: [GET, DELETE] [**][*] ? info 61 configured fuzzers out of 88 total fuzzers: [TrailingSpacesInHeadersFuzzer, VeryLargeValuesInFieldsFuzzer, RemoveFieldsFuzzer, StringFormatTotallyWrongValuesFuzzer, NamingsContractInfoFuzzer, AbugidasCharsInHeadersFuzzer, DummyContentTypeHeadersFuzzer, NewFieldsFuzzer, PathTagsContractInfoFuzzer, NullValuesInFieldsFuzzer, UnsupportedAcceptHeadersFuzzer, HappyFuzzer, StringFormatAlmostValidValuesFuzzer, SecurityFuzzer, OnlySpacesInHeadersFuzzer, LeadingSpacesInHeadersFuzzer, StringsInNumericFieldsFuzzer, MinimumExactValuesInNumericFieldsFuzzer, ExtremeNegativeValueIntegerFieldsFuzzer, MaximumExactValuesInNumericFieldsFuzzer, MinLengthExactValuesInStringFieldsFuzzer, DecimalValuesInIntegerFieldsFuzzer, NonRestHttpMethodsFuzzer, RecommendedHttpCodesContractInfoFuzzer, ExtremeNegativeValueDecimalFieldsFuzzer, DuplicateHeaderFuzzer, RecommendedHeadersContractInfoFuzzer, FunctionalFuzzer, ExtraHeaderFuzzer, SecuritySchemesContractInfoFuzzer, StringFieldsLeftBoundaryFuzzer, BypassAuthenticationFuzzer, TopLevelElementsContractInfoFuzzer, ExtremePositiveValueDecimalFieldsFuzzer, MaxLengthExactValuesInStringFieldsFuzzer, VersionsContractInfoFuzzer, VeryLargeUnicodeValuesInHeadersFuzzer, ZalgoTextInStringFieldsSanitizeValidateFuzzer, RemoveHeadersFuzzer, DummyRequestFuzzer, ZalgoTextInHeadersFuzzer, VeryLargeUnicodeValuesInFieldsFuzzer, DecimalFieldsLeftBoundaryFuzzer, HttpMethodsFuzzer, MalformedJsonFuzzer, VeryLargeValuesInHeadersFuzzer, DecimalFieldsRightBoundaryFuzzer, InvalidValuesInEnumsFieldsFuzzer, EmptyStringValuesInHeadersFuzzer, BooleanFieldsFuzzer, CheckSecurityHeadersFuzzer, EmptyStringValuesInFieldsFuzzer, DummyAcceptHeadersFuzzer, IntegerFieldsRightBoundaryFuzzer, HttpStatusCodeInValidRangeContractInfoFuzzer, AbugidasCharsInStringFieldsSanitizeValidateFuzzer, StringFieldsRightBoundaryFuzzer, ExtremePositiveValueInIntegerFieldsFuzzer, IntegerFieldsLeftBoundaryFuzzer, UnsupportedContentTypesHeadersFuzzer, XmlContentTypeContractInfoFuzzer] [**][*] ? skipping Skipping path /pet/{petId}/uploadImage [**][*] ? skipping Skipping path /store/inventory [**][*] ? skipping Skipping path /store/order [**][*] ? skipping Skipping path /store/order/{orderId} [**][*] ? skipping Skipping path /user [**][*] ? skipping Skipping path /user/createWithArray [**][*] ? skipping Skipping path /user/createWithList [**][*] ? skipping Skipping path /user/login [**][*] ? skipping Skipping path /user/logout [**][*] ? skipping Skipping path /user/{username} [**][*] ? skipping Skip printing time execution statistics. You can use --printExecutionStatistics to enable this feature! [**][***] ? complete CATS finished in 171 ms. Total (excluding skipped) requests 0. ? Passed 0, ? warnings: 0, ? errors: 0, ? skipped: 0. You can open the report here: file:///C:/Users/user1/Documents/TOOLS/CATS/cats-report/index.html

en-milie commented 2 years ago

Hi @Mato816. CATS only supports application/json as Content-Type. As I see in the Pet Store json contract the POST endpoint accepts application/x-www-form-urlencoded. You can provide additional contentTypes using the --contentType argument (like --contentType=application/x-www-form-urlencoded) but that might lead to some things breaking as it expects JSON elements to be present.

Mato816 commented 2 years ago

Hi @en-milie,

thank you for your response. If i provided additional parameter --contentType=application/x-www-form-urlencoded it worked well. I have only additional question. If i have API docs the POST endpoint accepts */* what can i provide in --contentType additional argument? API doc - petstoreTest.zip

en-milie commented 2 years ago

Hi @Mato816. To answer your question you shouldn't provide anything in --contentType as it adds json by default. I'm working now on a feature to provide out of the box support for application/x-www-form-urlencoded and multipart/form-data on top of the default json one.

Mato816 commented 2 years ago

Hi @en-milie,

I have a question on your last answer. I tried fuzzing for API doc petstoreTest.zip with following command:

java -jar cats.jar --contract=petstoreTest.yaml --server=https://petstore.swagger.io --paths=/pet/{petId} --urlParams=petId:PET-123 --httpMethods=POST

and got message:

The following HTTP methods won't be executed for path /pet/{petId}: [DELETE]

I know that this issues isn't related with issue I created but I have a question if I'm providing some bad arguments or not? Because you replied me that i shouldn't provide anything in --contentType argument. Can you please help me with this issue?

en-milie commented 2 years ago

Hi @Mato816. The way you run CATS restricts it only to POST (you provide --httpMethods=POST) which means that CATS won't run for DELETE. If you want to have CATS running for all http methods, you just need to remove the last bit: java -jar cats.jar --contract=petstoreTest.yaml --server=https://petstore.swagger.io --paths=/pet/{petId} --urlParams=petId:PET-123.

Looking at the sample contract it seems that the POST method does not have a declared body. This means that CATS cannot do any fuzzing, so there won't be any fuzzer running for POST. But you you run the above command you'll see that it runs for DELETE as it has path params.

Mato816 commented 2 years ago

Hi @en-milie,

thank you for your explanation. I'm closing this issue now.