search

Endava / cats

CATS is a REST API Fuzzer and negative testing tool for OpenAPI endpoints. CATS automatically generates, runs and reports tests with minimum configuration and no coding effort. Tests are self-healing and do not require maintenance.
Apache License 2.0
1.11k stars 76 forks source link

Configurable Error Code Expectations #89

Closed binaryArrow closed 5 months ago

binaryArrow commented 6 months ago

Issue While using CATS in my project, I've found it to be a valuable tool. However, a challenge has surfaced with the AbugidasInStringFields Fuzzer. This particular fuzzer anticipates an error code of 400, whereas my API, built with restEasy therefore implementing JAX-RS, is designed to return a 404.

For Example the fuzzer is applied to query parameters of type string with the format date. As per the JAX-RS specification: Link to JAX-RS Specification, query parameters that don't conform to the specified type (in this case, date) are expected to yield an error code of 404.

This discrepancy arises because certain Field Fuzzers, including AbugidasInStringFields, anticipate an error code of 400, which contradicts the JAX-RS specification.

Proposal Introduce configurability for expected error codes in Field Fuzzers.

en-milie commented 6 months ago

Hi @binaryArrow. Thank you for raising this. I'll do a bit of investigation and get back.

en-milie commented 5 months ago

Support for configurable response codes is available starting with this commit : https://github.com/Endava/cats/commit/0bb6d87eeb90228f1ec85c04046bf5e988407179. And also in the next release.

binaryArrow commented 5 months ago

nice!

en-milie commented 5 months ago

Details about usage here: https://endava.github.io/cats/docs/advanced-topics/fuzzers-config