search

Endava / cats

CATS is a REST API Fuzzer and negative testing tool for OpenAPI endpoints. CATS automatically generates, runs and reports tests with minimum configuration and no coding effort. Tests are self-healing and do not require maintenance.
Apache License 2.0
1.18k stars 73 forks source link

Cats not using provided API examples #143

Open blackgrease opened 1 week ago

blackgrease commented 1 week ago

Describe the bug When the flag --useExamples is explicitly set, cats still uses random data input. This is a problem as when running tests that require correct data in a POST/PUT/PATCH request, the request fails due to data validation not being passed. Which doesn't give proper insight into testing

To Reproduce Steps to reproduce the behaviour: 1.cats --contract {docs} --server {server} --maxRequestsPerMinute 120 -A -f ExtraHeaders -H 'User-agent={useragent}' -H 'Authorization={other_jwt}' -g note,info -j -o BrokenFunctionality --proxyHost localhost --proxyPort 8090 --useExamples

  1. The proxy is to see the result of the request. I have included a screenshot of the resulting request via the proxy. (Info identifying app has been removed). Any fuzzer that does not target fields has this issue.

  2. image

Expected behaviour When --useExamples is set, the API data should be used instead of random inputs that will definitely cause the test to fail

en-milie commented 6 days ago

Hi @blackgrease. Indeed --useExample does not have first priority, but rather the internal generators. It is indeed a better idea to consider examples if available as first option and then generate.

en-milie commented 5 days ago

Fix will be available in 12.2.0. Documentation here: https://endava.github.io/cats/docs/getting-started/examples

blackgrease commented 1 day ago

Thank you, thatll be much appreciated

Sent from Proton Mail Android

-------- Original Message -------- On 10/11/24 11:38 PM, Madalin Ilie wrote:

Fix will be available in 12.2.0. Documentation here: https://endava.github.io/cats/docs/getting-started/examples

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>