Endava / cats

CATS is a REST API Fuzzer and negative testing tool for OpenAPI endpoints. CATS automatically generates, runs and reports tests with minimum configuration and no coding effort. Tests are self-healing and do not require maintenance.
Apache License 2.0
1.11k stars 75 forks source link

--securityFuzzerFile Argument not showing any results #39

Closed stuti100 closed 2 years ago

stuti100 commented 2 years ago

I was curious to use the security fuzzer offered by CATS but when ran,does not give the right result.Is this the right way to use it,I am using the command below (attached in screenshot)? If not please provide some additional info and how it outputs will look like?

Screenshot 2022-04-17 at 7 14 13 PM Screenshot 2022-04-17 at 7 22 08 PM
en-milie commented 2 years ago

Hi @stuti100. The --securityFuzzerFile requires the httpMethod entry to be present. It's not very explicit in the message currently presented in the log. I'll commit some changes to make that more comprehensible. In the mean time, alte your file as:

/path
  test1:
    description: XSS Strings
    httpMethod: POST
    targetFieldTypes:
      - string
    stringsFile: xss.txt
    expectedResponseCode: 200

The command you've used runs all the Fuzzers + the SecurityFuzzer. There is an alternate command to run only the SecurityFuzzer:

> cats run --contract=XXX --server=SERVER --output=OUTPUT secure.yml 
stuti100 commented 2 years ago

Hi @en-milie,Thanks for answering. I made above suggested changes and have some queries :

en-milie commented 2 years ago

@stuti100 Can you please send the OpenAPI specs, especially the _meta path and corresponding request/response objects?

stuti100 commented 2 years ago

Also wanted to know,while using cats run command we are unable to use --ignoreResponseCode and --paths? Is this true that we cannot use them in cats run command and only specific arguments would be ran ?

en-milie commented 2 years ago

--paths and --ignoreXXX arguments cannot be used with 'cats run'. this is because you the security.yml file has the paths as the main keys so adding another --path argument is redundant. the security.yml file also has a expectedResponseCode entry which might get in conflict with the --ignoreXXX argument. but I'll think about this a bit and see what is the best way to introduce --ignoreXXX arguments in 'cats run'.

stuti100 commented 2 years ago

Yes,got that.How can we provide multiple expectedResponseCode ? I tried providing them seperated by comma(,),but it does not work.

Screenshot 2022-04-23 at 2 12 47 PM Screenshot 2022-04-23 at 2 13 13 PM
en-milie commented 2 years ago

Hi @stuti100. To tackle things in order:

  1. currently you cannot provide multiple expectedResponseCode codes. As mentioned I'll extend this in correlation with the --ignoreXXX arguments. Probably in the next release.
  2. Is there a way to get the OpenAPI specs for the error above, especially the /_meta path details?
stuti100 commented 2 years ago

Thanks @en-milie for updates.Hoping to see the next release soon.