Endava / cats

CATS is a REST API Fuzzer and negative testing tool for OpenAPI endpoints. CATS automatically generates, runs and reports tests with minimum configuration and no coding effort. Tests are self-healing and do not require maintenance.
Apache License 2.0
1.11k stars 76 forks source link

FunctionalFuzzer issue with array parameters #85

Closed olicau closed 7 months ago

olicau commented 7 months ago

Hello,

I unfortunately cannot share all the files so I am going to try to describe it as best as I can. I have a definition in my api.json (openapi v3.0.1) file like this "key_ops": { "maxItems": 16, "type": "array", "items": { "type": "string" }, "description": "Key usage array specifies key usage", "nullable": true },

In my functionalFuzzer file I have tried to pass the values like this: key_ops:

or like this

key_ops: [ "sign", "verify"]

Here is what I see in the log Test1.json when I run my test "request": { ... "payload": "{....",\"key_ops\":[\"sign\",\"sign\",\"sign\",\"sign\",\"sign\",\"sign\",\"sign\",\"sign\",\"sign\",\"sign\",\"sign\",\"sign\",\"sign\",\"sign\",\"sign\",\"sign\"]....}", ... },

Looks like the first item in the array pass in parameter is duplicated 16 times.... Any idea ? Am I doing something wrong ? Thanks

en-milie commented 7 months ago

Hi @olicau. I'll check in more details.

olicau commented 7 months ago

Hello, did try the fix, and I am still having the issue. I was trying to look in the modified files in the changelist, looks like the key_ops is being set with 16 fuzzed values then they are all replaced by sign. I have not dig deeper than that. Maybe to clarify, I was expecting to get "key_ops":["sign","verify"] in the payload

en-milie commented 7 months ago

would it be possible to put here the openapi file and the functionalFuzzer file?

olicau commented 7 months ago

I cannot share everything, here is more info Fuzzer test: /v1/servicekey: test_1: description: Create a new Key

key_name: "fuzz_keyname"

key_context: "fuzz_context"
key_type: "RSA"
key_size: 2048
key_ops: '[ "sign","verify" ]'
crv: "cats_remove_field"
key_material: "cats_remove_field"
public_exponent: "cats_remove_field"
keys: "cats_remove_field"
httpMethod: POST
expectedResponseCode: 2XX
# Capture the response value and put them into global variables
output:
  _keyId: key_id
  _keyName: key_name

Extract from api.json

"paths": { "/v1/servicekey": { "post": { "tags": [ "ServicekeyApi" ], "summary": "Create or imports customer service key", "description": "/v1/servicekey", "operationId": "CreateImportServiceKey", "requestBody": { "description": "Request payload", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/CreateImportServiceKeyRequest" } } }, "required": true }, "responses": { "201": { "description": "Returns id of a key created or imported", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/GetServiceKeyResponse" } } } }, "400": { "description": "Bad request error", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/ErrorsResponse" } } } }, "500": { "description": "Internal server error", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/ErrorsResponse" } } } }, "401": { "description": "Unauthorized", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/ErrorsResponse" } } } } } }

... "components": { "schemas": { "CreateImportServiceKeyRequest": { "required": [ "key_name" ], "type": "object", "properties": { "key_name": { "maxLength": 32, "minLength": 1, "type": "string", "description": "Key friendly name" }, "key_context": { "type": "string", "description": "Key Context, utf8 text string", "nullable": true }, "key_type": { "type": "string", "description": "The \"kty\" (key type) parameter identifies the cryptographic algorithm family used with the key", "nullable": true }, "key_size": { "type": "integer", "description": "Key size specifies key size in bits for oct and RSA keys", "format": "int32", "nullable": true }, "key_ops": { "maxItems": 16, "type": "array", "items": { "type": "string" }, "description": "Key usage array specifies key usage", "nullable": true }, "crv": { "type": "string", "description": "The \"crv\" (curve) parameter identifies the cryptographic curve used with the key.", "nullable": true }, "keys": { "type": "array", "items": { "$ref": "#/components/schemas/JsonWebKey" }, "description": "Json Web Key Set representing keys in request", "nullable": true } }, "additionalProperties": false, "description": "Create domain key model" }, /

en-milie commented 7 months ago

Is should be fixed in latest commits. Also, in the file please make sure you remove the single quotes from the table. It should be key_ops: [ "sign","verify" ] instead if key_ops: '[ "sign","verify" ]'.

olicau commented 7 months ago

Perfect 👍 works fine thank you

en-milie commented 7 months ago

Fix available in latest release now: https://github.com/Endava/cats/releases/tag/cats-10.1.0