microsoft / restler-fuzzer

RESTler is the first stateful REST API fuzzing tool for automatically testing cloud services through their REST APIs and finding security and reliability bugs in these services.
MIT License
2.6k stars 297 forks source link

take into account 'minItems' and 'maxItems' when generating arrays in the grammar #198

Open marina-p opened 3 years ago

marina-p commented 3 years ago

See https://swagger.io/docs/specification/data-models/data-types.

RESTler should generate the minimum number of required elements to successfully execute the payload. (Then, the payload body checker will fuzz element lengths outside this range). The values generated should be unique.

type: array items: type: integer minItems: 1 maxItems: 10

mmaitre314 commented 3 years ago

We hit this issue while trying to fuzz MLFlow Serve REST APIs: the ML inference endpoint requires arrays of 10 numbers to accept the request. In our case the numbers do not need to be unique. Is there a workaround until min/max items is supported?

Swagger:

swagger: "2.0"
info:
  version: "1.0.0"
  title: "MLFlow Model Serving"
host: "127.0.0.1:5000"
schemes:
- "http"
paths:
  /invocations:
    post:
      consumes:
      - "application/json; format=pandas-split"
      produces:
      - "application/json"
      parameters:
      - in: "body"
        name: "body"
        required: true
        schema:
          type: "object"
          required:
          - "data"
          properties:
            data:
              type: "array"
              minItems: 1
              items:
                type: "array"
                minItems: 10
                maxItems: 10
                items:
                  type: "number"
      responses:
        "200":
          description: "OK"
        "500":
          description: "Internal Server Error"
mmaitre314 commented 3 years ago

Adding examples in the Swagger helped a bit. The array appears to get truncated though: instead of containing 10 items, requests only contain 5.

swagger: "2.0"
info:
  version: 1.0.0
  title: MLFlow Model Serving
host: 127.0.0.1:5000
schemes:
- http
paths:
  /invocations:
    post:
      consumes:
      - application/json; format=pandas-split
      produces:
      - application/json
      parameters:
      - in: body
        name: body
        required: true
        schema:
          type: object
          required:
          - data
          properties:
            data:
              type: array
              minItems: 1
              items:
                type: array
                minItems: 10
                maxItems: 10
                items:
                  type: number
                example: [0.0199132142,0.0506801187,0.1048086895,0.0700725447,-0.0359677813,-0.0266789028,-0.0249926566,-0.002592262,0.0037117382,0.0403433716]
      responses:
        "200":
          description: Inference result
          schema:
            type: array
            minItems: 1
            items:
              type: number
            example: [235.11371081266924]
        "500":
          description: Internal server error
marina-p commented 3 years ago

Hello Matthieu,

More detailed reply coming soon, but a quick reply to your latest message: the maximum is hard-coded, you can increase it from 5 on line 327 of SwaggerVisitors.fs to quickly work around this issue.

            let maxArrayElementsFromExample = 5

Thanks,

Marina