stoplightio / prism

Turn any OpenAPI2/3 and Postman Collection file into an API server with mocking, transformations and validations.
https://stoplight.io/open-source/prism
Apache License 2.0
4.35k stars 349 forks source link

Prism proxy validation failing silently when using null as value in enum array of a property #1931

Open hekka opened 3 years ago

hekka commented 3 years ago

Describe the bug

When using an OAS document where we described nullable properties, we have earlier described null as value in the enum array of values(due to other library constraints..)

When validating models via prism proxy -command, that contained these enum properties it turns out that validation is not performed at all on the model, and no indication that validation was not performed is given, i.e silent fail/error.

Since the result is passed from API, via prism proxy server and back to the client without any sl-violation headers, it looks like validation has passed, while in reality it has failed or not been performed.

Also when using the --errors flag, there are no errors flagged.

When removing the null value from the enum array and keeping `nullable: true', validation is performed as expected.

To Reproduce

  1. Given this enum property in the OAS document, part of a larger model:
    enumProperty:
    type: string
    nullable: true
    enum: ['H','M','L', null]
  2. Validate this model through Prism proxy, where the model contains any error on any property in the response
  3. No sl-violation header is received with the response

Expected behavior

Would expect either Prism to notify that the OAS was malformed OR that validation was not performed.

A header could be passed, indicating that validations have been performed, e.g 'sl-validation' : 'performed-succesfully' / 'sl-validation' : 'performed-errored'

Additional context

Add any other context about the problem here.

Environment:

EdVinyard commented 2 years ago

@brendarearden , what's the status of this?

brendarearden commented 2 years ago

I was able to reproduce this error using the following steps:

  1. Using the provided prism examples, run prism proxy examples/petstore.oas3.yaml https://petstore.swagger.io/v2 --errors
  2. Then curl -X GET -s -D "/dev/stderr" http://localhost:4010/store/order/1\?__server=https://petstore.swagger.io/v2 which returns an error (the response property status returns an empty string, but the schema is expecting placed, approved or delivered)
  3. Add the enumProperty provided in the issue to the Order schema, and rerun the same curl commend. You will not see an error and the sl-violations header is not present, even though there is still an error in the schema.
Lingihn commented 1 year ago

Is there a deadline for fixing the bug? Unfortunately, we cannot remove null from enum due to schema generation on the backend.

Lingihn commented 1 year ago

@brendarearden I figured out what the problem was. The problem is the HTTP-Spec dependency. Apparently, it is also being developed by Stoplight.

Based on the file transferred to Swagger, a JSON-Schema is created, against which the data is validated. If the JSON-Schema is not valid (this is determined by Ajv.compile()) at the time of accessing the route, then the validation function will return an error, and then the error will be ignored.

JSON-Schema may become invalid if the popular solution for enum "nullable: true" and "enum -... - null" are used together in one field (as above): The HTTP-Spec does not have a check to see if there is already a null in the enum, which results in duplicate nulls in the enum.

This problem can be solved in different ways:

At a minimum, you need:

Further:

Either/and:

Please comment on possible solutions. Is it necessary to do PR in HTTP-Spec? Which option for solving the silent error problem is closest to you?

About the fact that the nullable property and enum: null can be in the same field: https://swagger.io/docs/specification/data-models/enums/ A little more information: https://github.com/OAI/OpenAPI-Specification/issues/1900