openapi-library / OpenAPIValidators

Use Jest or Chai to assert that HTTP responses satisfy an OpenAPI spec
MIT License
190 stars 35 forks source link

Nullable ignored on property values #240

Open Foorack opened 3 years ago

Foorack commented 3 years ago

OpenAPI version 3.0

Is your feature request related to a problem? Please describe.

Current validation does not respect nullable on propertiy fields.

User.yaml:

title: User
type: object
properties:
  accountDeletionDate:
    type: string
    format: date
    nullable: true

Response is accountDeletionDate is null. Error message: res did not satisfy it because: accountDeletionDate should be string

Describe the solution you'd like For the validation to pass.

Are you going to resolve the issue? I can make a PR if that will be fastest.

rwalle61 commented 3 years ago

Hi @Foorack, thanks for raising this 🙂 It seems nullable has been removed between OpenAPI 3.0.3 and 3.1.0, so I've asked the official maintainers to clarify how you should define nullable properties.

In any case, I think validation is done by one of our dependencies, openapi-response-validator, perhaps on this line. That code seems to support nullable as you're using it, so I'm not sure why you case doesn't work. If you need any to changes to validation, I think we'll need to submit a pull request to their repo.

To pinpoint this bug would you mind recreating it using this template?

rwalle61 commented 3 years ago

Fyi the OpenAPI maintainers got back to me on how to define nullable properties in OpenAPI, let me know if it works!

Foorack commented 3 years ago

Thank you @rwalle61! We will try that and get back to you.

Foorack commented 3 years ago

I'm having trouble testing this, as we are generating code with openapi-generator, and it's going crazy over that syntax. It doesn't expect type being anything other than a string...

rwalle61 commented 3 years ago

hmm the dependency we use to validate OpenAPI files does not seem to like the type being an array.

If I change our bugRecreationTemplate/openapi.yml to have type: 'null':

openapi: 3.0.0
info:
  title: Example OpenApi 3 spec
  description: Use to recreate a bug
  version: 0.1.0
paths:
  /recreate/bug:
    get:
      responses:
        '200':
          description: OK
          content:
            application/json:
              schema:
                type: 'null'

then this res passes the test:

 const res = {
    status: 200,
    req: {
      method: 'GET',
      path: '/recreate/bug',
    },
    body: null,
  };

However if I use an array in the openapi.yml:

        '200':
          description: OK
          content:
            application/json:
              schema:
                type: ['null']

Then our validator dependency outputs this validation error:

 Error: Invalid OpenAPI spec: [
  {
    instancePath: '/paths/~1recreate~1bug/get/responses/200/content/application~1json/schema/type',
    schemaPath: '#/properties/type/type',
    keyword: 'type',
    params: { type: 'string' },
    message: 'must be string'
  },
... etc

However, JSON schema specifically recommends type: [string, 'null'] for nullable strings.

Our validator dependency is openapi-response-validator. They use AJV, so perhaps the issue is in there. However if openapi-generator does not accept type as an array either, then I suspect we need to investigate further and clarify the relation between OpenAPI, JSON schema, AJV and openapi-response-validator.

Unfortunately, I don't have time to do this at the moment, but would appreciate if anyone else does!