Error if examples in JSON schema are present

coenvl commented 3 years ago

Expected Behaviour

Since the JSON schema allows for a (set of) examples to be included in the schema definition, I would expect prance to find the definitions valid.

Minimal Example Spec

api.yaml:

openapi: "3.0.3"
info:
  title: Animal provider
  version: '1'

paths:
  /animal:
    get:
      operationId: get_animal
      responses:
        "200":
          description: A random animal
          content:
            application/json:
              schema:
                $ref: 'schema.json#/definitions/animal'

And the corresponding JSON schema

{
    "$schemas": "https://json-schema.org/draft/2019-09/schema#",
    "definitions": {
      "animal": {
        "type": "string",
        "examples": [
            "monkey",
            "lion",
            "sheep"
        ]
      }
    }
  }

Actual Behaviour

I get a very long and (imho) hard to interpret error:

ERROR in "api.yaml" [ValidationError]: ("{'description': 'A random animal', 'content': {'application/json': {'schema': {'type': 'string', 'examples': ['cat', 'donkey', 'dog']}}}} is not valid under any of the given schemas", 'oneOf', deque(['paths', '/animal', 'get', 'responses', '200']), None, [<ValidationError: "{'type': 'string', 'examples': ['cat', 'donkey', 'dog']} is not valid under any of the given schemas">, <ValidationError: "'$ref' is a required property">, <ValidationError: "Additional properties are not allowed ('description', 'content' were unexpected)">], [{'$ref': '#/definitions/response'}, {'$ref': '#/definitions/reference'}], {'description': 'A random animal', 'content': {'application/json': {'schema': {'type': 'string', 'examples': ['cat', 'donkey', 'dog']}}}}, {'oneOf': [{'$ref': '#/definitions/response'}, {'$ref': '#/definitions/reference'}]}, deque(['properties', 'paths', 'patternProperties', '^/', 'properties', 'get', 'properties', 'responses', 'patternProperties', '^([0-9X]{3})$', 'oneOf']), None)

Steps to Reproduce

I try to validate the api using prance validate api.yaml

Environment

OS: Ubuntu 18.04
Python version: 3.7
Prance version: 0.20.2
Swagger/OpenAPI version used: 3.0.3
Backend: (flex, swagger-spec-validator or openapi-spec-validator): default, i assume the openapi-spec-validator?

@jfinkhaeuser

jfinkhaeuser commented 3 years ago

Errors aren't great, see #74.

However, each of the ValidationError you're seeing comes from the validation backend, not from prance itself. In this case likely the openapi-spec-validator.

What I can tell you offhand is that in the example you posted, the referenced path /components/schemas/animal does not exist. That cannot really lead to a validation success.

I also do not understand how that second document you posted plays into it. At best, it contains a /definitions/animal, i.e. a different path. But it seems to also be in a different file, and you're not referencing a different file in your example.

In the end, without further information, I cannot really provide any other pointers. I'm sorry!

coenvl commented 3 years ago

Ah, I see I pasted the wrong code in my original post, I editted it to actually use the reference. That's what I get for playing around, moving the definitions into the file.

But as I understand by now, it is because openapi spec is not actually conform JSON schema. In fact they say in their documentation that Additional properties defined by the JSON Schema specification that are not mentioned here are strictly unsupported., and examples is not a property that is mentioned.

I fixed the issue by stripping all examples from the schema before passing it to prance.

jfinkhaeuser commented 3 years ago

Great to hear you solved it!

Yes, OpenAPI specs are a bit dodgy that way - they say they're JSON Schema, but they're not JSON Schema in a number of places. This is one of them. It doesn't help that they bring YAML thinking to JSON as well and/or mix the two.

RonnyPfannschmidt / prance