interagent / committee

A collection of Rack middleware to support JSON Schema.
MIT License
884 stars 137 forks source link

Committee test assertions not working on object properties #399

Closed jasonsof closed 1 year ago

jasonsof commented 1 year ago

Hi!

I'm attempting to use this gem, not as a middleware, but to only to make use of the test assertions to validate an API implementation against an OAS.

I'm using a Rails app with Rspec and rack-test along with Committee but the assertions are passing even though response does not match schema

the test:

  describe "GET /posts" do
    it "conforms to response schema with 200 response code" do
      get "posts"

      assert_schema_conform
      assert_response_schema_confirm(200)
      assert_schema_conform(200)
      assert_request_schema_confirm
    end
  end
end

the actual response:

[
    {
        "idd": 1,
        "title": "Post 1"
    }
]

the schema:

{
    "openapi": "3.0.0",
    "info": {
        "version": "1.0.30",
        "title": "Posts API",
        "description": "This is the API specifications for all the endpoints that have to do with the Posts service.",
        "contact": {
            "name": "Me",
            "url": "https://www.google.com"
        }
    },
    "tags": [],
    "servers": [],
    "paths": {
        "/posts": {
            "get": {
                "tags": [
                    "Posts"
                ],
                "summary": "Get all posts",
                "description": "Get all posts",
                "operationId": "getPosts",
                "parameters": [],
                "responses": {
                    "200": {
                        "description": "OK",
                        "content": {
                            "application/json": {
                                "schema": {
                                    "$ref": "#/components/schemas/Posts"
                                }
                            }
                        }
                    }
                }
            }
        }
    },
    "components": {
        "schemas": {
            "Posts": {
                "type": "array",
                "items": {
                    "$ref": "#/components/schemas/Post"
                }
            },
            "Post": {
                "type": "object",
                "properties": {
                    "id": {
                        "type": "integer",
                        "format": "int64"
                    },
                    "title": {
                        "type": "string"
                    }
                }
            }
        },
        "responses": {}
    }
}

The full example is here

You can see there is a typo in the actual response idd: instead of id: My expectation is that test will fail because of this but it passes :(

Is there an issue with the assertions not checking object properties?

jasonsof commented 1 year ago

Was missing a required specification for the response properties 🤦

            "Post": {
                "type": "object",
                "properties": {
                    "id": {
                        "type": "integer",
                        "format": "int64"
                    },
                    "title": {
                        "type": "string"
                    }
                },
                "required": [
                    "id",
                    "title"
                ]
            }