Can't get Spectator to check responses against the specification

[ronaldwanink]

For some reason, I can't get Spectator to check the responses against the specification.

Controller has

*      @OA\Response(
*          response=200,
*          description="Successful operation",
*          @OA\JsonContent(ref="#/components/schemas/WorkplaceResource")
*       )

OpenAPI file is properly 3.0.0 generated according to Swagger Editor. Test use ->assertValidResponse(200) without giving an error, even if I deliberately make a typo in the Workplace model properties.

* @OA\Property(property="data", title="data", type="array", description="Data wrapper", @OA\Items(ref="#/components/schemas/Workplace"))
* @OA\Property(property="links", ref="#/components/schemas/Links")
* @OA\Property(property="meta", ref="#/components/schemas/Meta")

Does Spectator resolve the ref's two levels deep? Is there a way I can check this happening?

[ronaldwanink]

Update: nothing of the response seems to be validated, with or without $ref constructs.

Even a blatant incorrect response definition like

     *      @OA\Response(
     *          response=200,
     *          description="Successful operation",
     *          @OA\JsonContent(
     *              @OA\Property(type="object", property="incorrect", default="incorrect"),
     *         )
     *      )

does not lead to validation errors.

The response is parsed: changing the result code (fi to 201) leads to test failure of that specific test.. Confused of what I am seeing here.. Is it an upgrade issue? Am I overlooking something completely?

[bastien-phi]

Hi @ronaldwanink

Can you share your test to have more information ?

[ronaldwanink]

Even better, after digging deeper I think I found something that looks like the root cause of all my confusion and tests failing to flag incorrect responses. Between OpenAPI v2 and v3 the behavior of required + additionalProperties has changed (especially with ref's to ref's to schemas where constructs like AnyOf are used, which makes it very hard to notice).

Since I moved to OpenAPI v3, I am now forcing additonalProperties = false where possible, and combine this with an explicit & complete list of properties as a value for required={...}.

Ergo: what I perceived as an completely incorrect response, OpenAPI v3 interpreted as "that's just lots of additional properties" and since that was not forbidden the test did not fail.

About the combination of L5-swagger and Spectator... it works great and is easy to use, but hard to debug when something is wrong or unexpected.

Another observation, I realized I don't understand what using @OA\Schema inside other tags like @OA\Response really means.. type to study the spec a bit better.