Automatic "DefaultResponse" injection does not work when path spec is inside a different file

[rico666]

If I try to break down a humongous openapi.yaml file into several logically separated chunks, it simply does not work. In the minimum example below, if you remove the $ref and uncomment the lines, it works. If you keep the $ref to a file with exact these lines (uncommented ofc) there is an error - see below.

 "/audio/play":
    "$ref": "audio/play.yaml"
    # get:
    #   summary: Play Audio
    #   tags:
    #   - Audio
    #   operationId: playAudio
    #   x-mojo-name: play_audio
    #   x-mojo-to:
    #     controller: Audio::Play
    #     action: play
    #   responses:
    #     '200':
    #       description: OK

This error occurs when using $ref:

[2022-06-11 12:20:50.11036] [496042] [trace] [uKgQHphYKn2p] Routing to a callback
[2022-06-11 12:20:50.11916] [496042] [error] [uKgQHphYKn2p] Unable to resolve "#/components/schemas/DefaultResponse" from "file:///<correct abs path>/public/audio/play.yaml". (schema/base_url) at /home/user/perl5/lib/perl5/JSON/Validator/Store.pm line 95.

[2022-06-11 12:20:50.11989] [496042] [warn] OpenAPI >>> GET /api/v1 [{"message":"Internal Server Error.","path":"\/"}]
[2022-06-11 12:20:50.12042] [496042] [trace] [uKgQHphYKn2p] 500 Internal Server Error (0.0106s, 94.340/s)

I would have assumed this should work as described in https://swagger.io/docs/specification/using-ref/

No matter if I use

$ref: audio/play.yaml
$ref: ./audio/play.yaml

or any combination thereof with quotes or if I use JSON instead of YAML.

I wonder if the "Known Issues" section in documentation applies to this. Using newest versions of Mojo and this Plugin - fresh from CPAN.

[jhthorsen]

There's a bunch of tests that include relative files, so I don't see how this doesn't work. Here is one:

https://github.com/jhthorsen/json-validator/blob/main/t/relative-ref.t

[rico666]

Let me (cross-)check the whole thing with what the tests do.

So that "known issues" in the docs has nothing to do with it (I have to admit I do not understand the implications of what this "known issues" text says).

Known issues
File references
Relative file references like the following

"$ref": "my-cool-component.json#"
"$ref": "my-cool-component.json"
Will also be placed under '#/definitions/...', again producing a spec output which will not pass validation.

?

[rico666]

Yeah, definitely does not work. The tests do work, but they test a different thing.

So I extracted the test case. Main yaml file

  "/pcversion":
    get:
       operationId: File
       x-mojo-name: get_some_version
       x-mojo-to:
         controller: Some
         action: version
       parameters:
         - $ref: "valid_include.yaml#/components/parameters/PCVersion"
       responses:
         "200":
           description: thing
           content:
             "*/*":
               schema:
                 type: string

valid include:

components:
  parameters:
    PCVersion:
      name: pcversion
      in: query
      description: version of commands which will run on backend
      schema:
        type: string
        enum:
          - 9.6.1
          - 10.1.0
        default: 10.1.0

Yeah, works - no problems. Now let's try to include a larger part as I would like to do it and as the Swagger docs suggest:

Main file:

  "/pcversion":
    $ref: "valid2.yaml"

valid2 include:

get:
  operationId: File
  x-mojo-name: get_some_version
  x-mojo-to:
    controller: Some
    action: version
  parameters:
    - PCVersion:
        name: pcversion
        in: query
        description: version of commands which will run on backend
        schema:
          type: string
          enum:
            - 9.6.1
            - 10.1.0
          default: 10.1.0
  responses:
    "200":
      description: thing
      content:
        "*/*":
          schema:
            type: string

=> BOOM

[2022-06-13 14:20:29.54155] [625405] [error] [ztoxJP-2ufZL] Unable to resolve "#/components/schemas/DefaultResponse" from "file:///data/proj/valid2.yaml". (schema/base_url) at /home/user/perl5/lib/perl5/JSON/Validator/Store.pm line 95.

Same as before. It looks like refering to an entire file without the '#/yadda/foo/baz" trail to point to some innards of it does not work. So the easier, more primitive "simply refer to the whole file" does not work, while doing some cherry picking of parts of included files does work.

Is a bug - IMHO. I can of course work around it by making the included files and the refs to them artificially more complex.

[jberger]

I think a $ref must include a pointer. If you want the entire document I'd try $ref: "valid2.yaml#/". That's just a guess

[rico666]

Ok, I'll test that (and report), but if you have a closer look at the already posted https://swagger.io/docs/specification/using-ref/ that's not according to specification. See section "Places Where $ref Can Be Used" in that document. It cannot be placed at arbitrary places - which I don't - but it can refer to a whole file as such.

[rico666]

No dice.

Unable to resolve "audio/play.yaml#/ right on startup when

"$ref": "audio/play.yaml#/

When I try to fake it as before in some "components" layer:

 "/audio/play":
    "$ref": "audio/play.yaml#/components"

and the referred file as:

components:
  get:
    summary: Play Audio
    tags:
    - Audio
    operationId: playAudio
...

Then the app starts without an error, but as soon as I try to reload the API

Unable to resolve "#/components/schemas/DefaultResponse" and ofc Fetch error Internal Server Error /api/v1 in the Swagger frontend.

So I stay at the $SUBJ of this issue. Or more precisely: "I am not able to make inter-file $ref work". Intra-file $ref: no problem.

[jhthorsen]

I updated the title since it’s incorrect and I didn’t like what it’s insinuating. I use a bunch of relative refs without any issues, so this must be a new special case.

Which version of JSON::Validator and Mojolicious::Plugin::OpenAPI do you have?

And what do you mean about “reloading the spec?” There’s a lot of internal state in JSON::Validator, so modifying the spec after it’s loaded the first time can have unexpected results.

[rico666]

No problem changing the title of this issue, as I consider that for the moment to be more precise. I'm not able to make it work for the simple use case where it should work like an include. I think I am aware of the constraints, but still no luck.

Mojolicious::Plugin::OpenAPI => 5.05 JSON::Validator => 5.08

Not sure what you mean by "reloading the spec" - I can't find having written that phrase.

[jhthorsen]

Can you try something like this?

use JSON::Validator;
my $schema = JSON::Validator->new->schema('path/to/spec.yaml')->schema->bundle;
$app->plugin(OpenAPI => {spec => $schema});

[jhthorsen]

I'm sorry, but I was wrong that I had this working in one of my projects. I had it set up differently, so the bug got camouflaged. I've fixed this in https://github.com/jhthorsen/json-validator/commit/999e424fd6a89a027514483bd5afd01694037822 now and I've added a test in db243b2e3061e4bdfedf86816d80dcb2bb94beee.

Hope this works for you as well 👍