search

jhthorsen / json-validator

:cop: Validate data against a JSON schema
https://metacpan.org/release/JSON-Validator
56 stars 58 forks source link

v3 $ref support for private absolute paths #138

Closed gaiederer closed 5 years ago

gaiederer commented 5 years ago

Steps to reproduce the behavior

In my mojolicious app file:

  # load routes based on openAPI YAML file
  $app->plugin(
    "OpenAPI" => {
      url => $app->home->rel_file("openapi.yaml"),
      schema => 'v3', 
    }
  );

where $app->home->rel_file("openapi.yaml") contains:

openapi: 3.0.0
info:
  description: |
    This is the "content" API. It returns content, or information about
    content, depending on the endpoint
  version: GIT_TAG_GOES_HERE
  title: api-content
  contact:
    email: email@my_email.com
servers:
  - url: /api/v2/content
    description: Available service
tags:
  - name: /
    description: |
      returns the endpoints that this namespace responds to.
  - name: version
    description: |
      returns the GitLab Tag of this installed namespace.

paths:
  /:
    get:
      tags:
        - /
      description: |
        returns the endpoints that this namespace responds to.
      responses:
        '200':
          $ref: '#/components/responses/rootResponseSuccess'
        '400':
          $ref: '/srv/api-common/openapi.yaml#/components/responses/responseNotValid'
  /version:
    get:
      tags:
        - version
      description: |
        returns the GitLab Tag of this installed namespace.
      responses:
        '200':
          $ref: '/srv/api-common/openapi.yaml#/components/responses/versionResponseSuccess'
        '400':
          $ref: '/srv/api-common/openapi.yaml#/components/responses/responseNotValid'

components:
  responses:
    rootResponseSuccess:
      description: output from a / request
      content:
        application/json:
          schema:
            type: string

and the reference /srv/api-common/openapi.yaml document contains:

openapi: 3.0.0
info:
  description: >
    This document describes common responses and parameters that
    are used across the API V2 set of requests and responses
  version: GIT_TAG_GOES_HERE
  title: api-common
  contact:
    email: support-laads@earthdata.nasa.gov
paths:
  /dummy:
    get:
      description: "paths tag is required to pass validation, but not actually implemented"
      responses:
        200:
          description: "always returned"
components:
  responses:
    responseNotValid:
      description: Invalid status value
    responseNotFound:
      description: Resource not found
    versionResponseSuccess:
      description: output from a versions request

Expected behavior

I would expect Validator.pm to find the $ref file, because it does exist in the specified location and is readable.

Actual behavior

When I start mojolicious, it complains:

[JSON::Validator] Loading schema from file: /srv/api-content/openapi.yaml
[JSON::Validator] Using root_schema_url of '/srv/api-content/openapi.yaml'
Can't load application from file "/srv/api-mojo/bin/startup.pl": Can't load application from file "/srv/api-mojo/bin/../bin/mojo.pl": Can't load application from file "/srv/api-common/bin/api-v2.pl": Can't load application from file "/srv/api-content/bin/content.pl": Possibly a typo in schema? Could not find "/srv/api-common/openapi.yaml#/components/responses/responseNotValid" in "/srv/api-content/openapi.yaml" (/srv/api-common/openapi.yaml#/components/responses/resp

Basically, it cannot find the $ref file /srv/api-common/openapi.yaml, even though it exists in that location.

I was able to trace the issue to two places:

When I modified JSON::Validator.pm as follows, I was able to get everything to load properly (although I am not submitting a merge request, because I don't know that it might not break some other $ref use case -- there's a lot going on in this sub).

431a432,434
>   # handle absolute paths to files
>   return new Mojo::File($location) if ($location && -f $location);
>   
448a452
>     last if !$ref or ref $ref;
451,453c455
<     last if !$ref or ref $ref;
<     $fqn = $ref =~ m!^/! ? "#$ref" : $ref;
<     ($location, $pointer) = split /#/, $fqn, 2;
---
>     ($location, $pointer) = split /#/, $ref, 2;
456a459,460
>     $fqn = $ref;
>     $fqn = "#$ref" if $fqn =~ m!^/!;
jhthorsen commented 5 years ago

The special case for "/" is to be compatible with OpenAPI v2. I can agree that it's not a very good solution to put this in as a generic logic inside _resolve_ref(), but I also can't see any good reasons to use absolute paths.

Do you actually have a /srv/ directory on your filesystem?

gaiederer commented 5 years ago

We need absolute paths because the api for the system is spread across a number of git projects. Each project is its own mojolicious plugin with model and controllers, and some modules are shared between projects. Since mojolicious runs inside docker, we can mount the deployed directories anywhere we want, but they are relative to a common system root, not the web root or any specific project root. /srv was a convenient, not already used path to mount everything under.

jhthorsen commented 5 years ago

I see. I think the fix here is to add support for “file://...”.

What do you think about that?

jhthorsen commented 5 years ago

I just added support for "file://", since I cannot come up with another solution for this.

gaiederer commented 5 years ago

I'll try it out. The issue is that mojolicious is not yet fully running (it is still initializing when it starts the plugin), so it cannot serve anything.

gaiederer commented 5 years ago

Maybe not an issue, since Mojo is using its user agent to make the query, it is not actually serving it.

jhthorsen commented 5 years ago

I don’t understand what you’re talking about. “file://...” will look up a file on disk. It won’t use the user agent at all. So you do:

$ref: 'file:///srv/api-common/openapi.yaml#/components/responses/responseNotValid'

gaiederer commented 5 years ago

Sorry for the confusion -- just thinking out loud and didn't actually check the implementation. My apologies.

gaiederer commented 5 years ago

Finally got to try it out, works like a charm. Thanks much.