Internal references pointing out of the Components Object fixed fields

OAI / OpenAPI-Specification

The OpenAPI Specification Repository

https://openapis.org

Apache License 2.0

28.91k stars 9.07k forks source link

Internal references pointing out of the Components Object fixed fields #2038

Closed pleothaud closed 4 months ago

pleothaud commented 4 years ago

Hi,

I recently came across an OpenAPI Spec file from a customer where some path-level parameters/responses/headers/schemas were defined through an internal reference, but having $ref pointing to a parameter/response/header/schema defined in a previous path and not to one of the Components Object fixed fields, like in:

openapi: 3.0.2
paths:
  '/a':
    parameters:
      - name: myParam
        in: header
        get:
          responses:
            '200':
              description: some description
              headers:
                header1:
                  schema:
                    type: string
  '/b':
    parameters:
      - $ref: '#/paths/~1a/parameters/0' 
      get:
        responses:
          '200':
            description: some description
              headers:
                header1:
                  $ref: '#/paths/~1a/parameters/get/responses/200/headers/header1'

For parameters and responses the spec is somehow normative and says for instance (parameters property of the Operation Object) : "The list can use the Reference Object to link to parameters that are defined at the OpenAPI Object's components/parameters."

Questions:

Does such a formulation forbid the use of a Reference Object pointing outside of #/components/parameters or not? If yes, shouldn't we make it even more clear that the targetted parameters/responses MUST be defined in OpenAPI Object's components/parameters or components/responses
Does this forbid also the use of external references for parameters and responses?

For request bodies, callbacks, examples, schemas, links and headers the spec is not normative at all when it comes to where should be put the targets of JSON References, like for instance the definition of the schema property of the Media Type Object: "The schema defining the content of the request, response, or parameter."

Question: Is the difference between parameters/responses and the others on purpose? If yes, would someone be so kind as to explain me why (apart from the reason that external references can be used), and if not would it be possible to amend the spec so that it says that internal references MUST be linked to components defined in the corresponding fixed field of the Components Object?

I've read #1679 but I'm not sure what is the answer, between @darrelmiller position and @MikeRalphson one.

Thanks!

handrews commented 4 years ago

@darrelmiller @webron @philsturgeon this is related to #2092 and therefore also #2099.

tedepstein commented 4 years ago

The OpenAPI spec does not forbid reference properties that point to locations other than /components/[schemas|responses|parameters|...]. Internal and external references to other paths are permitted.

The Components Object is provided as a convenient place to put reusable definitions. In my view, it establishes this usage as an informal standard. Idiomatic OpenAPI documents will use the designated /components/... sub-objects for internally defined, reusable definitions.

This usage is suggested by the spec and the examples embedded in the spec, but it's non-normative. You don't have to use it. And it's not any more normative for Schema Objects, Parameter Objects, etc. It's equally non-normative for all of the contained object types.

(Personally, I also think it's helpful to use the standard /components/... structure for documents that are intended to be standalone libraries or catalogs of reusable OpenAPI definitions, to be included as external references from OpenAPI documents. A library document like this may or may not be a valid OpenAPI document in itself, with the minimum required OpenAPI Object properties, an empty Paths Object, etc. But either way, this usage pattern, AFAIK, is not as widely established, and is not explicitly recommended by the spec.)

webron commented 4 years ago

As @tedepstein said, $refs can point anywhere. The only thing that's required by the spec is that the target of the reference is a valid object that 'fits' into that place. If someone decides to define a response under components, and under that response have an inline schema definition, and then from a different place reference that inline schema as a Schema Object - that's valid.

The same way, if someone simply hates the word components (for obvious reasons), they can create an extension called x-my-awesome-reusable-stuff, under it whatever they want, however they want - that can be referenced as well.

pleothaud commented 4 years ago

Shouldn't we be more normative since OpenAPI won the spec format battle? I mean we can let people do really bad stuff and support that but what would be the point of supporting bad coding practices, if I may?

Le jeu. 23 janv. 2020 à 00:58, Ron notifications@github.com a écrit :

As @tedepstein https://github.com/tedepstein said, $refs can point anywhere. The only thing that's required by the spec is that the target of the reference is a valid object that 'fits' into that place. If someone decides to define a response under components, and under that response have an inline schema definition, and then from a different place reference that inline schema as a Schema Object - that's valid.

The same way, if someone simply hates the word components (for obvious reasons), they can create an extension called x-my-awesome-reusable-stuff, under it whatever they want, however they want - that can be referenced as well.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/OAI/OpenAPI-Specification/issues/2038?email_source=notifications&email_token=AFPJCWR5DBLQXCZJVTDWLXDQ7DMTFA5CNFSM4JDPBY72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJVRFCY#issuecomment-577442443, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFPJCWQUQWELJ2GCLRPCTKLQ7DMTFANCNFSM4JDPBY7Q .

darrelmiller commented 4 years ago

@webron As a mostly irrelevant datapoint, the OpenAPI parser that I work on does not support referencing anything internally that is not inside components. If someone wants to do that, they can use a different parser.

I will note that the spec says,

A simple object to allow referencing other components in the specification

I consider it a personal failing of mine that we did not close that loophole in 3.0.

handrews commented 4 years ago

@pleothaud in the JSON Schema project we allow many practices that seem "bad" at first glance because they become interesting in large and complex projects, or in use cases that are important but tend not to be thought of by most folks.

For us, it's easier to say $ref take as URI-reference, which is resolved per RFC 3986 like any other URI-reference. As long as the resulting URI points to a schema (since I'm just talking about JSON Schema here), then that's valid.

@darrelmiller @webron since fragment syntax and semantics are determined by the resource media type, if you want to close this loophole I would do so by defining that the application/vnd.oai.openapi media type uses JSON Pointer syntax, but that only pointers prefixed with #/components/ are valid.

There would be a bit of finesse needed to insure that this doesn't get weird within Schema Objects in OAS 3.1, but I think it's do-able.

But that would "fix" this no matter where the reference came from, because the restriction would be on the valid set of fragments for the media type, rather than constraining certain references (which you can only really do within the OAS document format anyway).

pleothaud commented 4 years ago

Hi Henry,

Could you please tell when it makes sense to have internal references pointing out of #/components (and I insist on internal)

Having reviewed (litterally) thousands of OAS files I still don't see the interest of that... But I'll be really happy to learn :-)

Thanks,

Philippe

Le ven. 24 janv. 2020 à 03:52, Henry Andrews notifications@github.com a écrit :

@pleothaud https://github.com/pleothaud in the JSON Schema project we allow many practices that seem "bad" at first glance because they become interesting in large and complex projects, or in use cases that are important but tend not to be thought of by most folks.

For us, it's easier to say $ref take as URI-reference, which is resolved per RFC 3986 like any other URI-reference. As long as the resulting URI points to a schema (since I'm just talking about JSON Schema here), then that's valid.

@darrelmiller https://github.com/darrelmiller @webron https://github.com/webron since fragment syntax and semantics are determined by the resource media type, if you want to close this loophole I would do so by defining that the application/vnd.oai.openapi media type uses JSON Pointer syntax, but that only pointers prefixed with

/components/ are valid.

There would be a bit of finesse needed to insure that this doesn't get weird within Schema Objects in OAS 3.1, but I think it's do-able.

But that would "fix" this no matter where the reference came from, because the restriction would be on the valid set of fragments for the media type, rather than constraining certain references (which you can only really do within the OAS document format anyway).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/OAI/OpenAPI-Specification/issues/2038?email_source=notifications&email_token=AFPJCWSKDLDUZZ7C6FQ33VDQ7JJW5A5CNFSM4JDPBY72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJZSDUA#issuecomment-577970640, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFPJCWWFELH4YJISHUBK6GTQ7JJW5ANCNFSM4JDPBY7Q .

handrews commented 4 years ago

@pleothaud I'm not the right person to ask for an authoritative answer for OAS- that would be @darrelmiller or @webron. For OAS 3.0, where the Schema Object does not support the definitions or $defs JSON Schema keywords, I would personally point all internal references at #/components/<whatever-the-type-is>/.... But it sounds like the spec does not strictly require this.

In OAS 3.1 the Schema Object allows all JSON Schema keywords so there are more options in that case. But OAS 3.1 is not even in rc1 yet, so that's not an immediate concern.

handrews commented 5 months ago

@darrelmiller I don't think that we can change this prior to Moonwalk because restricting $ref targets in 3.2 would invalidate OADs that are valid under 3.1. PR #3732 is the first of several PRs to try to put boundaries around the more confusing reference usages without causing a breaking change, which is as close as we could get.

That PR does not forbid $ref to non-Components, and (as much as I agree with you) I don't think we could get away with doing that. But if you think we should try, I can write a PR along the same lines as #3732. Otherwise I think this should just get closed - we're already debating a totally different approachin Moonwalk.

darrelmiller commented 5 months ago

The specification is not clear enough to make the claim that you cannot $ref to non-components. But I would caution authors that some tooling may not support these $refs and it may be necessary to preprocess the document to inline the references before the tooling can work. I think we can close this issue as I believe we will make sure to be very clear about this in moonwalk.

handrews commented 4 months ago

Previous comment says we can close, so closing :-)