Reference objects don't combine well with “nullable”

[Rinzwind]

Perhaps more of a question than an issue report, but I am not sure how to combine “nullable” with a reference object; I have the impression they don't combine well?

In Swagger-UI issue 3325 I posted an example with the following schema. The intent is to express that both “dataFork” and “resourceFork” contain a Fork, except that “resourceFork” can also be null, but “dataFork” cannot be null. As was pointed out though, the schema is not in accordance with the OpenAPI v3.0.0 specification, due to the use of JSON Schema's {"type":"null"} which has not been adopted in OpenAPI.

...
"File": {
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "dataFork": { "$ref": "#/components/schemas/Fork" },
    "resourceFork": {
      "anyOf": [
        { "type": "null" },
        {"$ref": "#/components/schemas/Fork" } ] },
    ...

I considered I can instead express the example as follows, which as far as I can tell is at least in accordance with the OpenAPI specification? But the use of “anyOf” with only one schema inside seems clumsy, is there a better way to handle this? Essentially, a better way to combine “nullable” with the reference object?

... 
"File": {
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "dataFork": { "$ref": "#/components/schemas/Fork" },
    "resourceFork": {
      "nullable": true,
      "anyOf": [
        { "$ref": "#/components/schemas/Fork" } ] },
    ...

[handrews]

@Rinzwind I use allOf with one entry regularly (for one entry, the *Of's behave the same and allOf is the simplest). And I do it exactly as you show, with the $ref in the allOf and everything else outside of it.

This is how boolean flags like nullable, readOnly, writeOnly, etc. are intended to be used AFAIK (certainly in JSON Schema for readOnly and as of draft-07 writeOnly).

See also #1389 for nullable vs "type": "null" discussion.

[tedepstein]

About the example given in the original post: if we're interpreting nullable as described in #1389, then I don't think an object with resourceFork: null would be valid against the File schema:

    File:
      type: object
      properties:
        name:
          type: string
        dataFork:
          $ref: '#/components/schemas/Fork'
        resourceFork:
          nullable: true
          anyOf:
          - $ref: '#/components/schemas/Fork'

    Fork:
      type: object
      description: Information aspect of a file...

I added a 'Fork' stub schema to illustrate. Fork itself is not nullable, it requires an object value.

(And BTW, even if Fork didn't specify a type, it's not clear to me that null is allowed. Does anyone know whether schemas without a type and without an explicit nullable: true would permit null values?)

In the File schema, resourceFork has nullable: true. But in OpenAPI Schema Object, as in JSON Schema, constraints are cumulative and cannot be relaxed or overridden. The allOf expression means that all assertions in the referenced subschema apply here. Those inherited assertions, whether implicit or explicit, prohibit a null value. Adding a new, more permissive type assertion on top of that doesn't change anything.

So I think the first attempted schema in the original post is closer to the mark. But we have to do something kind of awkward to allow either null or Fork:

    File:
      type: object
      properties:
        name:
          type: string
        dataFork:
          $ref: '#/components/schemas/Fork'
        resourceFork:
          oneOf:
          - $ref: '#/components/schemas/Fork'
          - $ref: '#/components/schemas/NullValue'

    Fork:
      type: object
      description: Information aspect of a file...

    NullValue:
      # Allow null values.
      nullable: true
      # Disallow any non-null value.
      not:
        anyOf:
        - type: string
        - type: number
        - type: boolean
        - type: object
        - type: array
          # Array schema must specify items, so we'll use an 
          # empty object to mean arrays of any type are included.
          items: {}

It might be cleaner to start with a nullable Fork object, and restrict it to non-null as needed:

    File:
      type: object
      properties:
        name:
          type: string
        dataFork:
          allOf:
          - nullable: false
          - $ref: '#/components/schemas/Fork'
        resourceFork:
          $ref: '#/components/schemas/Fork'

    Fork:
      type: object
      nullable: true
      description: Information aspect of a file...

[Rinzwind]

But in OpenAPI Schema Object, as in JSON Schema, constraints are cumulative and cannot be relaxed or overridden.

I'm not sure that's right. The JSON schema validation specification says that “validation keywords typically operate independent of each other” (section 4.3) (*), but then makes some exceptions.

The OpenAPI Specification 3.0.2 isn't exactly clear on how it defines “nullable”, but I think it would have to be an exception too. Otherwise “nullable” would seem to always conflict with other keywords, in particular “type”. For example, consider the schema {"nullable":true,"type":"object"}. The instance null would not match this schema if it has to match each keyword independently, as it matches the “nullable” keyword, but not the “type” keyword (null is not an object). In order for this schema to have the intended semantics, I think “nullable” would have to be defined as overriding other keywords: if “nullable” has the value true and the instance is null, it overrides all other keywords. So null would match the schema because null matches “nullable”, and “type” does not apply. Similarly, null would also match the schema {"nullable":true,"anyOf":[{"type":"object","nullable":false}]} because the “anyOf” does not apply.

(*) Section 4.3 in the version of the JSON schema validation specification linked to from the OpenAPI specification 3.0.2. In newer versions, the sentence moved to section 3.1.1.

But we have to do something kind of awkward to allow either null or Fork: […]

I hadn't considered that, if your schema “NullValue” is correct and allowed by OpenAPI, that seems like a way of sneaking the easier to understand {"type":"null"} back in!

[handrews]

Hi @Rinzwind , I'm one of the current primary editors of the JSON Schema spec (and yes, it's my fault that draft-08 is taking forever to appear 😭 )

What "validation keywords typically operate independently" is that in the majority of cases, each keyword's validation outcome can be determined without examining any other keyword.

As of draft-06 and later, that is true of any assertion keyword, such as type, maxLength, or required. These keywords produce their own pass/fail validation results.

Some applicator keywords (which apply subschemas and combine their assertion results, rather than producing their own results) do interact. The most notorious being that determining to which properties additionalProperties applies depends on properties and patternProperties. The fact that that inter-keyword dependency causes so many misunderstandings is good evidence of why we try to minimize this behavior.

Sadly, in draft-04 (which is effectively what OAS uses- there's not really a draft-05, it's a long story), the exclusiveMaximum and exclusiveMinimum assertion keywords were not independent keywords, which also causes problems. That has since been changed.

OAS's nullable is another assertion keyword that is not independent. And that causes many problems when you write complex schemas with *Of and similar keywords. Which is why we removed assertions with that behavior, and have done other things to mitigate the problems seen in the applicator keywords.

That's probably more than you wanted to know, but the upshot is that nullable (if indeed it's default is false and applies when type is absent) violates two fundamental principles of JSON Schema keyword design:

• The empty schema MUST successfully validate any and all possible instances
• Assertion keywords SHOULD function independently; failure to follow this guideline produces behavior that is very confusing in complex schemas

[Rinzwind]

OAS's nullable is another assertion keyword that is not independent. […] nullable […] violates two fundamental principles of JSON Schema keyword design

OK, but regardless of the violations of nullable with respect to those principles, the intent of the OpenAPI 3.0.2 specification was at least that nullableis not independent, and kind of “overrides” other keywords such that null is validated by all of the following schemas?

• { "nullable": true, "type": "object" }
• { "nullable": true, "anyOf": [ { "type": "object" } ] }
• { "nullable": true, "anyOf": [ { "type": "object", "nullable": false } ] }
• { "nullable": true, "anyOf": [ { "$ref": "#/components/schemas/Fork" } ] }

[tedepstein]

@Rinzwind,

There are discussions in #1389 and #1900 essentially trying to figure out how to interpret nullable.

OK, but regardless of the violations of nullable with respect to those principles, the intent of the OpenAPI 3.0.2 specification was at least that nullableis not independent, and kind of “overrides” other keywords such that null is validated by all of the following schemas?

That's how I assumed it worked. But then I dug deeper and now I'm much less certain of how the founding fathers intended nullable to work, or if they ever discussed use cases cases where nullable might conflict with enum, how it might behave in allOf inheritance hierarchies, etc.

We could say that nullable always takes precedence over other constraints, such that those other constraints only apply to non-null values. If that's what it means, we should document that, and we should expect that it's going to break some implementations that assumed otherwise.

I think we're still waiting to hear from the TSC about this one.

[Rinzwind]

There are discussions in #1389 and #1900 essentially trying to figure out how to interpret nullable.

It's not so clear to me what the alternative interpretation(s) is / are by which any of the following schemas would not allow null, could you clarify?

1. { "nullable": true, "type": "object" }
2. { "nullable": true, "anyOf": [ { "type": "object" } ] }
3. { "nullable": true, "anyOf": [ { "type": "object", "nullable": false } ] }
4. { "nullable": true, "anyOf": [ { "$ref": "#/components/schemas/Fork" } ] }

I think we can assume that in any interpretation, at least schema 1 should allow null otherwise there's no point to “nullable”? The second and third schema are essentially the same, as “nullable” is specified as having false as its default value, and the fourth schema is just another variation on these two where a reference is used. So I assume the point of disagreement here would be whether there's a difference between combining “nullable” with “type” (schema 1) and combining “nullable” with “anyOf” (schemas 2,3,4)? I would argue that in both cases, there's a validation keyword that only allows objects, but which is overridden by the “nullable” keyword to also allow null.

[tedepstein]

It's not so clear to me what the alternative interpretation(s) is / are by which any of the following schemas would not allow null, could you clarify?

One of the interpretations under discussion is nullable: true --> type: [t, 'null']. Under that interpretation, your schema 1 would allow null. Schemas 2 and 3 would not. Schema 4 would only allow null if the referenced Fork schema is also nullable.

Taking schema 2 as an example:

1. Start with nullable: true and apply the interpretation nullable: true --> type: [t, 'null'].
   
   
   • In this case, there is no type assertion, so t is the list of non-null types: string, number, integer, boolean, object, and array.
     
     
   • We'd append null to that list, so we have: type: [string, number, integer, boolean, object, array, null]. It's the complete list of all possible types, so it's equivalent to having no type constraint at all.
     
     
2. The "anyOf": [ { "type": "object" } ] applies independently. This disallows null.

So I assume the point of disagreement here would be whether there's a difference between combining “nullable” with “type” (schema 1) and combining “nullable” with “anyOf” (schemas 2,3,4)?

It's not limited to that. Interpretation of nullable also has implications in terms of how it combines with enum, and how it works with the other *Of applicators, including allOf which is commonly used for inheritance hierarchies.

I would argue that in both cases, there's a validation keyword that only allows objects, but which is overridden by the “nullable” keyword to also allow null.

The problem with that interpretation is that JSON Schema doesn't have any way of "overriding" a constraint. Constraints can be added, but they can't be taken away.

If you really want to introduce "override" as a whole new mechanism in OpenAPI schema, then we have no clean way of translating OpenAPI schemas to JSON Schemas. And in that case, API implementations, tools and libraries can't make use of standard JSON Schema validators; they have to use specialized OAS schema validators. And this mismatch makes for a steeper learning curve for developers moving from one language to the other. That's the gap we're trying to narrow, not just with nullable, but in other areas as well.

There is another possible interpretation that doesn't break with JSON Schema: nullable: true --> anyOf: [s, type: 'null']

That is much closer to an override. In this case, s means "the rest of the schema, aside from nullable." So null values are allowed, and non-null values are subject to the other constraints in s, including type, enum, *Of, etc. All four of your example schemas would allow nulls in this case.

But there are some issues:

• We'd have to change the spec to explain that enum, *Of and other keywords are not equivalent to JSON Schema, because they are subject to modification by nullable.
• If we amend the OpenAPI spec to say that this is the official interpretation, we will break some current implementations. (That's true of any official interpretation, because implementers have been interpreting nullable in different ways.)
• Nullable types can never be true subtypes. We have this very counterintuitive situation where allOf: [ {$ref: '#/foo'} ], nullable: true is interpreted as anyOf: [ {allOf: [ {$ref: '#/foo'} ] }, {type: 'null'} ] The allOf gets "wrapped" by this interpretation. The schema is no longer guaranteed to be a valid instance of #/foo, so it can't be treated as a subtype. Code generators will have a hard time with this, and API designers will be unpleasantly surprised to see this interpretation.

[Rinzwind]

Nullable types can never be true subtypes. We have this very counterintuitive situation where {allOf: ['#/foo'], nullable: true} is interpreted as anyOf: [{type: 'null'}, {allOf: ['#/foo']}]

Perhaps I'm missing something, but I don't get why that would be counterintuitive? I opened this issue because I wanted to express the JSON schema { "anyOf": [ { "type": "null" }, { "$ref": "#Fork" } ], and wondered whether the same thing could be expressed by the OpenAPI schema { "nullable": true, "anyOf": [ { "$ref": "#Fork" } ] }. The fact that the latter would by definition be equivalent to the former would not seem counterintuitive to me at all, but exactly what I intended. (Strictly speaking, according to your transformation rule, the latter OpenAPI schema would be equivalent to the JSON schema { "anyOf" : [ { "anyOf" : [ { "$ref": "#Fork" } ] }, { "type" : "null" } ] }, but I don't think that makes a difference, because the ‘inner’ “anyOf” can be simplified to just the reference.)

[tedepstein]

I opened this issue because I wanted to express the JSON schema { "anyOf": [ { "type": "null" }, { "$ref": "#Fork" } ], and wondered whether the same thing could be expressed by the OpenAPI schema { "nullable": true, "anyOf": [ { "$ref": "#Fork" } ] }. The fact that the latter would by definition be equivalent to the former would not seem counterintuitive to me at all, but exactly what I intended.

In your case, you knew exactly what you wanted, and the wrapping of your schema into an anyOf gives you just that. But it's much less intuitive if you're using allOf to define subtypes. Unless you're truly "thinking in JSON schema" and you know exactly how nullable gets interpreted, you wouldn't expect the presence of nullable to break inheritance. But it does.

(Strictly speaking, according to your transformation rule, the latter OpenAPI schema would be equivalent to the JSON schema { "anyOf" : [ { "anyOf" : [ { "$ref": "#Fork" } ] }, { "type" : "null" } ] }, but I don't think that makes a difference, because the ‘inner’ “anyOf” can be simplified to just the reference.)

Yes, that's right.

[Rinzwind]

But it's much less intuitive if you're using allOf to define subtypes.

I'm not quite sure I get what you mean, but are you perhaps talking about an example such as the following? “Override” may have been a bad choice of words with respect to an example like this, but I did not mean to imply that this would allow “foo” to be null in the interpretation of “nullable” I had in mind. There are two schemas within the “allOf”, while the second one allows the instance {"foo": null}, the first one does not; therefore, the schema as a whole does not either.

{ "allOf": [
  { "type": "object", "properties": { "foo": { "type": "integer" } } },
  { "type": "object", "properties": { "foo": { "type": "integer", "nullable": true } } } ] }

This is, as far as I get, in keeping with your transformation rule to JSON schema:

{ "allOf": [
  { "type": "object", "properties": { "foo": { "type": "integer" } } },
  { "type": "object", "properties": { "foo": { "anyOf": [ { "type": "integer" }, { "type": "null" } ] } } } ] }

The same applies: {"foo": null} is valid against the second subschema, but not the first; therefore it is not valid against the schema as a whole.

[tedepstein]

@Rinzwind , I have to get back to my day job. ;-) If I have time, I'll try to parse what you've written here and respond.

But IMO, there is no happy solution that preserves nullable as a keyword, works in harmony with JSON Schema, behaves the way people want and expect it to, and keeps breaking changes to a minimum. The best solution is to deprecate nullable and replace it with type: [null, t].

[Rinzwind]

I have to get back to my day job. ;-) […] The best solution is to deprecate nullable and replace it with type: [null, t].

Oh sure, no problem, and I would also prefer for OpenAPI to stick to JSON schema's {"type":"null"}. I was just trying to grasp your earlier statement that the OpenAPI schema I posted at the opening of this issue might not allow “resourceFork” to be null as I had intended.

Musings: the only possible benefit I see to “nullable” is that it makes it very easy to determine whether the instance null is allowed by a schema or not, one only needs to look at the schema's “nullable” keyword. Or at least this is the case if “nullable” would be defined according to these two transformation rules from OpenAPI schema to JSON schema:

{ "nullable": true, … } → { "anyOf": [ { … }, { "type": "null" } ] } { "nullable": false, … } → { "allOf": [ { … }, { "not": { "type": "null" } } ] }

Keep in mind that “nullable” is implicitly included in every OpenAPI schema with default value false.

This would imply though that the OpenAPI schema {"enum":[null]} would not allow null. It is, by default value, equivalent to {"enum":[null], "nullable": false}. By the above transformation rules, it would then be equivalent to a JSON schema that does not validate the instance null. That probably conflicts with some people's expectations.

We could instead use these two transformation rules:

{ "nullable": true, … } → { "anyOf": [ { … }, { "type": "null" } ] } { "nullable": false, … } → { … }

But that would imply losing the possible benefit of “nullable”. A schema like {"enum":[null],"nullable":false} would allow null, despite having the value false for “nullable”. It would not be sufficient to look at the “nullable” keyword's value to know whether the schema validates the instance null or not.

[Rinzwind]

One of the interpretations under discussion is nullable: true --> type: [t, 'null'].

I hadn't fully grasped what you meant by this. I'm not sure I do now; if you find the time to comment, I was wondering whether the following correctly reflects that interpretation w.r.t. to null for the following four OpenAPI schemas?

1. {"nullable": true, "enum":[42, null]}
   ⇒ null is valid
2. {"nullable": true, "enum":[42]} ⇒ null is not valid
3. {"nullable": false, "enum":[42, null]} ⇒ null is not valid
4. {"enum":[42, null]} ⇒ null is not valid

[tedepstein]

I was wondering whether the following correctly reflects that interpretation w.r.t. to null for the following four OpenAPI schemas?

1. {"nullable": true, "enum":[42, null]} ⇒ null is valid
2. {"nullable": true, "enum":[42]} ⇒ null is not valid
3. {"nullable": false, "enum":[42, null]} ⇒ null is not valid
4. {"enum":[42, null]} ⇒ null is not valid

Yes, exactly. :-)

In the fourth example, null is not valid because nullable: false is the default, even if type is not specified. If it weren't for that unfortunate design decision, nullable: true --> type: [t, 'null'] would be easy enough to reconcile with JSON Schema. But nullable: false by default blows the whole thing up.

[Rinzwind]

One of the interpretations under discussion is nullable: true --> type: [t, 'null'].

I was wondering whether the following correctly reflects that interpretation w.r.t. to null for the following four OpenAPI schemas?

2. {"nullable": true, "enum":[42]} ⇒ null is not valid

Yes, exactly. :-)

Ok, I think I got it then. I think I have to put the question back to @handrews, who was previously the first one to respond to this issue, saying:

I use allOf with one entry regularly (for one entry, the *Of's behave the same and allOf is the simplest). And I do it exactly as you show, with the $ref in the allOf and everything else outside of it.

This is how boolean flags like nullable, readOnly, writeOnly, etc. are intended to be used AFAIK (certainly in JSON Schema for readOnly and as of draft-07 writeOnly).

See also #1389 for nullable vs "type": "null" discussion.

@handrews: I'm not sure you intended to imply that the example I gave (opening comment of this issue) expresses (as intended) that both “dataFork” and “resourceFork” contain a Fork, except that “resourceFork” can also be null. Because, as far as I understand, this would imply that the OpenAPI schema { "nullable": true, "anyOf": [ { "$ref": "#/Fork" } ] } would be equivalent to the regular JSON schema { "anyOf" : [ { "type" : "null" }, { "$ref": "#/Fork" } ] }. You simultaneously seem to be, in the opening post of issue #1389, the originator of the idea that nullable is related to a different transformation of the “type” keyword, which according to @tedepstein implies the example does not express what was intended, as the schema would actually be equivalent to { "type": [ "object", "string", … , "null" ], "anyOf": [ { "$ref": "#/Fork" } ] }. I'm wondering what your take on this is?

Using a simpler example, I think the question is similar to whether this OpenAPI schema:

{ "nullable": true, "type": "integer", "enum": [ 42 ] }

… is equivalent to this JSON schema: (allows null)

{ "anyOf": [ { "type": "null" }, { "type": "integer", "enum": [ 42 ] } ] }

… or to this one: (does not allow null)

{ "type": [ "integer", "null" ], "enum": [ 42 ] }

[handrews]

@Rinzwind sorry, I didn't notice when you mentioned me a few months ago. In any event, @tedepstein is correct. I don't have time to read every comment in detail, but I think it just didn't occur to me that anyone would try the "nullable wraps the allOf in an anyOf" approach at the time. Fortunately this is all getting clarified in 3.0.3 (#2046 and #2050) and converged further with JSON Schema in 3.1 (#1977)

[Rinzwind]

@handrews: no problem; I haven't kept up with the discussions related to nullable, but I'm glad to see the plan is to deprecate it in favor of simply allowing {"type":"nullable"}.

[handrews]

@Rinzwind yeah this is all part of converging with JSON Schema while facilitating a transition period. SO in OAS 3.1 you will be able to use either {"type": ["integer", "null"]} or {"type": "integer", "nullable": true} which will have the same effect. But one will work identically in OAS 3.1 and in regular JSON Schema, while the other will work identically in OAS 3.0 and OAS 3.1.

I'm not 100% sure what the deprecation policy is, but I think nullable will be deprecated but allowed in OAS 3.1 (and future OAS 3.x), and perhaps removed in OAS 4? Given the pace of OAS releases that should give plenty of time for a migration path.

[tedepstein]

@handrews wrote:

Fortunately this is all getting clarified in 3.0.3 (#2046 and #2050) and converged further with JSON Schema in 3.1 (#1977)

Here is the proposal to clarify nullable. It is currently under review by the TSC, hopefully to be included in a v3.0.3 patch release of the OpenAPI spec:

Proposal: Clarify Semantics of nullable in OpenAPI 3.0

[philsturgeon]

This was fixed in #1977 so we can close this.

[sebastien-rosset]

It would be useful to provide some OAS examples for "nullable" properties, i.e. properties who can have a non-null value or a null value. Reading the OAS 3.1.0 wording, my understanding is the new recommended approach is to write something like this when the value is not a primitive type:

billTo:
  oneOf:
    - type: 'null'
    - $ref: "#/definitions/Address"

When the value is a primitive type, a nullable property can be written as:

name:
  - type: ['null', 'string']

which is equivalent to:

name:
  oneOf:
    - type: 'null'
    - type: string

As a shortcut one might be tempted to write the schema below, but afaik this is not valid.

billTo:
  - type: ['null', '#/definitions/Address']

In 3.0.2, the "type" attribute is a string (i.e. cannot be an array):

type - Value MUST be a string. Multiple types via an array are not supported.

In 3.1.0, the type attribute has changed, now it can be an array of primitive types:

type - Used to restrict the allowed types as specified in JSON Schema The companion validation vocabulary also includes a "type" keyword which can independently restrict the instance to one or more primitive types.

Because the OAS has changed, an example would go a long way to clarify the specification. It does not help that multiple tools and validators behave differently in this area, leading to inconsistencies, confusion and ambiguity.

An example for nullable array would be helpful too. Arrays can be empty, and it looks like they cannot have a null value, though I couldn't find an explicit statement in the JSON schema. If so, it looks like the OAS author would have to write something like this (?)

billTo:
  oneOf:
    - type: null
    - type: array
       - $ref: "#/definitions/Address"

[tedepstein]

@sebastien-rosset , if you have not done so already, please have a look at the proposal to clarify nullable:

Proposal: Clarify Semantics of nullable in OpenAPI 3.0

It would be useful to provide an OAS example with a nullable property. Reading the OAS 3.1.0 wording, my understanding is the new recommended approach is to write something like this:

billTo:
  oneOf:
    - type: null
    - $ref: "#/definitions/Address"

Small correction: you need single or double quotes around the null, as in type: 'null'.

I assume the schema at #/definitions/Address is of type: object. If that Address schema is nullable, meaning it has type: [object, 'null'] or type: object, nullable: true, then you don't need the oneOf construct in billTo. But if it's non-nullable, you would need to use oneOf as in your example, above.

Keep in mind that this will work in OpenAPI 3.1, which supports full JSON Schema. But it will not work in OpenAPI 3.0, because null is not permitted as a type. There is a roundabout way to specify a schema that requires a null value:

NullValue:
  not:
    anyOf:
    - type: string
    - type: number
    - type: boolean
    - type: object
    - type: array

By eliminating all types other than 'null', only the value null conforms to this schema.

Then you can reference it like this in OpenAPI 3.0:

billTo:
  oneOf:
    - $ref: "#/definitions/Address"
    - $ref: "#/definitions/NullValue"

From your comments and examples in #2120:

type: object
properties:
  address:
    $ref: '#/components/schemas/Address'
    nullable: true

The object containing a $ref property is a Reference Object, which follows the JSON Reference specification. JSON Reference says, "Any members other than $ref in a JSON Reference object SHALL be ignored."

The Technical Steering Committee is currently exploring possible ways to allow sibling properties, and to define the merge semantics in those cases.

The online validator does not complain this is an invalid OAS, but AFAIK this is not compliant with the spec. My interpretation is this snippet is not compliant with the spec because the OAS spec states:

properties - Property definitions MUST be a Schema Object and not a standard JSON Schema (inline or referenced). Alternatively, any time a Schema Object can be used, a Reference Object can be used in its place. This allows referencing definitions instead of defining them inline.

Correct. I cannot speak to the issue with the online validator you're using. This repo is concerned only with the specification, not with any specific commercial or open source implementations.

type: object
properties:
  address:
    allOf:
     - $ref: '#/components/schemas/Address'
    nullable: true

If Address is not already nullable, you cannot make it nullable in a subtype. JSON Schema, and therefore OpenAPI's Schema Object, is constraint-based. A subtype can add constraints, but it cannot relax constraints specified in the allOf subschemas.

type: object
properties:
  address:
    anyOf:
     - $ref: '#/components/schemas/Address'
    nullable: true

A few issues with this:

1. anyOf means that the value must conform to one or more of the subschemas specified or referenced therein. If anyOf has only one subschema, i.e. one element in the anyOf array, then it's logically the same as allOf. In both cases, the value must conform to the reference subschema. If you mean to say that the value must be a valid Address or must be null, you can refer to my examples above.
2. nullable in a schema by itself actually doesn't do anything. The value null is allowed if you don't specify a type.

type: object
properties:
  address:
    oneOf:
     - $ref: '#/components/schemas/Address'
    nullable: true

Same comments as above, with the anyOf example.

[sebastien-rosset]

@sebastien-rosset , if you have not done so already, please have a look at the proposal to clarify nullable:

Proposal: Clarify Semantics of nullable in OpenAPI 3.0

Yes, I have read it. One reason I am advocating for examples, is that without them, it's pretty challenging to understand what is valid in each of these specs. You have to read OAS 3.1.0, 3.0.2 and the JSON schema, carefully parse some of the crucial sentences and compare them. I think all of this confusion can go away with a few good examples.

[sebastien-rosset]

Small correction: you need single or double quotes around the null, as in type: 'null'.

Done.

I assume the schema at #/definitions/Address is of type: object. If that Address schema is nullable, meaning it has type: [object, 'null'] or type: object, nullable: true, then you don't need the oneOf construct in billTo. But if it's non-nullable, you would need to use oneOf as in your example, above.

Yes, in this example, Address would not be nullable. For example, suppose there is a shipping address, installedAt address, billTo address, etc. Some of the "address" properties can have a null value, others are mandatory. It may be a contrived example, but there are other cases when this could occur to.

[sebastien-rosset]

The Technical Steering Committee is currently exploring possible ways to allow sibling properties, and to define the merge semantics in those cases.

Interesting. Reading the entire thread, I can see the challenges of merging sibling properties, but perhaps naively, it would be really nice to specify that a $ref value can be null (without being required to make the referenced type: ['object', 'null']). That way generated SDKs could be easy to use, especially for casual users, because it's easy to generate simple accessor methods that can take a "pointer" value (I am using this term loosely here, the specifics depend on the programming language).

On the other hand, with "oneOf", it can be challenging because the SDK generator does not know the intent of the author. Maybe later the author will add a third possible value, such as an alternate way to do billing with a bitcoin URL.

billTo:
  oneOf:
    - type: 'null'
    - $ref: "#/definitions/Address"
    - $ref: "#/definitions/BitcoinUrl"

Correct. I cannot speak to the issue with the online validator you're using. This repo is concerned only with the specification, not with any specific commercial or open source implementations.

Understood. My point was to advocate for a couple of good examples in the spec, which would immediately clarify the confusion.

[tedepstein]

@sebastien-rosset ,

Yes, in this example, Address would not be nullable. For example, suppose there is a shipping address, installedAt address, billTo address, etc. Some of the "address" properties can have a null value, others are mandatory. It may be a contrived example, but there are other cases when this could occur to.

This pattern works:

Order:
  type: object
  properties:
  shipTo:
    $ref: "#/definitions/Address"
  billTo:
    $ref: "#/definitions/OptionalAddress"

OptionalAddress:
  type: object
  nullable: true
  properties:
    addressLine1: 
      type: string
    addressLine 2:
      type: string
    # other properties

Address:
  allOf:
   - $ref: "#/definitions/OptionalAddress"
  type: object
  #Setting nullable to false is really a no-op, but makes the intent more explicit.
  nullable: false 

I won't say that this is intuitive, or convenient to make an optional variant of any type that might be optional, but if it's important, this pattern does the job. Also, this pattern doesn't attempt to address your extensible type hierarchy requirement, but there are other ways to do that.

And of course, all properties are optional by default, not required. A pattern like this is only required if you want to distinguish between an absent billTo property vs. an explicitly null billTo.

(Editorial comment: If you are allowing both -- omitted property AND null-valued property -- you should specify clearly in your API docs how these two cases are different, i.e. how your service will behave differently in these two cases. Maybe that goes without saying, but failure to clearly define different flavors of empty or missing value is a surprisingly common pitfall that makes APIs less usable, and API clients more error-prone.)

[sebastien-rosset]

Setting nullable to false is really a no-op, but makes the intent more explicit.

nullable: false

IMO it "nullable: false" should not be a complete no-op, because from the tooling perspective, there should be pretty clear expectations about how to implement the validation, in such a way that it is consistent across tools for interoperability. It's hard to imagine that all tooling engineers across all products would literally ignore "nullable: false". If it's there in the spec, it's likely people will try to do something about it, unless the spec is very explicit that implementation MUST ignore this statement.

Building on your example, some OAS authors may consider the opposite pattern, where OptionalAddress is defined as allOf Address. In some cases, it might be more natural to think that way, because you start from an address, define all the properties of the address, then create a new "OptionalAddress" that builds on "Address" and attaches more semantic to it. Especially when the OAS spec evolves over time, maybe initially no address property was nullable, and then one year later there is a new use case to make an address nullable. In that case, OAS authors may try to minimize the impact on the API and SDK users.

But just to be clear, my point is not to argue whether it's better to have allOf Address or allOf OptionalAddress. It's that there are multiple ways to specify nullable properties, I see it's a very common pattern across multiple APIs, hence examples and clarifications in the spec would really help.

1. Use type array for primitive types, e.g. type: [ 'null', 'string']. This is only valid in 3.1.0. It wasn't supposed to be supported in 3.0.x, but some companies did that anyway.
2. Use oneOf for primitive types and complex types, e.g. oneOf: [ 'null', $ref: "#/definitions/Address"]
3. Use siblings attributes along with $ref. Also not supported in the spec.
4. Define two types Optional{XYZ} and {XYZ}. {XYZ} "extends" from Optional{XYZ} using allOf
5. Define two types Optional{XYZ} and {XYZ}. Optional{XYZ} "extends" from{XYZ} using allOf
6. Probably other ways as well.

Also, now 3.1.0 explicitly marks "nullable" as deprecated, so my interpretation is it's a very strong hint to stop using it for any purpose, therefore I don't think new examples should be provided with "nullable" (unless hypothetically the next iteration of the spec undos the deprecation status).

At some point I took a grammarian hat to read the OAS spec and figure out if siblings attributes are allowed with $ref; And how nullable properties are supposed to be written.

• I read the ABNF notations in the OAS spec along with a link to RFC 5234. However, the notations is not compliant with the "alternative" syntax. OAS uses "|" instead of "/", so at that point I thought maybe I should be careful not to take every OAS word literally. I know it's somewhat unrelated, but it would be good to be very precise to avoid any confusion.
• In the "Schema" section, there are definitions, with some exceptions, explaining how OAS is similar and different from the JSON schema.
• Then crucially, there is this sentence: "Other than the JSON Schema subset fields, the following fields MAY be used for further schema documentation:". On one hand this sentence is within the schema section, which led me to conclude that this does not allow siblings to $ref, but on the other hand I saw conflicting examples elsewhere on the Internet. For example https://swagger.io/docs/specification/using-ref/.
• It does not help that multiple tools either ignore or actually support siblings.

[tedepstein]

@sebastien-rosset , I can't reply in detail now. But there's a misunderstanding: the reason why nullable: false is a no-op is because, if the top-level Address schema specifies type: object and does not, in that very same scope, specify nullable: true, then Address is non-nullable.

This is true even though Address "inherits" from OptionalAddress. In JSON Schema, allOf doesn't work like inheritance. It means that the value must conform independently to each of the listed subschemas, and it must also conform to any other conditions specified directly in the schema.

OptionalAddress specifies type: object and nullable: true, which is equivalent to JSON Schema's type: [object, null]. If address specifies type: object and does not directly specify nullable: true, then the type must be an object, not a null. The nullable keyword cannot be inherited and applied to subtypes.

By specifying type: object and omitting nullable: true, the Address object gets the default nullable: false value, which means "no change to the specified type, if any."

Remember, subtypes can add restrictions, but they cannot relax restrictions that are already in place. So Address is restricting the value space that it "inherits" from OptionalAddress. Instead of object or null, Address only allows object.

That's why nullable: false is always a no-op. It never has any effect, because nullable is false by default, and cannot be inherited.

[sebastien-rosset]

@sebastien-rosset , I can't reply in detail now. But there's a misunderstanding: the reason why nullable: false is a no-op is because, if the top-level Address schema specifies type: object and does not, in that very same scope, specify nullable: true, then Address is non-nullable.

My suggestion was to provide examples in the spec to reduce lengthy debates. Sorry if I wasn't clear, but I wasn't primarily looking for a discussion about what is valid and what is not valid (although that's great to have to), but rather pointing out that the spec is not obvious, hence again my suggestion to add examples. If there is confusion, tooling will be inconsistent, and inconsistencies will lead to interoperability issues, leading to technology adoption issues.

[tedepstein]

Examples are always helpful. It is up to the TSC to decide if we will add examples of nullable to the OpenAPI specification.

[teohhanhui]

Order:
  type: object
  properties:
  shipTo:
    $ref: "#/definitions/Address"
  billTo:
    type: object
    nullable: true
    allOf:
      $ref: "#/definitions/Address"

Address:
  type: object
  properties:
    addressLine1: 
      type: string
    addressLine 2:
      type: string
    # other properties

~Does this achieve the same?~ Never mind, it doesn't:

In JSON Schema, allOf doesn't work like inheritance. It means that the value must conform independently to each of the listed subschemas, and it must also conform to any other conditions specified directly in the schema.

[ddombrowsky]

Between this one and https://github.com/RicoSuter/NSwag/issues/2071, really having trouble finding the final decision on this.

Is the "correct" syntax A:

          "timeZone": {
            "oneOf": [
              { "type": "null" },
              { "$ref": "#/components/schemas/TimeZoneModel" }
            ]
          },

or B:

          "timeZone": {
            "nullable": true,
            "oneOf": [
              { "$ref": "#/components/schemas/TimeZoneModel" }
            ]
          },

?

[philsturgeon]

A is not valid OpenAPI v3.0, but would be valid OpenAPI v3.1 as it has aligned with JSON Schema 2019-09.

I've never seen B, feels funny. Don't have all the answers for this. Most folks do the allOf thing but that is apparently incorrect and always has been, despite me doing it for bloomin ages in various tools which seemed to think it was just fine.

[handrews]

@philsturgeon

Most folks do the allOf thing but that is apparently incorrect and always has been, despite me doing it for bloomin ages in various tools which seemed to think it was just fine.

Wait what's incorrect?

[handrews]

Oh, I guess you mean the interaction of $ref and nullable regardless of allOf vs oneOf etc.

Yeah, it's "incorrect" but the error is very subtle and many tools just kinda glossed over it. Regardless, it's fixed in 3.1 by getting rid of nullable 🎉

[Poorva17]

Hello all, what's the convention to mark reference object as nullable for openApi 3.0.1?

StudentObject:
      type: object
      nullable: true

Did not work for me. @handrews what's the right conventions?

[philsturgeon]

@Poorva17 I think this thread establishes that its awkward unspecified behavior, and whether something works or not depends more on the tooling you're using than whether its officially right or wrong.

Some tools will accept this:

StudentObject:
  allOf:
      - type: object
      - nullable: true

The best solution is to upgrade to OpenAPI v3.1, which many tools now support, and leave this ambiguity to be an academic question lost in time.

[richardbmitchell]

So, can the 3.0.1 spec handle recursive schema's?

{ "openapi": "3.0.1", "components": { "schemas": { "person": { "type": "object", "properties": { "name": { "type": "string" }, "children": { "type": "array", "nullable": true, "items": { "$ref": "#/components/schemas/person" } } }, "example": { "name": "a", "children": [ { "name": "b" }, { "name": "c", "children": [ { "name": "d" } ] } ] } } } } }

[maRci002]

This was fixed in #1977 so we can close this.

That PR was merged into the OAI:v3.1.0-dev branch 4 years ago. What's the current status?

[handrews]

@maRci002 OAS 3.1.0 was released in 2021. That's the status unless I'm missing something about your question.

[maRci002]

@handrews, when I search inside 3.1.0.md, it doesn't mention anything about nullable. Am I missing something?

[handrews]

@maRci002 we solved the problem by getting rid of the specific-to-OAS nullable keyword and just using all of JSON Schema draft 2020-12. So you can replace {"type": "integer", "nullable": "true"} with {"type": ["integer", "null"]}, which all JSON Schema implementations understand.

The problem had two parts:

1. Some people assumed that nullable did more than was intended, and this assumed behavior was really not compatible with how JSON Schema works - this was clarified in 3.0.3, which noted that nullable only modified type in the same Schema Object (the same way that JSON Schema draft-04's boolean exclusiveMinimum and exclusiveMaximum modifiers only modified minimum and maximum in the same Schema Object)
2. JSON Schema solves null values differently, with type arrays, and we wanted to have full JSON Schema compatibility in 3.1.0 because the incompatibilities were a huge pain point in 3.0. (Also, boolean modifiers are just confusing - in JSON Schema 2020-12, exclusiveMinimum and exclusiveMaximum are numeric keywords independent from minimum and maximum)

[maRci002]

@handrews, thank you very much for the clarification. However, I have a question about how this would work with reference objects. For example:

This configuration:

{
  "type": "object",
  "properties": {
    "snippet": {
      "type": "object",
      "properties": {
        "username": {
          "type": "string"
        }
      },
      "required": [
        "username"
      ],
      "additionalProperties": false
    },
    "snippet2": {
      "allOf": [
        {
          "$ref": "#/properties/snippet"
        }
      ],
      "nullable": true
    }
  },
  "required": [
    "snippet",
    "snippet2"
  ],
  "additionalProperties": false
}

Becomes this?

{
    "type": "object",
    "properties": {
      "snippet": {
        "type": "object",
        "properties": {
          "username": {
            "type": "string"
          }
        },
        "required": [
          "username"
        ],
        "additionalProperties": false
      },
      "snippet2": {
        "type": ["object", "null"],
        "$ref": "#/properties/snippet"
      }
    },
    "required": [
      "snippet",
      "snippet2"
    ],
    "additionalProperties": false
}

[handrews]

@maRci002 the problem you are encountering is that JSON Schema is a constraint system, not a data definition system. When you want to re-use a schema, you need to start with the most permissive version of the schema, and then add further constraints. So you can start with "type": ["object", "null"]), and then after $ref-ing that, constraint it further to "type": "object" (or "not": {"type": "null"}, but you can't start with a constraint of "type": "object" and then loosen it by allowing null. This is the opposite of how most people think about it, but it's fundamental to JSON Schema's design.

[handrews]

Commenting to trigger email notification: A better example than what I initially wrote above would be adding "not": {"type": "null"} alongside of a $ref instead of something like "type": "object", because that way you can remove a null without needing to know the other type(s) involved.

[maRci002]

When you want to re-use a schema, you need to start with the most permissive version of the schema, and then add further constraints.

Thank you again; this makes sense. Is it a good practice to register my most permissive version of the schema under definitions or components/schemas? Then, I could just reference it and add further constraints as needed.

[hkosova]

@maRci002 to make a $ref nullable in OpenAPI 3.1, you can use:

  "snippet2": {
    "anyOf": [
      { "$ref": "#/properties/snippet" },
      { "type": "null" }
    ]
  }

[maRci002]

@maRci002 to make a $ref nullable in OpenAPI 3.1, you can use:

  "snippet2": {
    "anyOf": [
      { "$ref": "#/properties/snippet" },
      { "type": "null" }
    ]
  }

Thank you, but in my case, Fastify didn't like it and suggested that I should use anyOf only as a last resort.