`set_root_schema()` throws on valid `anyOf`

[rreed]

TL;DR: set_root_schema throws with array index '' is not a number.

The title is indicative of what I think is happening, but perhaps it's something different, so I'll obviously give as much info as I have.

So I have the following schema, modified to remove some irrelevant information and redact some specificity:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    // some stuff
    "outerThing": {
      "type": "array",
      "minItems": 1,
      "uniqueItems": true,
      "items": {
        "type": "object",
        "properties": {
          // some stuff
          "innerThing": {
            "anyOf": [
              {
                "type": "string",
              },
              {
                "type": "object",
                "properties": {
                  "integerOne": {
                      "type": "integer",
                      "minimum": 0,
                      "maximum": 3
                  },
                  "arrayOne": {
                    "type": "array",
                    "minItems": 1,
                    "maxItems": 1,
                    "uniqueItems": true,
                    "items": {
                      "type": "object",
                      "properties": {
                        "objectOne": {
                            "type": "array",
                            "minItems": 2,
                            "maxItems": 2,
                            "uniqueItems": true,
                            "items": {
                              "type": "array",
                              "minItems": 2,
                              "maxItems": 2,
                              "uniqueItems": true,
                              "item": {
                                "type": "number",
                                "minimum": -180,
                                "maximum": 180
                              }
                            }
                         }
                       }
                    }
                  }
                }
              },
              {
                // redacted object
              }
            ]
          }
        },
        "required": [
          // ...
        ],
        "additionalProperties": false
      }
    },
    // some more stuff
  },
  "required": [
    // stuff
  ],
  "additionalProperties": false
}

This validates against, say, jsonschemavalidator.net as a valid schema (well, at least if I remove the comment from it and replace it with the real code :p). But when we use this same schema in the json-schema-validator library, it crashes within nlohmann::json:

1: terminate called after throwing an instance of 'nlohmann::detail::parse_error'
1:   what():  [json.exception.parse_error.109] parse error: array index '' is not a number

A coworker provided me this screenshot of the code from his debugging:

And the backtrace via gdb contains the following:

$1 = (const nlohmann::basic_json blahblahblah @0x7fffffff39b0 {reference_token = std::vector of length 14, capacity 14 = {"properties", "outerThing", "items", "properties", "innerThing", "", "1", "properties", "arrayOne", "items", "properties", "objectOne", "items", "item"}}

(for clarity, the last field is actually named "item"; "outerThing", "innerThing", "arrayOne", and "objectOne" have had their names changed :p)

Presumably the "" in this vector is being interpreted as an array index, but hopefully you know more about what's up than I do, because this passes validation on any other library I've tried. Wasn't sure if i should file this here or with nlohmann::json, but since set_root_schema is in this library, I started here.

Thanks a bunch in advance for any help you can provide. <3

[pboettch]

After removing comments and commas which invalidated your schema when being used with upstream nlohmann::json (3.9.1) I could load the schema and validate the following instance:

{
    "outerThing": [
        {"innerThing": "str"}
    ]
}

Could you provide a full example and tell me what version of nlohmann::json you are using?

[pboettch]

Even when allowing comments it parses the schema correctly.

[rreed]

Thanks for looking into this so quickly! I checked and we're on nlohmann::json 3.7.3 (which looks like it's from ~Nov 2019, so a little old but not too old). I'll try updating this soon.

As for the other question: the full schema is a thousand plus lines, which I figured was too much for this bug report, so I just extracted a redacted version of the part that crashes. Here's a self-contained, runs-as-is version of the same, which hopefully helps:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    "outerThing": {
      "type": "array",
      "minItems": 1,
      "uniqueItems": true,
      "items": {
        "type": "object",
        "properties": {
          "innerThing": {
            "anyOf": [
              {
                "type": "string",
              },
              {
                "type": "object",
                "properties": {
                  "integerOne": {
                      "type": "integer",
                      "minimum": 0,
                      "maximum": 3
                  },
                  "arrayOne": {
                    "type": "array",
                    "minItems": 1,
                    "maxItems": 1,
                    "uniqueItems": true,
                    "items": {
                      "type": "object",
                      "properties": {
                        "objectOne": {
                            "type": "array",
                            "minItems": 2,
                            "maxItems": 2,
                            "uniqueItems": true,
                            "items": {
                              "type": "array",
                              "minItems": 2,
                              "maxItems": 2,
                              "uniqueItems": true,
                              "item": {
                                "type": "number",
                                "minimum": -180,
                                "maximum": 180
                              }
                            }
                         }
                       }
                    }
                  }
                }
              },
            ]
          }
        },
        "required": [
          "innerThing"
        ],
        "additionalProperties": false
      }
    }
  },
  "additionalProperties": false
}

I validated this against jsonschemavalidator.net using the following:

{
  "outerThing": [
    {
      "innerThing": {
        "integerOne": 1,
        "arrayOne": [
          {
            "objectOne": [
              [123, -123], [111, -111]
            ]
          }
        ]
      }
    },
    {
      "innerThing": "string blah blah"
    }
  ]
}

Though our code crashes in set_root_schema, before even trying to validate an object, so perhaps this is irrelevant.

[pboettch]

It works with nlohmann::json 3.9.1 and the head of the master-branch of this library. IOW I don't have an exception using your schema.

Using 3.7.x of nlohmann::json means you're using an old version of this library as the current head of master does not support versions less than 3.8.0.

[rreed]

Okay, cool, we'll just upgrade!

[rreed]

Hello again! So we upgraded to 3.9.1, and this crash was fixed...but replaced by a different crash, which turned out to be because our schema had item in an array field instead of items.

Apparently jsonschemavalidator.net and others won't catch this because the schema validates even if you replace "item" with "asdfasdf" or "itemsssssssss" or anything else. :p Strange. In any case, this explains the crash, so I'll close this ticket, and thanks for all of your help. <3

[pboettch]

It should not crash.

[pboettch]

What version of the validator are you using? I just tried your example from above, it works without crashing.

[rreed]

2.2.1 with nlohmann 3.9.1. Here's a full example (redacted the names, but it should be fairly obvious that this deals with lat/lng :p):

{
  "foo": {
    "type": "array",
    "minItems": 1,
    "maxItems": 1,
    "uniqueItems": true,
    "items": {
      "type": "object",
      "properties": {
        "bar": {
          "type": "array",
          "minItems": 2,
          "maxItems": 2,
          "uniqueItems": true,
          "items": {
            "type": "array",
            "minItems": 2,
            "maxItems": 2,
            "uniqueItems": true,
            "item": {
              "type": "number",
              "minimum": -180,
              "maximum": 180
            }
          }
        }
      }
    }
  }
}

Our code consistently crashes with

1: terminate called after throwing an instance of 'std::invalid_argument'
1:   what():  stoull

until we changed item to items; from reading the code, this crash is in nlohmann::json, not json-schema-validator itself (afaict stoull is never called in this repo). Hopefully this adds a bit more context!

[pboettch]

There is no version 2.2.1, do you mean 2.1.0? Does it also crash if you load this schema with the ./json-schema-validate which is built by default in the build-dir? (for it doesn't crash).

[pboettch]

Could you generate a backtrace for this exception?