json-schema-org / JSON-Schema-Test-Suite

A language agnostic test suite for the JSON Schema specifications
MIT License
603 stars 205 forks source link

Description and test case contradiction: `unevaluatedProperties` #723

Open AgniveshChaubey opened 4 months ago

AgniveshChaubey commented 4 months ago

Reference:

https://github.com/json-schema-org/JSON-Schema-Test-Suite/blob/bf0360f4b7c51b8f968aabe7f3f49e12b120fc85/tests/draft2020-12/unevaluatedProperties.json#L529

Schema:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "if": {
      "properties": {
          "foo": { "const": "then" }
      },
      "required": ["foo"]
  },
  "else": {
      "properties": {
          "baz": { "type": "string" }
      },
      "required": ["baz"]
  },
  "unevaluatedProperties": false
}

Instance:

"tests": [
    {
        "description": "when if is true and has no unevaluated properties",
        "data": {
            "foo": "then",
            "bar": "bar"
        },
        "valid": false
    },
    ...
]

The description of the above test states that it has no unevaluated properties, but isn't the bar property here unevaluated?

jdesrosiers commented 4 months ago

I agree that it's wrong, but I think the test is wrong, not the description. The test should be updated to match the description.

gregsdennis commented 4 months ago

The test case is right, technically (but there may also be an error in expressing the intent).

bar is never defined, so in that regard, the test is correct.

However, the test might have meant baz instead of bar. In that case, /else is skipped when /if is true. Because baz is defined only in the else subschema (which isn't evaluated), it remains unevaluated.

jdesrosiers commented 4 months ago

You're right, but that's not what I meant. Look at all the descriptions for all the tests. It's a matrix of the possibilities of two variables: (1) if result (2) unevaluated properties result.

The test should be changed to actually test what the description says otherwise it's just a another test for "when if is true and has unevaluated properties", which is already covered.