python-jsonschema / jsonschema

An implementation of the JSON Schema specification for Python
https://python-jsonschema.readthedocs.io
MIT License
4.58k stars 578 forks source link

Inaccurate error message while using "unevaluatedProperties": False #1169

Open MLL1228 opened 11 months ago

MLL1228 commented 11 months ago

I am using jsonschema to validate JSON data against a schema using "unevaluatedProperties": False. When my data contains an unexpected property, the error message lists all the evaluated properties as unexpected, instead of the actual unevaluated property.

Here is a schema as an example:

{
  "allOf": [
    {
      "properties": {
        "street_address": { "type": "string" },
        "city":           { "type": "string" },
        "state":          { 
          "type": "object",
          "properties": {
            "name": { "type": "string" },
          },
          "additionalProperties": False
        }
      }
    },
    {
      "properties": {
        "type":  { "type": "string" },
      }
    }
  ],
  "type": "object",
  "unevaluatedProperties": False
}

And here is a json data:

{
  "street_address": "21 2nd Street",
  "city": "New York",
  "state": {"name": "NY", "cc": "Unexpected field"},
  "type": "home"
}

In this case, 'cc' under 'state' is unexpected. If I remove 'cc', the validation is successful. However, the error message reported by jsonschema is: jsonschema.exceptions.ValidationError: Unevaluated properties are not allowed ('city', 'state', 'street_address' were unexpected).

My questions are:

Why is the error message not mentioning 'cc', the actual cause of the error? Are there any better ways to provide compatibility with the unevaluatedProperties keyword that could help us identify the specific unevaluated properties? Please let me know if you need additional details.

Information about my current environment:

Python version: 3.7.16 jsonschema version: 4.16.0 Thanks in advance for your help.

Julian commented 11 months ago

I have to take a closer look but just off the bat I'd recommend comparing with #1132 or #1026 it may be that's relevant.

(Again not having played with your example) I suspect what's happening has to do with the spec's behavior saying that when a subschema fails, it does not produce annotations -- specifically what that means is that if your subschema is not considering your instance fully valid (the schema in your allOf), then indeed those properties were not evaluated, so they should be considered unevaluated -- there is no "partially" evaluated.

The fact that that error isn't the one you see though is #1132 which is why I point you there.

But I can have a closer look at some point if the above doesn't help / is incorrect on closer inspection.

ClementDelannoySlateDigital commented 3 months ago

Hey,

I think I found the same type of issue here.

The real error was '44100' is not of type 'number' (this property is defined in an 'allOf' section. But jsonschema raises an 'Unevaluated properties' with all properties defined on the 'allOf' section.

This is an important issue for me because it leads to false leads that may complexify the debugging a lot.

That would be terrible if it was a design error in the standard...

Have you been able to investigate the issue? I can help looking for something wrong in the logic, please tell me what you've found on your side!

Thanks