json-schema-org / JSON-Schema-Test-Suite

A language agnostic test suite for the JSON Schema specifications
MIT License
625 stars 209 forks source link

Add a test for 2019's interaction between additional/unevaluatedItems #639

Closed Julian closed 1 year ago

Julian commented 1 year ago

The interesting bit here actually only applies to 2019, as it's the only draft containing both unevaluatedItems as well as additionalItems -- the latter in its specified behavior containing:

if "items" is absent or its annotation result is the boolean true, "additionalItems" MUST be ignored.

(from https://datatracker.ietf.org/doc/html/draft-handrews-json-schema-02#section-9.3.1.2)

In newer drafts (i.e. 2020), when additionalItems "became" items, this condition was dropped (i.e. items without a neighboring prefixItems now is indeed not ignored).

Refs: https://github.com/orgs/json-schema-org/discussions/57

Closes: #292

Julian commented 1 year ago

This Slack thread (re-)raised the missing test added here.

As mentioned in the thread, around half of all implementations supported by Bowtie get this wrong:

⊙  2>/dev/null bowtie validate -D 2019 $(find ~/Development/bowtie/implementations/ -mindepth 1 -maxdepth 1 -type d | sed 's/.*\/implementations\//-i /') <(printf '{
  "$schema": "https://json-schema.org/draft/2019-09/schema",
  "additionalItems": {"type": "number"},
  "unevaluatedItems": {"type": "string"}
}
') <(printf '["foo", 1]') | rg implementation | jq '{implementation: .implementation, valid: .results[0].valid}'
{
  "implementation": "ghcr.io/bowtie-json-schema/ts-vscode-json-languageservice",
  "valid": false
}
{
  "implementation": "ghcr.io/bowtie-json-schema/python-jsonschema",
  "valid": true
}
{
  "implementation": "ghcr.io/bowtie-json-schema/go-jsonschema",
  "valid": false
}
{
  "implementation": "ghcr.io/bowtie-json-schema/python-jschon",
  "valid": false
}
{
  "implementation": "ghcr.io/bowtie-json-schema/dotnet-jsonschema-net",
  "valid": false
}
{
  "implementation": "ghcr.io/bowtie-json-schema/rust-jsonschema",
  "valid": true
}
{
  "implementation": "ghcr.io/bowtie-json-schema/js-ajv",
  "valid": null
}
{
  "implementation": "ghcr.io/bowtie-json-schema/js-hyperjump",
  "valid": true
}

Where the reasoning is likely easy to understand -- any implementation which implements unevaluatedItems by essentially doing a if additionalItems in instance: short_circuit() is forgetting that that needs to be if additionalItems in instance *and* items in instance: ... to comply with that ignore behavior.

Julian commented 1 year ago

Thanks!