GSoC: Define upgrade/downgrade language agnostic declarative transformation rules for all JSON Schema dialects

[jviotti]

Project title

Define upgrade/downgrade language agnostic declarative transformation rules for all JSON Schema dialects.

Brief Description

The Alterschema project defines a set of JSON-based formal transformation rules for upgrading schemas between Draft 4 and 2020-12, and all dialects in between. These rules are defined using JSON Schema and JSON-e and live within the Alterschema project.

We would like to revise these rules, extend them to support every dialect of JSON Schema (potentially including OpenAPI's old dialects too), and attempt to support some level of downgrading.

Instead of having these rules on the Alterschema repository, we want to have them on the JSON Schema organization for everybody to consume, including Alterschema itself.

Revising the rule format should consider currently unresolved edge cases in Alterschema like tweaking references after a subschema is moved.

Expected Outcomes

A new repository in the JSON Schema organization with upgrade/downgrade rules defined using JSON.

Skills Required

Understanding of various dialects of JSON Schema and their differences.

Mentors

@jviotti

Expected Difficulty

Medium

Expected Time Commitment

350 hours

[mwadams]

It can be anything.

[mwadams]

(The value of the $ref is applied to the current scope and the schema is resolved from that reference.)

[Era-cell]

Hi @Era-cell

whose schema resource isnt present in the document which is being altered, at this point the external schema document(which is external resource) also needs to be altered?

Great question! Yes on both cases:

• A JSON Schema is allowed to externally reference another JSON Schema that makes use of a different draft. i.e. you can have a JSON Schema 2020-12 that externally references a JSON Schema Draft 4. So in that case, it is not really required to i.e. upgrade the other schema and we can simply ignore it if we don't have access to it
• That said, while this cross-version referencing is supposed to work, I think many implementations out there don't properly support it, and the JSON Schema test suite doesn't cover it either. For these cases, what you can do is perform JSON Schema Bundling (https://json-schema.org/blog/posts/bundling-json-schema-compound-documents) before upgrading that schema. Bundling will bring in all externally referenced schema into a single schema with nested schema resources, and then we upgrade them all together

But in both cases, our upgrader shouldn't really mind. If its passed a schema with unresolved remote references, it will do what it can, and if its passed a bundled schema, it will transform the entire thing.

Okay, so if we have access to external resource and it is resolved.. we dont change the external schema, but we bundle it in the present document itself right? BECAUSE the user may use the external schema for other purposes too.. Right?

[jviotti]

Keep in mind the project would not be able to "modify" any schema in place. What it does is create a copy of the input schema with the given transformations. So:

• If the schema is bundled, you transform the entire thing, including the bundled resources
• If the schema is NOT bundled, you just transform the immediate schema only

[benjagm]

🚩 IMPORTANT INSTRUCTIONS REGARDING HOW AND WHERE TO SUBMIT YOU APPLICATION 🚩

Please join this discussion in JSON Schema slack to get the last details very important details on how to better submit your application to JSON Schema.

See communication here.

[Era-cell]

Hi, @jviotti where should the qualification task be submitted, and what is the deadline for it?

[jviotti]

@Era-cell I believe there is a GSoC portal that you should use. @benjagm Can you clarify?

[Era-cell]

@Era-cell I believe there is a GSoC portal that you should use. @benjagm Can you clarify?

@jviotti I guess that is for the proposal, should I embed qualification task inside proposal itself..? @benjagm

[benjagm]

@Era-cell yes please. Make sure you add the details of the qualification task to the proposal! Feel free to join the #gsoc channel in our Slack workspace to get immediately response to these type of questions

[MeastroZI]

Hi @jviotti,

First of all, I apologize for using the Alterschima UI to display my DSL transformation. It's only temporary!

Could you please review the transformation from 2019 to 2020 draft on this site? I've embedded the qualification tasks' DSL transformation code and have tried my best to cover all edge cases. However, if I've missed any, please let me know."

[jviotti]

@MeastroZI Not much I can comment on given a single example, but looking forward to the explanations, proposed rules, etc in the proposal!

[MeastroZI]

@jviotti, I submitted my proposal (Name: Pandit Vinit ) in Json schema. Could you please review it and provide any suggestions if possible ?

[jviotti]

I will, thanks a lot for the submission! ❤️

[MeastroZI]

@jviotti in 2019-09 draft i am not able to find the any difference between additionalItems and unevaluatedItems here written as "Similar to additionalItems, but can "see" into subschemas and across references" but as i tested this schema , additionalItems also doing all of this here is the example

{
  "$schema": "https://json-schema.org/draft/2019-09/schema",
  "$def": {
    "stringArray": {
      "type": "array",
      "items": {
        "type": "string"
      }
    },
    "numberArray": {
      "oneOf": [
        {
          "type": "array",
          "items": [
            {
              "type": "number"
            },
            {
              "$ref": "#/$def/stringArray"
            }
          ]
        },
        {
          "type": "boolean"
        }
      ]
    }
  },
  "type": "array",
  "items": [
    {
      "$ref": "#/$def/stringArray"
    }
  ],
  "additionalItems": {
    "$ref": "#/$def/numberArray"
  }
}

validate against : [[""] , [5 , [""]] ] and [[""] , true ]

so my question is what is the difference between additionalItems and unevaluatedItems in 2019-09 draft and is there any example schema which show the difference between additionalItems and unevaluatedItems ?

[jviotti]

@MeastroZI Take a look at the official test suite examples: https://github.com/json-schema-org/JSON-Schema-Test-Suite/blob/main/tests/draft2019-09/unevaluatedItems.json. additionalItems matches any array element not covered by an adjacent items. unevaluatedItems applies to array items that were not evaluated (as its name implies) by any other relevant keyword (whether adjacent or not).

[MeastroZI]

"@jviotti, I need direction to think on how to approach downgrading of JSON schema. Is it even possible to do this for all the dialects? With each new version, new keywords are introduced, and I'm unsure if it's feasible to replicate their behavior using the previous version.

Regarding upgrading, I've developed the DSL, and I believe it's capable of handling all upgrades. Please review the recent changes I made in the repository and please provide feedback if possible."

[jviotti]

@MeastroZI It is not always feasible, but I think you can go a long way with it, and we can think how to handle the problematic cases. I think if the resulting downgraded schema is a superset of the schema (i.e. it doesn't add more constraints), then it's probably acceptable.

[github-actions[bot]]

Hello! :wave:

This issue has been automatically marked as stale due to inactivity :sleeping:

It will be closed in 180 days if no further activity occurs. To keep it active, please add a comment with more details.

There can be many reasons why a specific issue has no activity. The most probable cause is a lack of time, not a lack of interest.

Let us figure out together how to push this issue forward. Connect with us through our slack channel : https://json-schema.org/slack

Thank you for your patience :heart: