Validate theme.json files against the JSON Schema

[ajlende]

Adds a command to validate the theme.json schemas.

$ npm run schema:validate ./**/theme.json

> community-themes@1.0.0 schema:validate
> node ./theme-utils.mjs validate-schema ./archivist/theme.json ./atlas/theme.json ./blue-note/theme.json ./poetry/theme.json ./purr/theme.json ./stacks/theme.json ./term/theme.json ./tt1-blocks/theme.json

■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 8/8
[
  {
    "file": "./atlas/theme.json",
    "schema": "https://schemas.wp.org/wp/6.3/theme.json",
    "error": [
      {
        "instancePath": "/settings/color/duotone/0/slug",
        "schemaPath": "#/definitions/settingsPropertiesColor/properties/color/properties/duotone/items/properties/slug/type",
        "keyword": "type",
        "params": {
          "type": "string"
        },
        "message": "must be string"
      }
    ]
  },
  {
    "file": "./blue-note/theme.json",
    "schema": "https://schemas.wp.org/wp/6.2/theme.json",
    "error": [
      {
        "instancePath": "/styles/elements",
        "schemaPath": "#/additionalProperties",
        "keyword": "additionalProperties",
        "params": {
          "additionalProperty": "textarea"
        },
        "message": "must NOT have additional properties"
      },
      {
        "instancePath": "/styles/elements",
        "schemaPath": "#/additionalProperties",
        "keyword": "additionalProperties",
        "params": {
          "additionalProperty": "input"
        },
        "message": "must NOT have additional properties"
      }
    ]
  },
  {
    "file": "./stacks/theme.json",
    "schema": "https://schemas.wp.org/trunk/theme.json",
    "error": [
      {
        "instancePath": "/version",
        "schemaPath": "#/properties/version/enum",
        "keyword": "enum",
        "params": {
          "allowedValues": [
            3
          ]
        },
        "message": "must be equal to one of the allowed values"
      },
      {
        "instancePath": "/styles/elements/h1/typography/lineHeight",
        "schemaPath": "#/properties/typography/properties/lineHeight/oneOf/0/type",
        "keyword": "type",
        "params": {
          "type": "string"
        },
        "message": "must be string"
      },
      {
        "instancePath": "/styles/elements/h1/typography/lineHeight",
        "schemaPath": "#/definitions/refComplete/type",
        "keyword": "type",
        "params": {
          "type": "object"
        },
        "message": "must be object"
      },
      {
        "instancePath": "/styles/elements/h1/typography/lineHeight",
        "schemaPath": "#/properties/typography/properties/lineHeight/oneOf",
        "keyword": "oneOf",
        "params": {
          "passingSchemas": null
        },
        "message": "must match exactly one schema in oneOf"
      },
      {
        "instancePath": "/styles/elements/h2/typography/lineHeight",
        "schemaPath": "#/properties/typography/properties/lineHeight/oneOf/0/type",
        "keyword": "type",
        "params": {
          "type": "string"
        },
        "message": "must be string"
      },
      {
        "instancePath": "/styles/elements/h2/typography/lineHeight",
        "schemaPath": "#/definitions/refComplete/type",
        "keyword": "type",
        "params": {
          "type": "object"
        },
        "message": "must be object"
      },
      {
        "instancePath": "/styles/elements/h2/typography/lineHeight",
        "schemaPath": "#/properties/typography/properties/lineHeight/oneOf",
        "keyword": "oneOf",
        "params": {
          "passingSchemas": null
        },
        "message": "must match exactly one schema in oneOf"
      }
    ]
  },
  {
    "file": "./tt1-blocks/theme.json",
    "error": [
      {
        "instancePath": "",
        "schemaPath": "#/required",
        "keyword": "required",
        "params": {
          "missingProperty": "$schema"
        },
        "message": "must have required property '$schema'"
      }
    ]
  }
]

The command is set up to be run with lint-staged which runs on the precommit hook thanks to husky. It can be run manually too.

$ npx lint-staged

[MaggieCabrera]

Thank you for this @ajlende ! It seems like the action is running on the pre commit hook but when I made a change to one of the theme.json files it doesn't fail as expected. On bluenote I changed this line:

        "layout": {
            "contentSize": "874px",
            "wideSize": "1300px"
        },

to this

        "layout": {
            "contentSize": 874,
            "wideSize": "1300px"
        },

the precommit validation seemed to pass.

Running the command manually gives me a log like so:

➜  community git:(add/schema-validation) ✗ npm run schema:validate ./**/theme.json

> community-themes@1.0.0 schema:validate
> node ./theme-utils.mjs validate-schema ./archivist/theme.json ./atlas/theme.json ./blue-note/theme.json ./poetry/theme.json ./purr/theme.json ./stacks/theme.json ./term/theme.json ./tt1-blocks/theme.json

■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 8/8
[
  {
    "file": "./atlas/theme.json",
    "error": [
      {
        "instancePath": "/styles/blocks/core~1navigation/spacing/blockGap",
        "schemaPath": "#/properties/spacing/properties/blockGap/oneOf/0/type",
        "keyword": "type",
        "params": {
          "type": "string"
        },
        "message": "must be string"
      },
      {
        "instancePath": "/styles/blocks/core~1navigation/spacing/blockGap",
        "schemaPath": "#/definitions/refComplete/type",
        "keyword": "type",
        "params": {
          "type": "object"
        },
        "message": "must be object"
      },
      {
        "instancePath": "/styles/blocks/core~1navigation/spacing/blockGap",
        "schemaPath": "#/properties/spacing/properties/blockGap/oneOf",
        "keyword": "oneOf",
        "params": {
          "passingSchemas": null
        },
        "message": "must match exactly one schema in oneOf"
      },
      {
        "instancePath": "/styles/blocks/core~1navigation/spacing/blockGap",
        "schemaPath": "#/properties/spacing/properties/blockGap/oneOf/0/type",
        "keyword": "type",
        "params": {
          "type": "string"
        },
        "message": "must be string"
      },
      {
        "instancePath": "/styles/blocks/core~1navigation/spacing/blockGap",
        "schemaPath": "#/definitions/refComplete/type",
        "keyword": "type",
        "params": {
          "type": "object"
        },
        "message": "must be object"
      },
      {
        "instancePath": "/styles/blocks/core~1navigation/spacing/blockGap",
        "schemaPath": "#/properties/spacing/properties/blockGap/oneOf",
        "keyword": "oneOf",
        "params": {
          "passingSchemas": null
        },
        "message": "must match exactly one schema in oneOf"
      }
    ]
  },
  {
    "file": "./blue-note/theme.json",
    "error": [
      {
        "instancePath": "/settings/layout/contentSize",
        "schemaPath": "#/definitions/settingsPropertiesLayout/properties/layout/properties/contentSize/type",
        "keyword": "type",
        "params": {
          "type": "string"
        },
        "message": "must be string"
      },
      {
        "instancePath": "/styles/elements",
        "schemaPath": "#/additionalProperties",
        "keyword": "additionalProperties",
        "params": {
          "additionalProperty": "textarea"
        },
        "message": "must NOT have additional properties"
      },
      {
        "instancePath": "/styles/elements",
        "schemaPath": "#/additionalProperties",
        "keyword": "additionalProperties",
        "params": {
          "additionalProperty": "input"
        },
        "message": "must NOT have additional properties"
      }
    ]
  },
  {
    "file": "./stacks/theme.json",
    "error": [
      {
        "instancePath": "/version",
        "schemaPath": "#/properties/version/enum",
        "keyword": "enum",
        "params": {
          "allowedValues": [
            3
          ]
        },
        "message": "must be equal to one of the allowed values"
      },
      {
        "instancePath": "/styles/elements/h1/typography/lineHeight",
        "schemaPath": "#/properties/typography/properties/lineHeight/oneOf/0/type",
        "keyword": "type",
        "params": {
          "type": "string"
        },
        "message": "must be string"
      },
      {
        "instancePath": "/styles/elements/h1/typography/lineHeight",
        "schemaPath": "#/definitions/refComplete/type",
        "keyword": "type",
        "params": {
          "type": "object"
        },
        "message": "must be object"
      },
      {
        "instancePath": "/styles/elements/h1/typography/lineHeight",
        "schemaPath": "#/properties/typography/properties/lineHeight/oneOf",
        "keyword": "oneOf",
        "params": {
          "passingSchemas": null
        },
        "message": "must match exactly one schema in oneOf"
      },
      {
        "instancePath": "/styles/elements/h2/typography/lineHeight",
        "schemaPath": "#/properties/typography/properties/lineHeight/oneOf/0/type",
        "keyword": "type",
        "params": {
          "type": "string"
        },
        "message": "must be string"
      },
      {
        "instancePath": "/styles/elements/h2/typography/lineHeight",
        "schemaPath": "#/definitions/refComplete/type",
        "keyword": "type",
        "params": {
          "type": "object"
        },
        "message": "must be object"
      },
      {
        "instancePath": "/styles/elements/h2/typography/lineHeight",
        "schemaPath": "#/properties/typography/properties/lineHeight/oneOf",
        "keyword": "oneOf",
        "params": {
          "passingSchemas": null
        },
        "message": "must match exactly one schema in oneOf"
      }
    ]
  },
  {
    "file": "./tt1-blocks/theme.json",
    "error": [
      {
        "instancePath": "",
        "schemaPath": "#/required",
        "keyword": "required",
        "params": {
          "missingProperty": "$schema"
        },
        "message": "must have required property '$schema'"
      }
    ]
  }
]

[MaggieCabrera]

Looking at the log, which maybe could be simplified for readability, but we can look into that on a separate PR, it looks like it's treating the schema version being 2 instead of 3 as an error

[ajlende]

it looks like it's treating the schema version being 2 instead of 3 as an error

./stacks/theme.json looks like it's using the trunk schema causing version 2 error. I'm going to include which schema is being tested against in the error output so that is more clear.

[MaggieCabrera]

./stacks/theme.json looks like it's using the trunk schema causing version 2 error. I'm going to include which schema is being tested against in the error output so that is more clear.

Ugh, you are right, I made a PR to update the ones that had the wrong version but apparently I missed stacks...

[ajlende]

Ugh, you are right, I made a PR to update the ones that had the wrong version but apparently I missed stacks...

Imagine if you had a nice script like this when you were updating them to confirm which ones were using the wrong schema :)

[ajlende]

It might also be helpful to set up a GitHub action to run the verification on the files touched in the PR. I can try to do that in a follow-up. I didn't want to do that here for two reasons.

1. We have other errors in the existing schemas which we might want to update alongside the update so folks don't have to fix things when changing something unrelated.
2. The additionalProperties errors in Blue Note seemed suspicious, like they might be an error with the schema.

[ajlende]

Looking at the log, which maybe could be simplified for readability, but we can look into that on a separate PR

I updated it to use console.dir which formats JSON closer to the AJV CLI and uses colors.

They format the errors like this:

$ npx ajv-cli -s ../gutenberg/schemas/json/theme.json -d ./blue-note/theme.json --allowMatchingProperties --allErrors
./blue-note/theme.json invalid
[
  {
    instancePath: '/styles/elements',
    schemaPath: '#/additionalProperties',
    keyword: 'additionalProperties',
    params: { additionalProperty: 'textarea' },
    message: 'must NOT have additional properties'
  },
  {
    instancePath: '/styles/elements',
    schemaPath: '#/additionalProperties',
    keyword: 'additionalProperties',
    params: { additionalProperty: 'input' },
    message: 'must NOT have additional properties'
  }
]

I didn't use that CLI here because it requires the schemas to be saved to disk. It was more complicated to download all the schemas to a temp directory and run the CLI rather than just use the AJV library directly.

[ajlende]

It seems like the action is running on the pre commit hook but when I made a change to one of the theme.json files it doesn't fail as expected.

Fixed in 8fb8156ffc15b77314b71a5646a0611f75e9f1f1.

It wasn't failing because I missed renaming the command in .lintstagedrc.json and the script doesn't exit with an error when you type an incorrect command.

[MaggieCabrera]

./stacks/theme.json looks like it's using the trunk schema causing version 2 error. I'm going to include which schema is being tested against in the error output so that is more clear.

Ugh, you are right, I made a PR to update the ones that had the wrong version but apparently I missed stacks...

[MaggieCabrera]

Now it works as intended when I run the command manually, but the pre commit hook says

→ No staged files match any configured task.

When I try to commit a faulty theme.json. I tried to debug why but your code seems fine and it should be working :S

[github-actions[bot]]

Preview changes

I've detected changes to the following themes in this PR: Purr.

You can preview these changes by following the links below:

• Preview changes for Purr

I will update this comment with the latest preview links as you push more changes to this PR.

⚠️ Note: The preview sites are created using WordPress Playground. You can add content, edit settings, and test the themes as you would on a real site, but please note that changes are not saved between sessions.