search

hyperjump-io / json-schema

JSON Schema Validation, Annotation, and Bundling. Supports Draft 04, 06, 07, 2019-09, 2020-12, OpenAPI 3.0, and OpenAPI 3.1
https://json-schema.hyperjump.io/
MIT License
216 stars 22 forks source link

Error: Encountered unknown dialect https://schemas.amazon.com/selling-partners/definitions/product-types/meta-schema/v1 #45

Closed akycop closed 7 months ago

akycop commented 8 months ago

Info

Details

export async function validator(payload:any) { defineVocabulary(schemaId,{ vocabularyToken: "$vocabulary" }) const customKeywords = { maxUniqueItems : maxUniqueItemsKeyword as any, minUtf8ByteLength : minUtf8ByteLengthKeyword as any, maxUtf8ByteLength : maxUtf8ByteLengthKeyword as any }; defineVocabulary(customVocabularyId, customKeywords); const metaSchemaJSON = JSON.parse(fs.readFileSync(metaSchemaPath,"utf8")); // loadDialect(schemaId, {}) addSchema(metaSchemaJSON, schemaId); // Error: Encountered unknown dialect 'https://schemas.amazon.com/selling-partners/definitions/product-types/meta-schema/v1' const luggageSchemaJSON = JSON.parse(fs.readFileSync(luggageSchemaPath, "utf8")); addSchema(luggageSchemaJSON, luggageSchemaId, schemaId); const result = await validate(luggageSchemaId, payload, BASIC) // Error: Encountered unknown keyword '$id' at https://schemas.amazon.com/selling-partners/definitions/product-types/meta-schema/v1# return result }

- metaSchemaPath（amazon-product-type-definition-meta-schema-v1.json）
```amazon-product-type-definition-meta-schema-v1.json
{
  "$schema": "https://schemas.amazon.com/selling-partners/definitions/product-types/meta-schema/v1",
  "$id": "https://schemas.amazon.com/selling-partners/definitions/product-types/meta-schema/v1",
  // etc...
}

Question

I encountered the following error: Error: Encountered unknown dialect 'https://schemas.amazon.com/selling-partners/definitions/product-types/meta-schema/v1'

I thought that this dialect was not set correctly, so I added the following code: loadDialect(schemaId, {}) However, now I encounter this error: Encountered unknown keyword '$id' at https://schemas.amazon.com/selling-partners/definitions/product-types/meta-schema/v1#

I have been stuck on this for quite some time. Could you provide any advice?

jdesrosiers commented 8 months ago

It looks like you're using a guide that was based on v0 of this library, but you're using v1 and the API is a little different. That example has some errors even for v0. See, https://github.com/hyperjump-io/json-schema#meta-schemas-keywords-vocabularies-and-dialects-experimental for a current example.

If they had used "$schema": "https://json-schema.org/draft/2019-09/schema" in the meta-schema you wouldn't have to load the dialect manually. This was probably just a poor choice on their part, but it's possible they did it on purpose because they wanted to use their custom keywords in their meta-schema. Anyway, the meta-schema should have a $vocabulary keyword that looks like this,

  "$vocabulary": {
    "https://json-schema.org/draft/2019-09/vocab/core": true,
    "https://json-schema.org/draft/2019-09/vocab/applicator": true,
    "https://json-schema.org/draft/2019-09/vocab/validation": true,
    "https://json-schema.org/draft/2019-09/vocab/meta-data": true,
    "https://json-schema.org/draft/2019-09/vocab/format": false,
    "https://json-schema.org/draft/2019-09/vocab/content": true,
    "https://schemas.amazon.com/selling-partners/definitions/product-types/vocabulary/v1": false
  },

The loadDialect function takes the value of that keyword.

loadDialect("https://schemas.amazon.com/selling-partners/definitions/product-types/meta-schema/v1", {
  "https://json-schema.org/draft/2019-09/vocab/core": true,
  "https://json-schema.org/draft/2019-09/vocab/applicator": true,
  "https://json-schema.org/draft/2019-09/vocab/validation": true,
  "https://json-schema.org/draft/2019-09/vocab/meta-data": true,
  "https://json-schema.org/draft/2019-09/vocab/format": false,
  "https://json-schema.org/draft/2019-09/vocab/content": true,
  "https://schemas.amazon.com/selling-partners/definitions/product-types/vocabulary/v1": false
}, true);

That should get you past your current hurdle. Note that the API for creating keywords and vocabularies is slightly different from v0. The documentation I linked to above should point you in the right direction, but feel free to ask more questions if you get stuck again.

akycop commented 8 months ago

Thank you for your reply. I have edited the code based on the advice you gave me, but I encounter the following error in the validate part: InvalidSchemaError: Invalid Schema

I have a question regarding this error. When validating, I specify luggageSchemaId as the first argument. Since this is a SchemaId, it cannot actually be accessed. According to your documentation, it is possible to specify a local file, so I did that, but I still get the same error. Could you give me any advice on this?

Also, if there is anything odd about my code, please point it out.

The entire code:

import fs from "fs";
import { addSchema, validate} from "@hyperjump/json-schema/draft-2019-09";
import { defineVocabulary, BASIC, loadDialect, addKeyword } from "@hyperjump/json-schema/experimental";

import * as maxUniqueItemsKeyword from "./Schemes/keywords/MaxUniqueItemsKeyword.js";
import * as maxUtf8ByteLengthKeyword from "./Schemes/keywords/MaxUtf8ByteLengthKeyword.js";
import * as minUtf8ByteLengthKeyword from "./Schemes/keywords/MinUtf8ByteLengthKeyword.js";

const schemaId = "https://schemas.amazon.com/selling-partners/definitions/product-types/meta-schema/v1";
const luggageSchemaId = "https://schemas.amazon.com/selling-partners/definitions/product-types/schema/v1/SHIRT";
const metaSchemaPath = "./src/functions/SP-API/Schemes/MetaSchemes/amazon-product-type-definition-meta-schema-v1.json";
const luggageSchemaPath = "./src/functions/SP-API/Schemes/productSchemes/SHIRT.json";
const customVocabularyId = "https://schemas.amazon.com/selling-partners/definitions/product-types/vocabulary/v1";

export async function validator(payload:any, url:string) {
  loadDialect("https://schemas.amazon.com/selling-partners/definitions/product-types/meta-schema/v1", {
    "https://json-schema.org/draft/2019-09/vocab/core": true,
    "https://json-schema.org/draft/2019-09/vocab/applicator": true,
    "https://json-schema.org/draft/2019-09/vocab/validation": true,
    "https://json-schema.org/draft/2019-09/vocab/meta-data": true,
    "https://json-schema.org/draft/2019-09/vocab/format": false,
    "https://json-schema.org/draft/2019-09/vocab/content": true,
    "https://schemas.amazon.com/selling-partners/definitions/product-types/vocabulary/v1": false
  }, true);

  addKeyword({
    id: customVocabularyId,
    compile: maxUniqueItemsKeyword.compile,
    interpret: maxUniqueItemsKeyword.interpret
  })

  addKeyword({
    id: customVocabularyId,
    compile: minUtf8ByteLengthKeyword.compile,
    interpret: minUtf8ByteLengthKeyword.interpret
  })

  addKeyword({
    id: customVocabularyId,
    compile: maxUtf8ByteLengthKeyword.compile,
    interpret: maxUtf8ByteLengthKeyword.interpret
  })

  defineVocabulary(schemaId,{
    vocabularyToken: "$vocabulary"
  })

  defineVocabulary(customVocabularyId,{
    "$vocabulary": customVocabularyId
  })

  const metaSchemaJSON = JSON.parse(fs.readFileSync(metaSchemaPath,"utf8"));
  addSchema(metaSchemaJSON, schemaId);

  const luggageSchemaJSON = JSON.parse(fs.readFileSync(luggageSchemaPath, "utf8"));
  addSchema(luggageSchemaJSON, luggageSchemaId, schemaId);

  // const result = await validate(luggageSchemaId, payload, BASIC) // InvalidSchemaError: Invalid Schema
  const result = await validate("./src/functions/SP-API/Schemes/productSchemes/SHIRT.schema.json", payload, BASIC);
  return result
}

Reference guide

I have rewritten the following code to version 1. Reference:Payload Validation

const luggageSchemaId = "https://schemas.amazon.com/selling-partners/definitions/product-types/schema/v1/LUGGAGE";
const JsonSchema = require("@hyperjump/json-schema");
var luggageSchema = await JsonSchema.get(luggageSchemaId);
const result = await JsonSchema.validate(luggageSchema, payload, JsonSchema.BASIC);
jdesrosiers commented 8 months ago

InvalidSchemaError means the schema doesn't successfully validate against the meta-schema. You can find out more by doing this, 

setMetaSchemaOutputFormat(BASIC); // The default output format if FLAG, which isn't very helpful in this case.
try {
  return validate("./src/functions/SP-API/Schemes/productSchemes/SHIRT.schema.json", payload, BASIC);
} catch (error: unknown) {
  if (error instanceof InvalidSchemaError) {
    console.log(error.output); // InvalidSchemaError includes property "output" that can tell you why validation failed.
  }
}

Meta-schema validation output can be difficult to interpret, but it will contain reason the schema failed validation.

Looking at your code, I see you're not adding keywords and defining the vocabulary correctly. Have another look at the example I linked to. Keywords need to have their own unique id. It doesn't matter what it is as long as it's unique. You can make something up. You're using the vocabulary id for all of them. That's not going to work.

Then use defineVocabulary to map keyword names to those ids. Something like this, 

defineVocabulary(customVocabularyId, {
  maxUniqueItems: "https://schemas.amazon.com/selling-partners/keywords/maxUniqueItems/v1", // <= This URI is made up, but needs to match the `id` you use in the keyword handler.
  minUtf8ByteLength: "https://schemas.amazon.com/selling-partners/keywords/minUtf8ByteLength/v1",
  maxUtf8ByteLength: "https://schemas.amazon.com/selling-partners/keywords/maxUtf8ByteLength/v1"
});
akycop commented 8 months ago

Thank you very much. Thanks to your help, the validator function worked without any problems. However, I'm finding it very difficult to pinpoint the specific issues in the payload based on the validation messages. For instance, the messages are output like this:

[
  {
    keyword: "https://json-schema.org/keyword/allOf",
    absoluteKeywordLocation: "https://schemas.amazon.com/selling-partners/definitions/product-types/schema/v1/SHIRT#/allOf",
    instanceLocation: "#",
    valid: false,
  },
  {
    keyword: "https://json-schema.org/evaluation/validate",
    absoluteKeywordLocation: "https://schemas.amazon.com/selling-partners/definitions/product-types/schema/v1/SHIRT#/allOf/0",
    instanceLocation: "#",
    valid: false,
  },
  {
    keyword: "https://json-schema.org/keyword/then",
    absoluteKeywordLocation: "https://schemas.amazon.com/selling-partners/definitions/product-types/schema/v1/SHIRT#/allOf/0/then",
    instanceLocation: "#",
    valid: false,
  },
  {
    keyword: "https://json-schema.org/evaluation/validate",
    absoluteKeywordLocation: "https://schemas.amazon.com/selling-partners/definitions/product-types/schema/v1/SHIRT#/allOf/0/then",
    instanceLocation: "#",
    valid: false,
  },
  {
    keyword: "https://json-schema.org/keyword/required",
    absoluteKeywordLocation: "https://schemas.amazon.com/selling-partners/definitions/product-types/schema/v1/SHIRT#/allOf/0/then/required",
    instanceLocation: "#",
    valid: false,
  },
]

Based on the information from this absoluteKeywordLocation, when I check the values of allOf in the schema, there are 83 items, and it's incredibly challenging to go through each one of them.

Is it possible to produce more specific validation messages? For example, in the case of this error, I would like to know which items are missing.

I would appreciate your assistance in this matter. SHIRT.json

jdesrosiers commented 8 months ago

Sorry, error reporting is something this library doesn't do well. At the moment, it's just designed to give you the machine readable information necessary to locate the error. If you need more human readable results, you'd have to process the output to construct it.

Here's an example to get you started. It handles required and you can expand it to handle additional keywords as needed.

import * as Schema from "@hyperjump/json-schema/schema/experimental";
import * as Instance from "@hyperjump/json-schema/instance/experimental";

const getErrorMessage = async (outputUnit, instance) => {
  if (outputUnit.keyword === "https://json-schema.org/keyword/required") {
    const schemaDocument = await Schema.get(outputUnit.absoluteKeywordLocation);
    const required = new Set(Schema.value(schemaDocument));
    const object = Instance.get(outputUnit.instanceLocation, instance);
    for (const propertyName of Instance.keys(object)) {
      required.delete(propertyName);
    }

    return `"${outputUnit.instanceLocation}" is missing required property(s): ${[...required]}. Schema location: ${outputUnit.absoluteKeywordLocation}`;
  } else {
    // Default message
    return `"${outputUnit.instanceLocation}" fails schema constraint ${outputUnit.absoluteKeywordLocation}`;
  }
};

// These are probably not very useful for human readable messaging, so we'll skip them.
const skip = new Set([
  "https://json-schema.org/evaluation/validate",
  "https://json-schema.org/keyword/ref",
  "https://json-schema.org/keyword/properties",
  "https://json-schema.org/keyword/patternProperties",
  "https://json-schema.org/keyword/items",
  "https://json-schema.org/keyword/prefixItems",
  "https://json-schema.org/keyword/if",
  "https://json-schema.org/keyword/then",
  "https://json-schema.org/keyword/else"
]);
export const getHumanErrors = (output, value) => {
  if (output.valid) {
    return "Valid";
  }

  const instance = Instance.cons(value);
  const errorMessages = output.errors
    .filter((outputUnit) => !skip.has(outputUnit.keyword))
    .map((outputUnit) => getErrorMessage(outputUnit, instance));

  return Promise.all(errorMessages);
};

Usage:

const schemaId = "https://example.com/schema";
const dialect = "https://json-schema.org/draft/2020-12/schema";
addSchema({ required: ["foo", "bar"] }, schemaId, dialect);

const instance = { foo: 42 };
const output = await validate(schemaId, instance, BASIC);
console.log(await getHumanErrors(output, instance));
// Expected Output:
// [
//   '"#/foo" is missing required property(s): bar. Schema location: https://example.com/schema#/$defs/r/required'
// ]