Beginner issue: load schema from file and bundle into one

[the42]

Please help me out.

I have two schemas S1.json and defs.json which are only local and cannot be fetched via https.

They are properly declared with $id

S1 contains business entities, $defs and $ref to defs.json defs.json only contains $defs and $ref to #

I am under the impression, that this tool can help me to bundle into one schema.json, yet for me without javascript knowledge, I do not know where to start.

I guess I have to write a tiny program and execute with npx, correct? I start from the sample https://github.com/hyperjump-io/json-schema#bundling

• How can I load a local file?
• How is it supposed to work out that $id in defs.json is the one $ref in S1 are referring to? Do I need a mapping between https://unavailble.schema in defs.json is actually a local file instead to be fetched from the internet?
• How can I save the bundle result to a final file?

Thank you for your support!

[jdesrosiers]

How this works depends a bit on what you're doing with $ids. The easiest approach is not to use $ids and use file path relative references ($ref). In that case, you don't need to use addSchema because you're working directly with the schema files.

import { bundle } from "@hyperjump/json-schema/bundle";

(async function () {
  const bundledSchema = await bundle("file:///full/path/to/S1.json");
  console.log(bundledSchema);
}());

How can I load a local file?

If you don't use files directly, you can load them manually by reading them into memory and loading them with addSchema.

import { readFileSync } from "node:fs";

const json = readFileSync("./S1.json", "utf-8");
const schema = JSON.parse(json);

addSchema(schema);

How is it supposed to work out that $id in defs.json is the one $ref in S1 are referring to? Do I need a mapping between https://unavailble.schema/ in defs.json is actually a local file instead to be fetched from the internet?

I don't understand the question, but this might answer some of your questions. Beyond that, I think I'd need to see at least some of your schemas in order to guide you further.

How can I save the bundle result to a final file?

The easiest way is probably to console.log(bundledSchema) at the end of your script and capture the output into a file from your terminal. If you want to write to a file from the script, something like this should work ...

import { writeFileSync } "node:fs";

writeFileSync("./bundle.json", bundledSchema);

WARNING: I wrote all of this code off the top of my head and tested none of it, so it may not actually work, but it should at least point you in the right direction.

[the42]

Thank you very much for your quick reply. With your snippets I am almost up and running except, as I guessed, I run into the following issue:

const json = readFileSync("./S1.json", "utf-8");
const schema = JSON.parse(json);

addSchema(schema);
const bundledSchema = await bundle("https://unavailble.schema/schemas/S1");

I get an

  cause: Error: getaddrinfo ENOTFOUND unavailable.schema
      at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:107:26) {
    errno: -3008,
    code: 'ENOTFOUND',
    syscall: 'getaddrinfo',
    hostname: 'unavailable.schema'
  }

I did expect that error. A minimal reproduction:

S1.json

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://unavailble.schema/schemas/S1",
  "type": "object",
  "properties": {
    "dummy": {
      "$ref": "https://unavailble.schema/schemas/defs/#$defs/dummy"
    }
  }
}

defs.json (unfortunate naming, I stick now to it otherwise the thread gets confusing)

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://unavailble.schema/schemas/defs",
  "$defs": {
    "dummy": {
      "type": "object",
      "properties": {
        "name": {
          "type": "string"
        }
      }
    }
  }
}

What I am saying is I develop a schema which should finally be made available under https://unavailble.schema/schemas/, yet while developing I neither have access to this domain nor does anything exist there for fetching.

I sure can alter the schema to file uris, yet I was hoping that there is another approach:

1. using hyperjump-io/json-schema; OR
2. running a local web server which listens to https://unavailble.schema/schemas/ and maps /schemas to "."

I would rather not go through 2 as this involves creating self-signed certificate with all its intricacies and configuring that a fetch of https://unavailble.schema/schemas/defs/#$defs/dummy ($ref in S1) should actually rewrite to a GET https://unavailble.schema/schemas/defs*.json*. And I am sure I cannot get it running under privileged port 443.

[the42]

Thinking more about it and with this in mind (schema-database) I did change my small helper-program to load both files like

const json1 = readFileSync("./S1.json", "utf8");
const schema1 = JSON.parse(json1);

const json2 = readFileSync("./defs.json", "utf8");
const schema2 = JSON.parse(json2);

addSchema(schema1);
addSchema(schema2);

const bundledSchema = await bundle("https://unavailble.schema/schemas/S1");
console.log(bundledSchema);

yet I still receive the error, that unavailble.schema cant be fetched.

[the42]

Last attempt: I did change S1 to

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://unavailble.schema/schemas/S1",
  "type": "object",
  "properties": {
    "dummy": {
      "$ref": "/defs/#$defs/dummy"
    }
  }
}

so that the $ref is a relative uri and would expect that https://unavailble.schema/schemas/defs can now be found in the schema-database, yet still the same fetch exception.

[jdesrosiers]

You don't need to setup a local web server. The library will only try to fetch the schema if it doesn't already have it loaded with addSchema. If you get an error like this, it means either you haven't loaded all the necessary schemas (which you fixed), or you have a typo in one of your references.

These are your typos,

• First schema with absolute URI
  • https://unavailble.schema/schemas/defs/#$defs/dummy <= Original
  • https://unavailble.schema/schemas/defs#/$defs/dummy <= Fixed
• Second schema with relative URI
  • /defs/#$defs/dummy <= Original
  • /schemas/defs#/$defs/dummy <= Fixed
  • defs#/$defs/dummy <= Alternate fix

Ideally the library should tell you the offending URI to allow you to debug which reference is broken, but when the domain is unreachable, the error message doesn't include the full URI. This same problem came up recently, but I haven't had a chance to address it. It seems that I'm going to have to catch the error from the fetch library and throw a new more helpful error.

[the42]

Thank you very much for pointing me to this embarrassing mistakes.

The script now runs without any issues and the output is:

{
      '$id': 'https://unavailble.schema/schemas/S1',
      '$schema': 'https://json-schema.org/draft/2020-12/schema',
      type: 'object',
      properties: { dummy: { '$ref': 'defs#/$defs/dummy' } },
      '$defs': {
        'https://unavailble.schema/schemas/defs': {
          '$id': 'https://unavailble.schema/schemas/defs',
          '$defs': *[Object]*
    }
  }
}

But the bundled defs.json contains for the property dummy just [Object], my guess would have been the bundle to incorporate the whole definition, which would include the property 'name' as 'type': 'string'. Do I miss some options?

In order to print nested json objects to the console I changed console.log(bundle) to console.log(JSON.stringify(bundle)) and now I get the expected result:

{
  "$id": "https://unavailble.schema/schemas/S1",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "properties": {
    "dummy": {
      "$ref": "defs#/$defs/dummy"
    }
  },
  "$defs": {
    "https://unavailble.schema/schemas/defs": {
      "$id": "https://unavailble.schema/schemas/defs",
      "$defs": {
        "dummy": {
          "type": "object",
          "properties": {
            "name": {
              "type": "string"
            }
          }
        }
      }
    }
  }
}

[jdesrosiers]

Ah, yes, sorry. Glad you figured out that I missed the stringify step.

What you have would produce JSON with no newlines. If it's more useful to you to have it emit formatted JSON, you can use JSON.stringify(bundle, null, " ") instead. The last parameter is two spaces and indicates the size of indentation. If you prefer four spaces or tabs, you can pass that instead.

[the42]

Yes, and many thanks to get this going!

Here is my final solution which I generalized to accept file names on the command line and as an additional parameter the "root uri" (proper nomenclature?) as the start for bundling.

I hope others find that useful.

import { readFileSync } from "node:fs";
import { addSchema } from "@hyperjump/json-schema/draft-2020-12";
import { bundle } from "@hyperjump/json-schema/bundle";
import  minimist  from "minimist";

var argv = minimist(process.argv.slice(2));
var rooturi = argv['rooturi'];

if(rooturi==undefined) {
    console.log('required argument "--rooturi" not provided\n');
    console.log('\tusage: sbundle --rooturi="rooturi" file1.json file2.json ...\n');
    process.exit(-1);
}

for(const file of argv._) {
    addSchema(JSON.parse(readFileSync(file, "utf8")))
}

const bundledSchema = await bundle(rooturi);

console.log(JSON.stringify(bundledSchema, null, 2));