Tool to validate invocation.json against manifest.json

[gsfr]

As we strive to get clarity on a formal gear specification, automated validation tools will be important to help developers successfully create conformant gears.

Below is a proposal for a naming convention. Happy to hear other suggestions.

manifest.schema.json
 ↓ validate
manifest.json
 ↓ derive
invocation.schema.json
 ↓ validate
invocation.json

The fundamental gear specification will define the manifest.schema.json. The gear author will provide the manifest.json. Code will derive an invocation.schema.json from the manifest.json. The gear user, assisted by a UI, will create the invocation.json.

The invocation.schema.json will likely not need to exist explicitly. Rather, the user experience should be to validate an invocation.json directly against a manifest.json.

This tool should do a number of things:

• validate a manifest.json against the manifest.schema.json
• derive an invocation.schema.json from a manifest.json
• validate an invocation.json against a manifest.json

cc @ryansanford, @lmperry

[kofalt]

That looks like what I've got cooking, yeah. I would turn the arrows around on validate though; the schemas validate what's listed under them (edit: change made). Replace - with . to follow this filename, which I've seen used elsewhere with json schemas. Cleaner with multiple extensions.

Happily accepting alternatives to invocation to avoid wordiness. This approach is robust & verifiable, which we want, but I want to avoid falling for the trap of an overly-clever abundance of new terms to learn. I can already see how this would be confusing, and I'd rather take run for example, even though it's less specific.

[kofalt]

We'll have to be very clear when documenting this, both for gear designers and tool builders. It might help if we pulled in some ideas from sites like this one to show what you can do.

"Add json schema snippets to this map" might work for someone who has used json schemas before - read: nobody - instead we could go straight to things like maxLength: 3 because that's much simpler.

[gsfr]

Why don't we get started by building this tool for gear spec v0?

Also, in the context of this tool, filenames should be irrelevant.

We should be careful when making assumptions about our users' prior exposure to certain tech, or lack thereof, or which terminology they might find intuitive or not.

[kofalt]

I intend to put an argparser wrapper around jsonschema.validate(mf, MANIFEST_SCHEMA) basically. The tool should live in this repo, and be something we can pip-import into the core as a library & use during an HTTP request. Any advice on anything I should do beyond "make a folder & put code in it" to make that work best?

Yup on filenames, they will be passed as arguments to the CLI or function call. I'll make that change in the ticket so it can match our examples then.

Double yup on technology exposure :) Sounds like you agree with my idea of talking about the possible declarations directly. We'll want to talk later about documenting strategy.

[kofalt]

This was accomplished in spades awhile back and we didn't close the ticket.