Specify the protocol metadata schema

[SyntaxColoring]

JSON protocols and Python protocols both have a notion of protocol metadata, which, roughly, is nonessential information like the protocol author. However, they don't have a unified notion of exactly what can be in this metadata.

The new HTTP API will pass protocol metadata straight through to the client. So, it will be more client-friendly if the protocol metadata follows a consistent, unified spec.

Questions to consider

• What exactly is metadata? What does and does not belong there?
• What types can appear in metadata? Are nested dicts/objects allowed?
• What specific fields (like author) are known?
  • What are their types?
  • What are the expected formats of their values? (e.g. is author an email address, or a name? Can description have Markdown?)
  • Is a field being null equivalent to it being omitted?
  • Note: We should probably scrape our corpus of custom protocols to see what fields they use.
• Can Opentrons add more fields? If, so, where/how?
• Are fields camelCase or kebab-case or something else?
• Can users invent their own fields? If so, how should they keep them separate?
  • Like this? {"myCompanyMyCustomField": "foo"}
  • Or maybe like this? {"userExtra": {"myCustomField": "foo"} }

Definition of "done"

• A public spec addressing at least the above
  • Either machine-readable or human-readable or both

[SyntaxColoring]

It would be a good idea to get input from the participant(s) of #8236.

[mcous]

Current de-facto "standard" according to the app: https://github.com/Opentrons/opentrons/blob/6697e394458dfebf7974f7ff0edfd99f8a2e944b/app/src/redux/protocol/selectors.ts#L92-L138

[SyntaxColoring]

Thought:

What happens if something like this happens?

1. The JSON protocol schema starts with some blessed metadata field, say authors, typed as a str.
2. This blessed field makes its way into the Pydantic bindings that the robot software uses to parse the metadata.
3. Later, we change its type to optionally be a list of strings: list[str] | str.

Now, when you upload a new JSON protocol (with a list of authors) to an old robot whose Pydantic models haven't been updated yet, the robot will throw a parse error. Even though it really has no reason to care about that metadata's schema; it should just pass it on to the client transparently. There's unnecessary coupling and fragility.

Options:

• Do we have to make it a policy that after we add a metadata field, we can never change its type?
• Or should the robot server's Pydantic bindings to the JSON protocol schema not include any blessed metadata fields?
• Or should we make that JSON protocol metadata change a schema bump? (Basically, say this is still going to be an error even if it theoretically doesn't have to be—but at least make the error message nicer.)
• Or something else?

[mcous]

I think that's a good thought. I'd be pretty amenable to option 1: do not change the type of a field once "blessed". Add a new field if new information (or shape of information) is needed. Your example makes me think of npm's package.json spec, where there is both an "author" (singular) field and a "contributors" (plural) field