[Idea] Format to exchange information between systems

[matmair]

North star

My idea is to have an interchange format to support 3 main use cases:

• export/import data vendor-neutral in a simple
• exchange information between systems regularly
• create converters / API adapters that enable transitioning between systems

The format should work as a file and REST API, and be either very efficient or human-readable and at least theoretically writeable.

The format should be defined in a publicly accessible repo, with SemVer versions, a vendor-neutral name and made available in a widely-used technology that allows client generation. Distribution of clients is optional at this point.

Technical details

Every object in the format should:

• Require the least amount of fields possible to be filed out[^1]
• Provide a "stable" primary reference field
• Allow additional reference fields[^2]
• Provide a canonical URL field (optional)
• Provide a free-form key-value section that can be extended with vendor-specific data[^3][^4]
• Provide a parent field to enable the reconstruction of hierarchies (optional)

Each file should contain an optional export section with:

• Exporting system software[^3]
• Exporting system public name (optional)
• Export timestamp
• Export reason (optional)
• Free form key-value section [^3][^4]

Entities:

• Part
• Organisation (Manufacturer, Owner, Supplier)
• Category
• Location
• Stock

Possible Entities:

• Projekt
• File
• Order
• Build
• Test

Suggestions

As a file format/definition, I suggest Protobuf for exchange-files/(streamed) API communication and JSON Schema (that means YAML could also be used and should be preferred IMO) for human-readable files. GitHub should be used to define and publish the format for now.

[^1]: Implementing software can decide how it wants to handle missing information it requires (via backfilling or similar strategies) [^2]: For optional matching / de-duplication [^3]: There should be an index of official adaptors of the format and their vendor identification. [^4]: Keys in the free-form section should use https://de.wikipedia.org/wiki/Uniform_Resource_Name / https://datatracker.ietf.org/doc/html/rfc2141

[jbtronics]

The format should work as a file and REST API, and be either very efficient or human-readable

I think it is important to be human readable, so verbose keys, etc. I do not think that size of the resulting files should be a real concern, as even thousands of parts contained inside a single file would just be a few MBs.

Provide a "stable" primary reference field

Do you mean stable across a single application/file or globally like with a hash of the data or an UUID?

Entities:

I think for now it would be enough to define only directly part related stuff. So what properties a part has, how and where you can buy it, etc. I think related files like datasheets, etc. would be important here too (even though there is the question how to share local files/non-URLs easily).

[matmair]

I mean stable as stable between exports from the exporting application

I think all the entities I listed are needed to describe parts. not sure how to handle files if they should be portable between systems.

I will create a small POC to test that out. What do you think about the text in general @jbtronics? Should I transfer that to a preamble?