proposal: thing-model-catalog: handling attachments across federated repositories

[alexbrdn]

[Description]

Context

TMs across multiple federated repositories have unique IDs which include content hash. Attachments, on the other hand, do not. They are identified only by their file name and the reference to attachment container (either a TM name or a TM ID). Hence, while it is possible to unambiguously join lists of TMs from several federated repositories, the same is not possible for attachments. This means that reading a single TM's metadata, which includes the list of attachments, or performing other operations on attachments requires specifying a single repository for disambiguation.

However, specifying a repository is not possible when using the REST API. A.t.m. there is only the push repository config option which allows selecting one of the multiple configured repositories as the target for write operations, but that option cannot be influenced by the API consumer.

[How]

Proposal

To work around these issues altogether, instead of merging and deduplicating the result lists from several federated repositories, the results will be simply joined together by appending and sorting. The results of list and versions, as well as single TM metadata will include the name of the repository where the data comes from. When there is only one repository configured (or an ad-hoc repository used), this name will be omitted. Requests for attachment list / single TM metadata will return an error if the exact id is contained in more than one repository. The user then will have to specify a repository name when making requests which might be ambiguous. E.g. all attachment operations are presumed to be ambiguous w.r.t. target repository. All write operations must have the target repository specified as well.

That is, the both CLI commands and the REST API will be extended with respective field in the results and a request parameter/CLI flag where relevant.

Giving the repositories descriptive names, which would allow the consumer to make an informed decision, is left for the published of the TMC. Repos config will be extended with an optional description field to help with this decision.

We will provide a separate API endpoint to request the list of available repos names, analogous to repo list, e.g. for POSTing new TMs.

cc => @hadjian @EVO-Antoniazzi

[alexbrdn]

EDIT: rewritten proposal part in light of discussion

[alexbrdn]

EDIT: add proposal for repos list endpoint

[alexbrdn]

EDIT: added some more details for clarity