Request for more business-specific metadata types

[ivaylomitrev]

---

       Prosjekt  NOARK 5 Tjenestegresesnitt
       Kategori  Noark 5.5.0 TG versjon 1.0
    Alvorlighet  kommentar
   Meldingstype request
Brukerreferanse  user@example.com
    Dokumentdel  #6

---

Beskrivelse

Is it possible to add more business-specific metadata types in the specification? VSM metadatakatalog.

Namely, the Noark 5.5 standard lets vendors decide how to implement business-specific metadata whereas the Noark 5 API specification imposes very specific limits. Luckily, our implementation almost matches what the specification requires, but emphasis is on almost.

We support multiple values per VSM field. In other words, our implementation requires:

"virksomhetsspesifikkeMetadata": {
    "ppt-v1:henvisningdato": ["2018-04-22T13:30:00"],
    "ppt-v1:skoleaar": ["2018/2019"],
    "ppt-v1:saksnummer": [123]
}

instead of the suggested:

"virksomhetsspesifikkeMetadata": {
    "ppt-v1:henvisningdato": "2018-04-22T13:30:00" ,
    "ppt-v1:skoleaar": "2018/2019",
    "ppt-v1:saksnummer": 123
}

Technically, the current VSM requirements can be supported by us only if we (1) allow loss of data and/or if we (2) limit integrating parties to only seeing one value instance per field (which is error-prone).

1. The loss of data scenario occurs on updates where we do not know which exact field a client may have requested to update and we could only "replace" all VSM (especially considering the PUT requirements for replacing as opposed to PATCH).
2. The limitation occurs when we need to return VSM to clients. We must decide to return a subset of the VSM to clients, thus, misleading them, and setting them up for the first scenario.

Ønsket endring

Is it possible to add support for new VSM types. Namely, arrays of each of the already supported ones (which would be backwards-compatible).

The currently supported types are:

• boolean
• date
• datetime
• integer
• decimal
• string
• uri

We will be able to comply with the specification if the following new types were added:

• boolean array
• date array
• datetime array
• integer array
• decimal array
• string array
• uri array

[tsodring]

This is a bit tricky. This is not a version 1.0 functionality and any addition will have to be a new version. The editorial board has planned a meeting in 2 weeks. Perhaps you could send a list over changes you feel are relevant and this can be discussed.

[petterreinholdtsen]

The current simple structure was selected to provide a simple key/value system that is easily represented in a XSD. What is the use case for introducing an array as well? An alternative to providing several array types is to allow each key to show up several times (ie the metadata set is a array, not a dict. Not sure how easy it is to represent such structure in XML.

[ivaylomitrev]

The reason is compatibility with existing vendor solutions :) Otherwise, this is a limitation that would force vendors to modify their archival cores in an attempt to match the requirements of the specification. For us, such modification of archival core is technically impossible as it is backwards-incompatible for existing customer data (since business-specific metadata has been in use long before the requirements were laid out in this specification) and there are existing integrations using it. Considering the fact that business-specific metadata is not defined in the standard, it is likely that other vendors will experience the same issue.

[petterreinholdtsen]

We touched on this idea during todays editorial meeting, and I had a closer look. It is unclear from your example why it would make sense to use multiple values for the fields "ppt-v1:henvisningdato", "ppt-v1:skoleaar" and "ppt-v1:saksnummer"? All of these seem to be single value fields? Can you explain the rationale?

Also, given that a boolean only have two valid values, true and false, what would be the use case for a list of booleans?

Note, I am not against adding support for lists as a VSM type, but want to make sure it is added in a way that make sense. It would also help me if the XSD structure to generate for such arrays were proivded as an example here.

My gut feeling is that if we introduce array types, it should be specified per field if it is a single value or a list.

[ivaylomitrev]

It is unclear from your example why it would make sense to use multiple values for the fields "ppt-v1:henvisningdato", "ppt-v1:skoleaar" and "ppt-v1:saksnummer"? All of these seem to be single value fields? Can you explain the rationale?

These are just examples I got from the specification itself so that I could exemplify my point. I did not intend to scrutinize these particular fields/names specifically. Please do not pay attention to the particular naming of the fields. Valid examples (I assume) could be project types, or people, or functions - the options are endless and entirely depend on a chosen archive structure.

It would also help me if the XSD structure to generate for such arrays were provided as an example here.

A (much older) version of the Documaster BSM schema can be found in the company's GitHub repository. There are some additions (boolean and url) as a result of the necessity to support this specification, but it has generally not changed in its core.

Also, given that a boolean only have two valid values, true and false, what would be the use case for a list of booleans?

The recommendation was just for consistency. This is also the reason we ourselves support boolean arrays. This way single-value and multi-value types are consistent and there are no exceptions to the rule. For the boolean array type we have left to our customers to decide whether they have a valid use case or not. I have no use case to provide other than the argumentation above.