UtrechtUniversity / yoda

A system for reliable, long-term storing and archiving large amounts of research data during all stages of a study.
https://utrechtuniversity.github.io/yoda/
GNU General Public License v3.0
44 stars 26 forks source link

[FEATURE] Dynamically extend JSON metadata #307

Open jjkoehorst opened 10 months ago

jjkoehorst commented 10 months ago

Is your feature request related to a problem? Please describe.

We are currently looking into converting metadata documents into the Yoda JSON documents. However not all fields overlap with the Yoda schema and some projects have additional fields on top of the shared fields. I tried to add an additional field to the Yoda JSON file but this is not allowed as the following message appeared:

The metadata file is not compliant with the schema.
Metadata form - /research-xxx-projects-example

It is not possible to load this form as the yoda-metadata.json file is not in accordance with the form definition.

Please check the following in your JSON file:

This extra field is not allowed: Some free text
When using the 'Delete all metadata' button beware that you will lose all data!

Describe the solution you'd like

It would be great if we can have a more dynamic system where the core is preserved but additional metadata fields can be added dynamically on top of the Yoda or community templates as the basis never covers all metadata that could be potentially be provided.

Describe alternatives you've considered

In theory it is possible to generate a Yoda template for each project but it would be great if it is possible to dynamically extend a core schema using the Yoda-metadata.json document. Users themselves are not the preferred group to modify these json files but metadata systems that are in place during the project could generate this.

Additional context

    "Some free text": "some free text content in the box",
    "Data_Access_Restriction": "Restricted - available upon request", 
    "Title": "some title", 
    "Description": "some description", 
    "Version": "1.0 en nog wat", 
    "Retention_Information": "minimum 5 years", 
    "Data_Classification": "Basic", 
    "Collection_Name": "name of a collection", 
    "License": "Open Data Commons Attribution License (ODC-By) v1.0"
}

The "some free text" is a label of content provided but this sentence is not part of the official schema.

Danny-dK commented 10 months ago

As I understand, from Yoda 1.9 onwards, metadata forms / templates can be made available per Yoda research-group (and thus per project). That already gives a bit more flexibility in getting different metadata forms available. Making it available is another thing. Once a metadata form is altered (as base form + alteration) it needs some attention when updates are made to the base Yoda metadata form. If only a couple alterations of the metadata form are present, it can be doable. If 100s of iterations exist, it may become cumbersome to keep all forms updated to the correct form. In the case of Yoda@WUR we for now rely on SURF providing hosting services keeping everything updated, it is not part of the services now to keep different metadata forms / templates (although Yoda is capable of doing this). In the near future we (WUR) hope that we will be able to this ourselves with increased expertise and manpower. I would imagine that full freedom in creating metadata forms is less desirable as it may create an uncontrollable chaos of different versions of metadata. At this point, the Yoda metadata terms are also translated to DataCite terms so that they are harvestable by and findable in DataCite. I'm unfamiliar how this would work for other metadata added (would those metadata fields/terms still be searchable and would you need mappings to other metadata standards).

RobvanSchip commented 1 month ago

Thank you for the feature request. We are currently designing a more modular metadata system. This is planned for version 2.0 and we have registered it in our product backlog as ticket nr. YDA-5802