search

UtrechtUniversity / yoda

A system for reliable, long-term storing and archiving large amounts of research data during all stages of a study.
https://utrechtuniversity.github.io/yoda/
GNU General Public License v3.0
42 stars 26 forks source link

[BUG] Yoda uses two sources of truth for its metadata #413

Closed chStaiger closed 1 week ago

chStaiger commented 2 months ago

Is there an existing issue for this?

Current Behavior

Metadata for collections can be manipulated through the Yoda portal but also directly in the iCAT by employing the icommands or any other iRODS API. When metadata is altered trough the webportal the changes are synchronised with the iCAT. However, the metadata is not synchronised from the the iCAT to the frontent leading to different information in the two sources.

Effect: For the Archiving the metadata in the portal/metadata.json is the source of truth and the altered iCAT metadata corresponding to the portal information is forgotten. For the Search the metadata in the iCAT is the source of truth leading to wrong search results for portal users.

Expected Behavior

There needs to be one source of truth for the metadata for all Yoda processes and features.

Either synchronse the respective metadata from the iCAT to the portal when metadata keys which are also used by Yoda are changed through the icommands or any other API. Or protect the metadata keys in the iCAT to which you map the portal metadata such that they can only be written through the portal.

Steps To Reproduce

  1. I am using Yoda 1.9
  2. Create a collection 'Test_metadata' and annotate it through the portal with the title 'TEST_TITLE_CS'
  3. Test in the portal that you can find the collection by typing in 'TEST_TITLE_CS'
  4. Change e.g. the title of the collection with the icommands imeta set -C Test_metadata Title blabla or use an iRODS API of your choice to manipulate the title
  5. Check the metadata in the portal and try your search again in the portal --> collection is no longer listed
  6. Archive your data --> The new title is ignored

Environment

- Yoda: 1.9, SURF instance https://surf-yoda.irods.surfsara.nl/
- Ansible: N/A
- Operating System:N/A
- Browser:N/A

Anything else?

No response

chStaiger commented 2 months ago

Linked to #44

Danny-dK commented 2 months ago

I know far too little about this, but is this a helper?:

image

https://github.com/MaastrichtUniversity/irods_avu_json-ruleset Though I don't know whether "not allowed" means you'll get an error warning stating you're not allowed to adjust (or whether no error is given). Perhaps then specifically prepending the attribute names with 'yoda' (eg Yoda_Title, Yoda_Affiliation_Name, etc), may allow easier protecting of adjusting avu's (for those that don't want to modify the yoda-metadata.json and just want to add avu's in iRODS for whatever reason).

chStaiger commented 2 months ago

I know far too little about this, but is this a helper?:

image

https://github.com/MaastrichtUniversity/irods_avu_json-ruleset Though I don't know whether "not allowed" means you'll get an error warning stating you're not allowed to adjust (or whether no error is given). Perhaps then specifically prepending the attribute names with 'yoda' (eg Yoda_Title, Yoda_Affiliation_Name, etc), may allow easier protecting of adjusting avu's (for those that don't want to modify the yoda-metadata.json and just want to add avu's in iRODS for whatever reason).

I think this is exaclty what we need to ensure the Yoda processes while still giving flexibility to our users to define their own metadata.

lwesterhof commented 1 week ago

Duplicate of https://github.com/UtrechtUniversity/yoda/issues/44