nuxeo.nuxeo validation results

[christinklez]

mapping issues (see UCI university archives reports):

• [x] rights / enrichment(?) issue: rights statement of last resort (when a rights statement is not provided by the contributor or registry) is not displaying (see examples in UCI UA #29, #26713) . (2024-01-23: As a temp workaround, we added the statement of last resort into Registry for these collections; see ttps://github.com/ucldc/rikolti/issues/706).
• [x] location: location is not mapping; please map physical location (see examples in UCI UA #192, #194, #3434, #26702, #26713, #26851, #26880)
• [x] temporal: temporal is not mapping; please map temporal to time period (see example in UCI UA #26855)
• [x] creator: map creator (see UCR WRCA #26142) [this is fetching/mapping fine; no fix needed]
• [x] subject: no subject provided in Nuxeo; Rikolti is mapping [''] (see example in UCI UA #25889)
• [x] date: no date provided in Nuxeo; Rikolti is mapping [''] (see example in UCI UA #26851)
• [x] date: extra blank space appears before the date, example: expect ['2004-09-01'] but Rikolti maps [' 2004-09-01']. (see examples in UCI UA #29, #26557 -- these are single dates)
• [x] date: if date is a range (['1980-05-16']), Rikolti adds an extra space after ['1980-05-16 '] (see: UCSF UA #105) or before [' 1960'] (see: UCSF UA #27010)

expected is_shown_by issues? (since we will not be deep harvesting anymore); will get these thumbnails as part of the content harvest process:

• [ ] is_shown_by for audios: ""invalid Rikolti value type"" error reported. For example, expect 'https://s3.amazonaws.com/static.ucldc.cdlib.org/ucldc-nuxeo-thumb-media/8d9045ce-599a-433f-9f43-9a73663ed8f4' but Rikolti maps 'None' (see examples in UCI UA #26711, #27795, #27808--these are all complex objects with an audio file as the main content file.) I suspect there is supposed to be logic to grab a thumbnail from another object (i.e., pdf transcript thumbnail image)
• [ ] is_shown_by for videos: ""Invalid Rikolti value"" -- Expect: ""https://s3.amazonaws.com/static.ucldc.cdlib.org/ucldc-nuxeo-thumb-media/635da522-de0e-45f6-96be-8a5852b9351d"" Actual: ""None"" -- This is a main content video fileThis is a complex object with no file at the parent level; legacy provides an s3 URL for a thumbnail image, Rikolti needs updated logic (see UCB ESL #27549)

id mismatch issues:

• [x] id: some records have new id's; the expected id matches Nuxeo uuid, which continues to resolve to Nuxeo/CSphere; unsure where the new id is coming from (see examples in #4734, #17154, #26927, #27755, #27808) -- these collections do not have ARKs assigned, and it seems Rikolti is assigning new id's, instead of repurposing the Nuxeo UUID.

expected validation discrepancies!

• [x] is_shown_by: ""Unusual Solr value"" error reported. For example, expect 'https://nuxeo.cdlib.org/Nuxeo/nxpicsfile/default/00047287-275f-4e78-b0e7-4d6a7b7ba278/Medium:content/' but Rikolti maps '{'url': 'https://nuxeo.cdlib.org/nuxeo/nxfile/default/00047287-275f-4e78-b0e7-4d6a7b7ba278/picture:views/2/content/Medium_J525-15.jpg?changeToken=9-0', 'mimetype': 'image/jpeg', 'filename': 'Medium_J525-15.jpg', 'nuxeo_type': 'SampleCustomPicture'}' (see examples in #38, #194, #4734, #17154, #25589, #26713, #26880, #26927, #27808) --
  • simple-objects image issues only #38, #194, #4734, #17154, #25589
  • complex-objects with only images: #25889, #26880
  • complex-object: main content is a pdf, includes images: #26927
  • complex-object: main content is a video, includes text: #27808

[barbarahui]

I'm not seeing any problem for collection 26142 re mapping creator. Can you re-run the validate_by_mapper DAG for it?

[christinklez]

@barbarahui you're right! I'm going to strike the "creator" mapping one from the list. Thanks!

[barbarahui]

@christinklez re the extra blank space appearing before dates: this is how the data is actually entered in Nuxeo. I've added whitespace trimming to the Nuxeo mapper, but if this is widespread then there was some sort of data entry issue...

[barbarahui]

Re thumbnail_source for complex objects, we need to replicate this logic, which grabs a thumbnail from a component object if it's available:

https://github.com/ucldc/harvester/blob/master/harvester/fetcher/nuxeo_fetcher.py#L79-L141

I created an issue for this work: Issue: https://github.com/ucldc/rikolti/issues/720

[barbarahui]

@christinklez the is_shown_by for video example you provide above is actually for a complex object. Same issue as for the audio example.

[christinklez]

Thanks so much, @barbarahui! Reviewed a set of collections and the mapping fixes look good. Thanks!