[data provider issue] `oai.samvera` records with blank thumbnails -- `isShownBy` URL formatting (9 records from 5 collections)

[christinklez]

Mapper: oai.samvera Problem: The OAI for certain records do not include the full/!200,200/0/default.jpg tail in the URL construction. Adding these in (manually) point to a working image file. Observation: This seems to occur for select records that have ARKs that contain the text characters png or jpg. Proposed next step: We need to ask UCLA to fix these URLs.

Note: This is not harvest-stopping error, and these collections are currently on -stage! The records in question have a grey tile for a thumbnail.

To Do:

• [ ] Contact UCLA to let them know about the isShownBy URL formatting issues
• [ ] Reharvest collections:
  • [ ] 153 (map index 4, 5, 14, 35)
  • [ ] 154 (map index 16, 37)
  • [ ] 28108 (map index 5)
  • [ ] 28111 (map index 23)
  • [ ] 28230 (map index 29)

Registry ID: 153 - errors in 4 mapped index

• Run ID: manual__2024-05-06T22:47:31+00:00
• permalink to airflow gridview

oai:library.ucla.edu:ark:/21198/zz002dcpng

• mapped index 4
• Exception: PIL.UnidentifiedImageError for calisphere-id oai:library.ucla.edu:ark:/21198/zz002dcpng: cannot identify image file '/tmp/ark%3A%2F21198%2Fzz002dcpng'
• OAI: https://digital.library.ucla.edu/catalog/oai?verb=GetRecord&metadataPrefix=oai_dpla&identifier=oai:library.ucla.edu:ark:/21198/zz002dcpng
  • https://iiif.library.ucla.edu/iiif/2/ark%3A%2F21198%2Fzz002dcpng
• on -test: not on -test

oai:library.ucla.edu:ark:/21198/zz0002pngj

• mapped index 5
• Exception: PIL.UnidentifiedImageError for calisphere-id oai:library.ucla.edu:ark:/21198/zz0002pngj: cannot identify image file '/tmp/ark%3A%2F21198%2Fzz0002pngj'
• OAI: https://digital.library.ucla.edu/catalog/oai?verb=GetRecord&metadataPrefix=oai_dpla&identifier=oai:library.ucla.edu:ark:/21198/zz0002pngj
  • https://iiif.library.ucla.edu/iiif/2/ark%3A%2F21198%2Fzz0002pngj
• on -test: https://calisphere-test.cdlib.org/item/ark:/21198/zz0002pngj/
  • (has a thumbnail on -test)

oai:library.ucla.edu:ark:/21198/zz0002nzwn

• also on mapped index 5 -- Note that this may be a non-breaking error. It is definitely a different error than the other records listed in this issue.
• Error downloading https://iiif.library.ucla.edu/iiif/2/ark%3A%2F21198%2Fzz0002nzwn/full/!200,200/0/default.jpg: 401 Client Error: Unauthorized for url: https://iiif.library.ucla.edu/iiif/2/ark%3A%2F21198%2Fzz0002nzwn/full/!200,200/0/default.jpg ERROR: no thumbnail found for ['image']record oai:library.ucla.edu:ark:/21198/zz0002nzwn in page 153/vernacular_metadata_2024-05-06T22:47:43/mapped_metadata_2024-05-06T22:52:54/data/98.jsonl
• OAI: https://digital.library.ucla.edu/catalog/oai?verb=GetRecord&metadataPrefix=oai_dpla&identifier=oai:library.ucla.edu:ark:/21198/zz0002nzwn
  • https://iiif.library.ucla.edu/iiif/2/ark%3A%2F21198%2Fzz0002nzwn/full/!200,200/0/default.jpg
• on -test: https://calisphere-test.cdlib.org/item/ark:/21198/zz0002nzwn/
  • (has a thumbnail on -test)

oai:library.ucla.edu:ark:/21198/zz002ctjpg

• mapped index 14
• Exception: PIL.UnidentifiedImageError for calisphere-id oai:library.ucla.edu:ark:/21198/zz002ctjpg: cannot identify image file '/tmp/ark%3A%2F21198%2Fzz002ctjpg'
• OAI: https://digital.library.ucla.edu/catalog/oai?verb=GetRecord&metadataPrefix=oai_dpla&identifier=oai:library.ucla.edu:ark:/21198/zz002ctjpg
  • https://iiif.library.ucla.edu/iiif/2/ark%3A%2F21198%2Fzz002ctjpg
• on -test: https://calisphere-test.cdlib.org/item/ark:/21198/zz002ctjpg/
  • (has a thumbnail on -test)

oai:library.ucla.edu:ark:/21198/zz002cpng1

• mapped index 35
• Exception: PIL.UnidentifiedImageError for calisphere-id oai:library.ucla.edu:ark:/21198/zz002cpng1: cannot identify image file '/tmp/ark%3A%2F21198%2Fzz002cpng1'
• OAI: https://digital.library.ucla.edu/catalog/oai?verb=GetRecord&metadataPrefix=oai_dpla&identifier=oai:library.ucla.edu:ark:/21198/zz002cpng1
  • https://iiif.library.ucla.edu/iiif/2/ark%3A%2F21198%2Fzz002cpng1
• on -test: https://calisphere-test.cdlib.org/item/ark:/21198/zz002cpng1/
  • (has a thumbnail on -test)

Registry ID: 154 - errors in 2 mapped index

• Run ID: manual__2024-05-03T23:58:47+00:00
• permalink to airflow gridview

oai:library.ucla.edu:ark:/21198/zz00280jpg

• permalink to map index 16 error log
• PIL.UnidentifiedImageError for calisphere-id oai:library.ucla.edu:ark:/21198/zz00280jpg: cannot identify image file '/tmp/ark%3A%2F21198%2Fzz00280jpg'
• OAI: https://digital.library.ucla.edu/catalog/oai?verb=GetRecord&metadataPrefix=oai_dpla&identifier=oai:library.ucla.edu:ark:/21198/zz00280jpg
  • https://iiif.library.ucla.edu/iiif/2/ark%3A%2F21198%2Fzz00280jpg
• on -test: https://calisphere-test.cdlib.org/item/ark:/21198/zz00280jpg/
  • (has a thumbnail on -test)

oai:library.ucla.edu:ark:/21198/zz00288png

• permalink to map index 37 error log
• PIL.UnidentifiedImageError for calisphere-id oai:library.ucla.edu:ark:/21198/zz00288png: cannot identify image file '/tmp/ark%3A%2F21198%2Fzz00288png'
• OAI: https://digital.library.ucla.edu/catalog/oai?verb=GetRecord&metadataPrefix=oai_dpla&identifier=oai:library.ucla.edu:ark:/21198/zz00288png
  • https://iiif.library.ucla.edu/iiif/2/ark%3A%2F21198%2Fzz00288png
• on -test: https://calisphere-test.cdlib.org/item/ark:/21198/zz00288png/
  • (has a thumbnail on -test)

Registry ID: 28108 - errors in 1 mapped index

• Run ID: manual__2024-05-03T23:59:53+00:00
• permalink to airflow gridview

oai:library.ucla.edu:ark:/21198/zz002hpngp

• permalink to map index 5 error log
• PIL.UnidentifiedImageError for calisphere-id oai:library.ucla.edu:ark:/21198/zz002hpngp: cannot identify image file '/tmp/ark%3A%2F21198%2Fzz002hpngp'
• OAI: https://digital.library.ucla.edu/catalog/oai?verb=GetRecord&metadataPrefix=oai_dpla&identifier=oai:library.ucla.edu:ark:/21198/zz002hpngp
  • https://iiif.library.ucla.edu/iiif/2/ark%3A%2F21198%2Fzz002hpngp
• on -test: https://calisphere-test.cdlib.org/item/ark:/21198/zz002hpngp/
  • (No thumbnail on -test; displays empty tile.)

Registry ID: 28111 - errors in 1 mapped index

Run ID: manual__2024-05-04T00:00:02+00:00

• permalink to airflow gridview

oai:library.ucla.edu:ark:/21198/zz0025pngj

• permalink to map index 23 error log
• PIL.UnidentifiedImageError for calisphere-id oai:library.ucla.edu:ark:/21198/zz0025pngj: cannot identify image file '/tmp/ark%3A%2F21198%2Fzz0025pngj'
• OAI: https://digital.library.ucla.edu/catalog/oai?verb=GetRecord&metadataPrefix=oai_dpla&identifier=oai:library.ucla.edu:ark:/21198/zz0025pngj
  • https://iiif.library.ucla.edu/iiif/2/ark%3A%2F21198%2Fzz0025pngj
• on -test: https://calisphere-test.cdlib.org/item/ark:/21198/zz0025pngj/
  • (has a thumbnail on -test)

Registry ID: 28230 - error in 1 mapped index

• Run ID: manual__2024-05-06T22:20:12+00:00
• permalink to airflow gridview

oai:library.ucla.edu:ark:/21198/zz0002fpng

• permalink to map index 29 error log
• Exception: PIL.UnidentifiedImageError for calisphere-id oai:library.ucla.edu:ark:/21198/zz0002fpng: cannot identify image file '/tmp/ark%3A%2F21198%2Fzz0002fpng'
• OAI: https://digital.library.ucla.edu/catalog/oai?verb=GetRecord&metadataPrefix=oai_dpla&identifier=oai:library.ucla.edu:ark:/21198/zz0002fpng
  • https://iiif.library.ucla.edu/iiif/2/ark%3A%2F21198%2Fzz0002fpng
• on -test: This is a new collection (not on -test)

[barbarahui]

@christinklez You can investigate these by downloading the mapped metadata, searching for the item, and checking the is_shown_by. For example, for the first error, I can see in the log that the mapped metadata file is 154/vernacular_metadata_2024-05-03T23:59:02/mapped_metadata_2024-05-04T00:01:07/data/80.jsonl.

I download this file from S3 and search for oai:library.ucla.edu:ark:/21198/zz002hpngp. The is_shown_by for this record is https://iiif.library.ucla.edu/iiif/2/ark%3A%2F21198%2Fzz00280jpg . When I go to this URL in a browser, I get an info.json page. This is not actually an image, which is what is causing the content harvester to throw an error.

I'm not sure if this is a provider issue or a rikolti mapper error? I'm guessing it's a provider issue since we were able to get thumbnails for most of the items in this collection.

[barbarahui]

For collection 154, most of the objects have is_shown_by URLs with this kind of format:

https://iiif.library.ucla.edu/iiif/2/ark%3A%2F21198%2Fzz00288pmz/full/!200,200/0/default.jpg

The 2 objects that are failing have these is_shown_by URLs:

oai:library.ucla.edu:ark:/21198/zz00280jpg - https://iiif.library.ucla.edu/iiif/2/ark%3A%2F21198%2Fzz00280jpg oai:library.ucla.edu:ark:/21198/zz00288png - https://iiif.library.ucla.edu/iiif/2/ark%3A%2F21198%2Fzz00288png

If I add /full/!200,200/0/default.jpg to the end, then I get a thumbnail image:

oai:library.ucla.edu:ark:/21198/zz00280jpg - https://iiif.library.ucla.edu/iiif/2/ark%3A%2F21198%2Fzz00280jpg/full/!200,200/0/default.jpg oai:library.ucla.edu:ark:/21198/zz00288png - https://iiif.library.ucla.edu/iiif/2/ark%3A%2F21198%2Fzz00288png/full/!200,200/0/default.jpg

[christinklez]

Look at the vernacular metadata to review the isShownBy thumbnail URLs.

[christinklez]

Looked at the OAI for each of these records, and confirming that they do not have the full/!200,200/0/default.jpg at the tail of the IIIF URLs. We can plan to send a heads up to UCLA about these records.

In the meantime, we can consider ETL'ing these collections instead.

[christinklez]

Thanks to #899, we're able to get these collections through to -stage! But these records (as expected) have broken thumbnails.

153

(map index 4, 5, 14, 35) https://calisphere-stage.cdlib.org/item/ark:/21198/zz002dcpng https://calisphere-stage.cdlib.org/item/ark:/21198/zz0002pngj/ https://calisphere-stage.cdlib.org/item/ark:/21198/zz002cpng1/

154

(map index 16, 37) https://calisphere-stage.cdlib.org/item/ark:/21198/zz00280jpg/ https://calisphere-stage.cdlib.org/item/ark:/21198/zz00288png/

28108

(map index 5) https://calisphere-stage.cdlib.org/item/ark:/21198/zz002hpngp/

28111

(map index 23) https://calisphere-stage.cdlib.org/item/ark:/21198/zz0025pngj/

28230

(map index 29) https://calisphere-stage.cdlib.org/item/ark:/21198/zz0002fpng/