Nuxeo `content_harvest` issues for #27124 - various error messages

christinklez commented 1 month ago

Registry ID: 27124 Mapper: Nuxeo Run ID: manual__2024-10-08T05:57:09+00:00 Link to Airflow: https://7a8067cb-3b99-477e-a883-7e311175a9b4.c3.us-west-2.airflow.amazonaws.com/dags/harvest_collection/grid?dag_run_id=manual__2024-10-08T05%3A57%3A09%2B00%3A00&num_runs=365

`convert: cache resources exhausted`

[ ] mapped task 2:

ERROR: make_jp2 command failed: ['/usr/bin/convert', '-quiet', '-format', '-jp2', '-define', 'jp2:rate=10', '/tmp/sim_137_001_0001.jpg[0]', '/tmp/sim_137_001_0001.jp2']
returncode was: 1
output was: 
convert: cache resources exhausted `/tmp/sim_137_001_0001.jpg' @ error/cache.c/OpenPixelCache/4095.
convert: no images defined `/tmp/sim_137_001_0001.jp2' @ error/convert.c/ConvertImageCommand/3229.

Nuxeo: https://nuxeo.cdlib.org/nuxeo/nxdoc/default/8b0931c4-d523-4960-9980-2c5b2845feca/view_documents
Calisphere-stage: https://calisphere-stage.cdlib.org/item/ark:/86086/n25m679x/

[ ] mapped task 3:

ERROR: make_jp2 command failed: ['/usr/bin/convert', '-quiet', '-format', '-jp2', '-define', 'jp2:rate=10', '/tmp/sim_145_000_116_0776.jpg[0]', '/tmp/sim_145_000_116_0776.jp2']
returncode was: 1
output was: 
convert: cache resources exhausted `/tmp/sim_145_000_116_0776.jpg' @ error/cache.c/OpenPixelCache/4095.
convert: no images defined `/tmp/sim_145_000_116_0776.jp2' @ error/convert.c/ConvertImageCommand/3229.

Nuxeo: https://nuxeo.cdlib.org/nuxeo/nxdoc/default/8206df43-c038-49c6-a892-fe65c7ae0c41/view_documents
Calisphere-stage: https://calisphere-stage.cdlib.org/item/ark:/86086/n29025fd/

`convert: width or height exceeds limit`

[ ] mapped task 4:

ERROR: make_jp2 command failed: ['/usr/bin/convert', '-quiet', '-format', '-jp2', '-define', 'jp2:rate=10', '/tmp/sim_156_000_114_1247.jpg[0]', '/tmp/sim_156_000_114_1247.jp2']
returncode was: 1
output was: 
convert: width or height exceeds limit `/tmp/sim_156_000_114_1247.jpg' @ error/cache.c/OpenPixelCache/3909.
convert: no images defined `/tmp/sim_156_000_114_1247.jp2' @ error/convert.c/ConvertImageCommand/3229.

Nuxeo: https://nuxeo.cdlib.org/nuxeo/nxdoc/default/6a55236e-9a18-48db-a590-ce3a0c9d0992/view_documents
Calisphere-stage: https://calisphere-stage.cdlib.org/item/ark:/86086/n2xp76qs/

`make_jp2 command failed`

[ ] mapped task 21: - failed on retry, same file

--------------------------------
ERROR: make_jp2 command failed: ['/usr/bin/convert', '-quiet', '-format', '-jp2', '-define', 'jp2:rate=10', '/tmp/sim_057_000_001_0037.tif[0]', '/tmp/sim_057_000_001_0037.jp2']
returncode was: -6
output was: 
--------------------------------; 13632)

Nuxeo: https://nuxeo.cdlib.org/nuxeo/nxdoc/default/98110137-29b5-4c7e-ba81-c71c3124c444/view_documents
Calisphere-stage: https://calisphere-stage.cdlib.org/item/ark:/86086/n2fj2j1b/?order=36 (child record)

[ ] mapped task 28:

--------------------------------
ERROR: make_jp2 command failed: ['/usr/bin/convert', '-quiet', '-format', '-jp2', '-define', 'jp2:rate=10', '/tmp/sim_063_009_001_0027.tif[0]', '/tmp/sim_063_009_001_0027.jp2']
returncode was: -6
output was: 
--------------------------------; 210)

https://nuxeo.cdlib.org/nuxeo/nxdoc/default/fa825f58-34cc-4f80-be5e-c897afb33923/view_documents Calisphere-stage: https://calisphere-stage.cdlib.org/item/ark:/86086/n2wh2r8n/?order=26 (child record)

`convert: Failed to seek in the stream.`

[ ] mapped task 32: - failed on retry, same file

ERROR: make_jp2 command failed: ['/usr/bin/convert', '-quiet', '-format', '-jp2', '-define', 'jp2:rate=10', '/tmp/sim_055_008_001_0182.tif[0]', '/tmp/sim_055_008_001_0182.jp2']
returncode was: 1
output was: 
convert: Failed to seek in the stream.
`OpenJP2' @ error/jp2.c/JP2ErrorHandler/194.
convert: unable to encode image file `/tmp/sim_055_008_001_0182.jp2' @ error/jp2.c/WriteJP2Image/1141.
convert: unable to write file `/tmp/sim_055_008_001_0182.jp2' @ error/constitute.c/WriteImage/1346.

Nuxeo: https://nuxeo.cdlib.org/nuxeo/nxdoc/default/c2a880f5-9e77-4062-a854-e546ba10fac1/view_documents
Calisphere-stage: https://calisphere-stage.cdlib.org/item/ark:/86086/n2hq4151/?order=181 (child record)

[ ] mapped task 38:

ERROR: make_jp2 command failed: ['/usr/bin/convert', '-quiet', '-format', '-jp2', '-define', 'jp2:rate=10', '/tmp/sim_025_000_002_0094.tif[0]', '/tmp/sim_025_000_002_0094.jp2']
returncode was: 1
output was: 
convert: Failed to seek in the stream.
`OpenJP2' @ error/jp2.c/JP2ErrorHandler/194.
convert: unable to encode image file `/tmp/sim_025_000_002_0094.jp2' @ error/jp2.c/WriteJP2Image/1141.
convert: unable to write file `/tmp/sim_025_000_002_0094.jp2' @ error/constitute.c/WriteImage/1346.

Nuxeo: https://nuxeo.cdlib.org/nuxeo/nxdoc/default/bf1ee640-7b8d-418f-bfba-fd04f9142194/view_documents
Calisphere-stage: https://calisphere-stage.cdlib.org/item/ark:/86086/n2pg1t7f/?order=93 (child record)

[ ] mapped task 44:

ERROR: make_jp2 command failed: ['/usr/bin/convert', '-quiet', '-format', '-jp2', '-define', 'jp2:rate=10', '/tmp/sim_180_000_001_0049.tif[0]', '/tmp/sim_180_000_001_0049.jp2']
returncode was: 1
output was: 
convert: Failed to seek in the stream.
`OpenJP2' @ error/jp2.c/JP2ErrorHandler/194.
convert: unable to encode image file `/tmp/sim_180_000_001_0049.jp2' @ error/jp2.c/WriteJP2Image/1141.
convert: unable to write file `/tmp/sim_180_000_001_0049.jp2' @ error/constitute.c/WriteImage/1346.

Nuxeo: https://nuxeo.cdlib.org/nuxeo/nxdoc/default/87c80fac-effb-44f5-95d4-75b88a518a2f/view_documents
Calisphere-stage: https://calisphere-stage.cdlib.org/item/ark:/86086/n2cc12h3/?order=48 (child record)

`OSError: [Errno 28] No space left on device`

[x] mapped task 1: OSError: [Errno 28] No space left on device - failed on retry, made it to the same file
[x] mapped task 8: OSError: [Errno 28] No space left on device
[x] mapped task 29: OSError: [Errno 28] No space left on device
[x] mapped task 39: OSError: [Errno 28] No space left on device
[x] mapped task 40: OSError: [Errno 28] No space left on device
[x] mapped task 41: OSError: [Errno 28] No space left on device
[x] mapped task 42: OSError: [Errno 28] No space left on device
[x] mapped task 43: OSError: [Errno 28] No space left on device
[x] mapped task 45: OSError: [Errno 28] No space left on device
[x] mapped task 46: OSError: [Errno 28] No space left on device
[x] mapped task 47: OSError: [Errno 28] No space left on device
[x] mapped task 49: OSError: [Errno 28] No space left on device

barbarahui commented 1 month ago

The make_jp2 errors should be resolved with this PR: https://github.com/ucldc/rikolti/pull/1137

The No space left on device errors should be resolved with this PR: https://github.com/ucldc/rikolti/pull/1135

christinklez commented 1 month ago

This collection harvests through without harvest-stopping errors! There are several outstanding issues with the Nuxeo image content files themselves; see new issue: https://github.com/orgs/ucldc/projects/2?pane=issue&itemId=84240640

ucldc / rikolti