search

pulibrary / figgy

Valkyrie-based digital repository backend.
Other
36 stars 4 forks source link

PDF Derivative Generation - FileAppender: Failed to append the new file #5964

Open tpendragon opened 1 year ago

tpendragon commented 1 year ago 
FileAppender: Failed to append the new file converted_from_pdf_page_18.tiff for 330f1de1-c945-41e5-b7b8-007b10e0a18c to resource 265da7e9-e4e4-43f5-8a4a-cd91e53a883d: No such file or directory @ rb_sysopen - /tmp/pdf_derivatives20230830-20415-csde2d/intermediate_file17.tif

Backtrace

line 137 of [PROJECT_ROOT]/app/services/file_appender.rb: rescue in create_node
line 114 of [PROJECT_ROOT]/app/services/file_appender.rb: create_node
line 144 of [PROJECT_ROOT]/app/services/file_appender.rb: block in file_nodes

View full backtrace and more info at honeybadger.io

Something to do with?: https://github.com/pulibrary/figgy/blob/fb6b79492979701bb7919bedee070e7b83ef2b9a/app/derivative_services/pdf_derivative_service.rb#L90

Sudden Priority Justification

@aberdini is trying to ingest a sudden priority request on her side for 7 PDFs listed here: https://figgy.princeton.edu/?f%5Bmember_of_collection_titles_ssim%5D%5B%5D=Board+of+Trustees+Records+AC120&f%5Bstate_ssim%5D%5B%5D=pending&q=AC120 that have all failed because of this error

tpendragon commented 1 year ago

Example of a broken record: https://figgy.princeton.edu/catalog/265da7e9-e4e4-43f5-8a4a-cd91e53a883d

tpendragon commented 1 year ago

We want this to fail and retry, and maybe clean up after itself if it fails. Right now we're getting broken records that you can only tell are broken if you do manual QA.

tpendragon commented 1 year ago

Sometimes this happens during ingest and sometimes it happens for PDF derivative generation. The previous example is a PDF generation error, https://figgy.princeton.edu/catalog/c938a0dd-9af8-48f4-8d93-0c6e1d4fa807 is one where there should be a second page but it didn't get added because of a FileAppender error and now it's complete with one less page than it should have.

tpendragon commented 1 year ago

The sudden priority portion of this ticket is closed by #6054.

hackartisan commented 1 year ago

@tpendragon do you have a sense of what remains here?

tpendragon commented 1 year ago

@hackartisan I think something like "FileAppender shouldn't error quietly" - the remaining issue is ones like https://figgy.princeton.edu/catalog/c938a0dd-9af8-48f4-8d93-0c6e1d4fa807 where it errored and now is missing a file and nobody knew.