expand deletion step - Githubissues

wellcomecollection / goobi-infrastructure

Wellcome Collection digital workflow infrastructure

MIT License

0 stars 3 forks source link

expand deletion step #416

Closed mgeerdsen closed 8 months ago

mgeerdsen commented 3 years ago

We should expand the existing deletion step, to also remove the following:

the source directory/prefix of a process which contains the downloaded IA files
the contents of the ocr prefix (which contains OCR files like ALTO files for IA processes or from older OCR processing)

We should also discuss if this should only happen for harvested material (19c/MOH) or for all processes.

aray-wellcome commented 3 years ago

Christy is checking on some old emails to see if there was a decision made long ago about saving jpgs for archival items or not (because we have January openings for archival items so it's easier to do them if the jpgs are still there). But the IA files seem like the best place to start so far.

mgeerdsen commented 1 year ago

I am not totally sure if we have adapted the deletion step in the mean time, we should check.

mgeerdsen commented 1 year ago

Robert checked and the deletion step does delete the source prefix and the ocr prefix as mentioned above.

The JP2_Auto_Edit_METS workflow appears to have jpeg deletion disabled, I am not sure if this should stay this way. @aray-wellcome ?

aray-wellcome commented 1 year ago

I remember shutting off jpg creation for the auto edit mets workflow because humans don't look at Edit METs and it was wasting resources...I think we can keep it off.