esmero / strawberryfield

A Field of strawberries
GNU Lesser General Public License v3.0
10 stars 5 forks source link

Update Hook for setting Multipart to S3 threshold. #293

Open DiegoPino opened 7 months ago

DiegoPino commented 7 months ago

What?

On the absence of this value, on the latest 1.3.0 branch, all files will be handled via the multipart uploader, which will fail for medium sized (30Mbytes or so)files in MIN.IO is mediating/routing S3 bc of a bug in MIN.IO deprecated S3 routing startup option that sends an invalid header for multipart

This leads to files staying in their original place (or temp) but still being made permanent which means basically yeah, not lost, but also not visible to your IIIF.

This issue will simply set, via an update hook the 5Gbyte setting for you, that way you don't have to go to/admin/config/archipelago/filepersisting and save that form to avoid multipart being the choice always.

I will also open another pull later that will allow, files that ended in a limbo (should not happen but files are important) to be moved a next time/ on ADO re-save.

DiegoPino commented 7 months ago

The method is here: https://github.com/esmero/ami/blob/830e6d9c255a4ac9e7e0243945f78cc3f978bcb7/src/Plugin/QueueWorker/IngestADOQueueWorker.php#L1105

Which also reminds me, that the file "identification" is present only for the current user ... wonder if we need to make sure that Hydroponics changes to the user in the queue item? So we have a global state? OR...we don't use the "temp" one and we use the global key storage shared by all users? Suggestions are welcomed