search

wellcomecollection / goobi-infrastructure

Wellcome Collection digital workflow infrastructure
MIT License
0 stars 3 forks source link

bagit creation: investigate multi-threaded zip generation #430

Open mgeerdsen opened 2 years ago

mgeerdsen commented 2 years ago

Currently the use of only one CPU core appears to be a bottle neck, we should look into this a bit more and see if or how it can be improved.

mgeerdsen commented 1 year ago

I think this would still be nice to have. Since we use autoscaling for the bagit workernodes, this is probably not a high priority. It might be worth estimating the needed time though.