Add benchmark for serial smart_open vs parallel gs_fastcopy transfers

dchaley / deepcell-imaging

Tools & guidance to scale DeepCell imaging on Google Cloud Batch

8 stars 2 forks source link

Add benchmark for serial smart_open vs parallel gs_fastcopy transfers #287

Closed dchaley closed 1 month ago

dchaley commented 1 month ago

Write up a quick benchmark script to gather data on downloading & uploading using smart_open (serial) and gs_fastcopy (parallel) read/write implementations.

dchaley commented 1 month ago

All done here 🎉