jump-cellpainting / datasets

Images and other data from the JUMP Cell Painting Consortium
BSD 3-Clause "New" or "Revised" License
149 stars 13 forks source link

Provide instructions for downloading images #67

Open shntnu opened 1 year ago

shntnu commented 1 year ago

I’m pretty new to boto3 and mostly following the template in the Jupyter notebook where images are downloaded one-by-one. This mostly seems ok download speed wise if I parallelize with multiple workers, but it seems to have a nasty habit of hanging at times and needing a reset – I’m not sure if I’m being throttled trying to access files systematically like this. Let me know if I am doing something terrible that I should not be doing, and if you’d have any better guidance.

We recommend doing this:

https://github.com/jump-cellpainting/2023_Chandrasekaran_submitted#step-1-download-cell-images

ErinWeisbart commented 5 months ago

We provide more comprehensive download instructions in the Cell Painting Gallery as well. https://github.com/broadinstitute/cellpainting-gallery/blob/main/download_instructions.md