search

dchaley / deepcell-imaging

Tools & guidance to scale DeepCell imaging on Google Cloud Batch
8 stars 2 forks source link

Test a 451M pixel image #154

Open dchaley opened 6 months ago

dchaley commented 6 months ago

Prepare image for DeepCell: https://portal.hubmapconsortium.org/browse/dataset/f998d229419480586317be5866079c2f

dchaley commented 6 months ago

Uploaded OMETIFF to cloud storage: gs://davids-genomics-data-public/cellular-segmentation/hubmap/hbm458.nffx.729_451Mpx/VAN0048-LK-1-32-preAF-registered.ome.tiff

lynnlangit commented 6 months ago

can't wait to hear how it goes...

dchaley commented 6 months ago

We're using channel 0 (DAPI) as the nucleus channel:

Screenshot 2024-02-23 at 12 54 33 PM

We're using channel 1 (eGFP) as the membrane channel: Screenshot 2024-02-23 at 12 55 11 PM

dchaley commented 6 months ago

Data generated! Output path: gs://davids-genomics-data-public/cellular-segmentation/hubmap/hbm458.nffx.729_451Mpx/input_channels.npz

See sample documentation from this PR: https://github.com/dchaley/deepcell-imaging/pull/162

Cannot benchmark now since our highmem-96 instance is using all the CPUs! 😅

lynnlangit commented 4 months ago

what is the next step here? try to split and run on Batch? something else?

dchaley commented 4 months ago

We ended up taking a break on this one as we got even bigger ones. The next steps here are to prep it for processing: extract channels from tiff, get numpy arrays, etc.

langitlynn commented 1 month ago

where is this work at presently? seems to be a key use case for the client...

dchaley commented 1 month ago

We went straight to larger images actually. We still could benefit from converting this TIFF to numpy but we got the main results (sizing for running 1B pixels)