search

fractal-analytics-platform / fractal-tasks-core

Main tasks for the Fractal analytics platform
https://fractal-analytics-platform.github.io/fractal-tasks-core/
BSD 3-Clause "New" or "Revised" License
11 stars 5 forks source link

Make Cellpose task work for OME-Zarrs without pyramids #742

Open BioinfoTongLI opened 1 month ago

BioinfoTongLI commented 1 month ago

Hi there,

I've managed to run

  1. import_ome_zarr
  2. drop_t

But not the cellpose_seg. The script was asking for coarsening_xy which is not present in my metadata file.

  2024-05-18 10:05:10,048; INFO; NGFF image has num_levels=1
  2024-05-18 10:05:10,048; INFO; NGFF image has coarsening_xy=None
......
    File "/opt/conda/lib/python3.11/site-packages/fractal_tasks_core/roi/v1.py", line 255, in convert_ROI_table_to_indices
      prefactor = coarsening_xy**level
                  ~~~~~~~~~~~~~^^~~~~~
  TypeError: unsupported operand type(s) for ** or pow(): 'NoneType' and 'int'

I've tried to find the definition of this, but couldn't find it. I assume it's for helping defined the cell size parameter for cellpose? Just wondering what is the proper way adding it to all the wells.

jluethi commented 1 month ago

Hey Tong,

Hmm, interesting. coarsening_xy is the downsampling factor between pyramid levels. It's computed on the fly based on the actual OME-Zarr, so shouldn't be something required in the metadata anymore.

My only suspicion of how this can occur: Do your images not have any pyramid levels? If so, is it possible to add one? It would also be valuable to make our tasks robust against this of course, we just haven't had any test cases for this so far :)

If they do have pyramid levels, can you share a small example OME-Zarr that has this issue?

What I'd debug there:

2024-05-18 10:05:10,048; INFO; NGFF image has coarsening_xy=None

=> the coarsening_xy wasn't calculated.

I'd check what the load_NgffImageMeta function returns for that specific Zarr, see: https://github.com/fractal-analytics-platform/fractal-tasks-core/blob/650e0a373fd8665f56d8b8e629139d7dae908385/fractal_tasks_core/ngff/zarr_utils.py#L26

BioinfoTongLI commented 1 month ago

oh, got it! I specifically asked to not generate pyramids to reduce the large number of file issue. I've regenerated the pyramid and it seems to be working. Thanks!

jluethi commented 1 month ago

ok, so the issue is indeed that our Cellpose task (& potentially other tasks) currently don't work for images without pyramids. We should drop the assumption that pyramids are present where-ever we can. Not sure when we'll get to it, but I'll take this up as a maintenance issue :)

BioinfoTongLI commented 1 month ago

I actually don't think it's a bad idea to make that assumption. Not all people wants analysis to be performed on the raw level afterall. But this will help reducing the number of files, which is the big blocker for us.