search

MouseLand / cellpose

a generalist algorithm for cellular segmentation with human-in-the-loop capabilities
https://www.cellpose.org/
BSD 3-Clause "New" or "Revised" License
1.24k stars 359 forks source link

[FEATURE] Package Cellpose models for BioImage.IO model zoo #956

Open qin-yu opened 3 weeks ago

qin-yu commented 3 weeks ago

Is your feature request related to a problem? Please describe.

Hi @carsen-stringer!

I propose adding an export_bioimageio() function/method in Cellpose to facilitate packaging Cellpose models for the BioImage.IO model zoo. This feature could greatly benefit the community by simplifying the integration of Cellpose models into BioImage.IO.

On the BioImage.IO side, Cellpose post-processing has not been finalized yet, but I've developed scripts for saving and packaging Cellpose models into the BioImage.IO format (spec-bioimage-io). I wanted to submit this issue before moving on to other tasks to see if you think it would be useful.

Describe the solution you'd like

Plan A: Implement a function/method for one-click packaging of Cellpose models into the BioImage.IO format.

  1. Add the function to cellpose/utils.py or create a new module.
  2. Add bioimageio_deps = ["bioimageio.spec"] to setup.py to handle dependencies.
  3. Update the documentation to include this new feature.

Describe alternatives you've considered

Plan B: Describe the packaging steps in the documentation only, without implementing a dedicated function/method.

carsen-stringer commented 3 weeks ago

This sounds great thanks! It would be great if it was easy for the community to host their Cellpose 2 fine-tuned models there! I'm not sure if I fully understand the packaging issue -- do you mean for users to add metadata? Our models are 26MB of pytorch weights -- that's all the user needs to run the model. I see for example I can download the stardist tensorflow weights from bioimage zoo, even though they also have non-tensorflow post-processing steps. Or maybe I misunderstand what you mean by post-processing.

qin-yu commented 3 weeks ago

This sounds great thanks! It would be great if it was easy for the community to host their Cellpose 2 fine-tuned models there!

Great! Your understanding is correct. The main goal here is to simplify the process for users to package and share their models with the necessary metadata included, making existing models easily discoverable and accessible to the community.

I'll start working on this and may reach out to you for some design decisions!

I'm not sure if I fully understand the packaging issue -- do you mean for users to add metadata? Our models are 26MB of pytorch weights -- that's all the user needs to run the model. I see for example I can download the stardist tensorflow weights from bioimage zoo, even though they also have non-tensorflow post-processing steps. Or maybe I misunderstand what you mean by post-processing.

This is a different matter, but your understanding is absolutely correct again. What I was saying is that while BioImage.IO allows users to run networks directly, spatial gradients from CPnet are not "post-processed" there (though it could happen, I'm not pushing for this). The immediate benefit lies in hosting community Cellpose models for downloading and running within Cellpose itself (which sounds great already!).