search

kreshuklab / plant-seg

A tool for cell instance aware segmentation in densely packed 3D volumetric images
https://kreshuklab.github.io/plant-seg/
MIT License
88 stars 31 forks source link

Allow direct search and download from BioImage Model Zoo #210

Open qin-yu opened 5 months ago

qin-yu commented 5 months ago

Hey @oeway are you coming to EMBL on 11 March? I'll pass by and consult you the best way of allowing users to use models from Zoo in PlantSeg.

oeway commented 5 months ago

Unfortunately I won't be able to join, however Weize @Nanguage will join, so please discuss with him.

Not sure what you mean exactly, but maybe you can create a bioengine app for plantseg, basically a tailored version of https://github.com/bioimage-io/bioengine-web-client

Nanguage commented 5 months ago

Hi qin-yu, I will participate in the Hackathon. We can communicate face-to-face then!

qin-yu commented 5 months ago

You are right, @oeway, as of now, there's only a vague idea rather than a concrete plan. I was trying to explore what options are currently available for users to access or search for models in the zoo via Napari and CLI.

I'm looking forward to meeting you next week, @Nanguage.

qin-yu commented 5 months ago

Current PlantSeg model zoo

PlantSeg has a YAML file with hard-coded records of models including the link to weight files on Zenodo.

Adding BioImage.IO model zoo: Method 1

Since all models from PlantSeg are uploaded to BioImage.IO and Zenodo, we can simply update the list of models in the PlantSeg model zoo at start-up.

  1. Discover all compatible models
  2. Avoid duplicate in the local record (by removing current hard-coded record completely?)
  3. Separate the dropdown list by a horizontal line and ideally a section name

Adding BioImage.IO model zoo: Method 2

To minimise the changes, add a widget to let users add a model from BioImage.IO model zoo just like adding a custom model.

  1. Discover all compatible models
  2. Show models in that widget for users to add
qin-yu commented 5 months ago

Adding BioImage.IO model zoo: Method 3

Why bother running inference on our own code and filter BioImage.IO model zoo? Just use bioimageio.core and use all models available on the zoo : )

qin-yu commented 5 months ago

My Plan

Maybe I don't have to directly change the core of PlantSeg inference to bioimageio.core.

  1. Add compatible BioImage.IO model zoo records in PlantSeg on startup
  2. Replace PlantSeg hard-coded records by 1
  3. Add bioimageio.core as alternative inference engine
  4. Replace PlantSeg inference by 3
qin-yu commented 5 months ago

Potential Issues

  1. If PlantSeg wants to go generic in bioimage community, using bioimageio.core would greatly benefit PlantSeg; otherwise, PlantSeg loses the generalist capability of growing into segment-everything tool