MedSAMLite1 sample data : brain segmentation quality issue

bowang-lab / MedSAM

Segment Anything in Medical Images

https://www.nature.com/articles/s41467-024-44824-z

Apache License 2.0

2.94k stars 402 forks source link

MedSAMLite1 sample data : brain segmentation quality issue #227

Closed ALdevonia closed 7 months ago

ALdevonia commented 7 months ago

Hello,

First, I would like to thank you for sharing your work with us!

I have already opened this issue here : https://github.com/bowang-lab/MedSAMSlicer/issues/21 But maybe I should have opened it here (sorry for the mess).

So the issue:

I am using Lite-MedSAM and the MedSAM-Lite 3D Slicer Plugin to segment the brain (only the cerebral hemispheres) in the MedSAMLite1 sample data, but the result is not really satisfying as you can see in the screenshot. I applied the preprocess for MR Images here, but I cannot get any usable results with or without selecting preprocessing options.

So maybe:

My ROI box is not drawn as it should be to reach a good result (I have tried several drawings though)
I should preprocess differently the data
The MedSAMLite1 sample data is not ideal for brain segmentation
It might be out of the scope of the model
Something else (?)

3D Slicer version: 5.6.1 MedSAMSlicer version: 0.05

Capture d'écran 2024-03-21 135855

JunMa11 commented 7 months ago

Hi @AlDevonIA ,

Thanks for raising this interesting issue.

Could please show me an example of what the expected segmentation mask is?

ALdevonia commented 7 months ago

Hi,

The first screenshot below shows what kind of hemisphere segmentation mask I expect. It is a 3D represention from the website brainfacts.org, so not an actual segmentation mask (sorry).
The second screenshot shows a 2D hemisphere segmentation mask I've obtained by using the app available here: https://huggingface.co/spaces/linfanluntan/Grounded-SAM This is close to what I expect. I've obtained this mask not by drawing a ROI box though, but by typing the word "brain" as text prompt, since the app uses the Grounded-SAM project work whose repository can be found here: https://github.com/IDEA-Research/Grounded-Segment-Anything/tree/main

Thanks again for your time and your work!

1st screenshot: Capture d'écran 2024-03-28 182752

2nd screenshot: tmpdzpef5oa

JunMa11 commented 7 months ago

Hi @AlDevonIA ,

Honestly, MedSAM is not suitable for this task. We excluded these kinds of images from the training set because providing such a large bounding box would introduce the ambiguity, e.g., the target could be the whole brain, white matter, or grey matter.

Maybe this brain segmentation tool fits your purposes. https://www.pnas.org/doi/full/10.1073/pnas.2216399120

ALdevonia commented 7 months ago

Hi @JunMa11 ,

Thank you very much for the answer and the suggestion!