search

halleewong / ScribblePrompt

[ECCV 2024] ScribblePrompt: Fast and Flexible Interactive Segmentation for Any Medical Image
http://scribbleprompt.csail.mit.edu/
Apache License 2.0
147 stars 14 forks source link

How were the synthetic scribbles generated during training? #8

Open aL3x-O-o-Hung opened 2 months ago

aL3x-O-o-Hung commented 2 months ago

So I generated some fake scribbles on our dataset using the ground truth labels but the pretrained model cannot predict anything meaningful while using the generate clicks and bounding boxes are fine. I am wondering how the scribbles are generated and how the generated scribbles can shift the input distribution so much that it completely confuse the network.

halleewong commented 2 months ago

Can you show some examples?

The scribbles were generated using the code in scribbleprompt/scribbles.py during training.

aL3x-O-o-Hung commented 2 months ago

Hi!

Thanks for the reply. Please find these examples attached.

microUS_test_01_positive_slice_11 microUS_test_01_positive_slice_16

halleewong commented 2 months ago

Those scribbles look quite thick, relative to the size of the images. The random deformations in our scribble generation code vary the scribble thickness a bit but the training scribbles are rarely as wide and as sharp as these examples. I suspect the predictions will be much better with thinner scribbles and/or lightly blurring the scribble mask.