What's the training clicking strategy?

[sdfghjkyuio]

In paper Interactive image segmentation via backpropagating refinement scheme, the user-annotations are imitated through a clustering strategy when they trained on the sbd dataset. I'm wondering if you have applied the same way to generate clicks when training?

[ksofiyuk]

We simplify points sampling strategy in our experiments. We observed that if points are just sampled randomly, there is no difference in performance compared to more complicated sampling with points clustering. You can check our training sampling strategy in this class.

[sdfghjkyuio]

I guess it's in the points_sampler.py🤔

[sdfghjkyuio]

Omg we just commented at the same time! Thank you🤩

[ksofiyuk]

Yeah, it's a funny coincidence :smiley:. Thank you for your interest in our work.

[sdfghjkyuio]

I just met another problem😫: After I trained for one epoch on my own dataset, I checked the instance segmentation images in the experiment folder and found that some positive points (in green) appeared in the background area, which led to an error in predicting. How can I correct it? Thank you soooo much!

[ptrvilya]

Could you please share a sample from visualization folder with erroneous point placement?

[sdfghjkyuio]

Like here, the blue points (background points) appear on the person's face (which is the foreground). So if I hope the network to learn to segment just one specific type from the image, I'm not sure if it can learn the characteristics of this type so well due to the wrong clicks during the training.

[ptrvilya]

It seems to me that these points lie near the border of the ground-truth mask (on the background part of it). I guess these points appear erroneous due to errors in mask.

[ksofiyuk]

I combined your mask with the image and Ilia seems to be right. There are no any mistakes.

[sdfghjkyuio]

Yes, the mask can match with the image based on the green and blue points. But I am confused whether the blue points are at the right location. I mean it seems that not all the blue points are sampled from the background pixels.

[ksofiyuk]

There are no mistakes with points sampling. If you want train a good interactive segmentation model, you need to train it on a dataset with nearly perfect segmentation masks. It seems that your dataset has some very coarse masks.

[sdfghjkyuio]

Actually this image is one example from the sbd dataset...Thank you for your advice, I will check the masks with wrong sampled points.

[ksofiyuk]

Yeah, we know, SBD is not good dataset. We use it only for academic research to compare with previous works.

[sdfghjkyuio]

Thank you so much for answering my questions!