SSC

[lk0429]

“Given that the predicted segments may overlap, we sort them based on area and prioritize smaller segments. Specifically, when a pixel belongs to multiple segments, we select the segment with the smallest area.” Where is the code for this part of the idea? Can you tell me in detail? Thank you! In addition, why are we considering the smallest segments in order of area? Why not give priority to the segments with larger areas?

[sangrockEG]

Thanks for your interest.

In fact, the mentioned process is not included in this repository. Sorry for the inconvenience. But we believe that it is quite straightforward to follow (just performing segment-everything on every training images). Please refer to automatic_mask_generator_example.ipynb of the official SAM repository.

And the idea behind giving priority to the small segments is based on the characteiristics of SAM. The masks obtained by segment-everything are not mutually exclusive. In other words, the masks can be overlapped. Moreover, small masks are often entirely covered by the larger masks. To avoid this, we sort the masks by its area and prioritize the smaller masks. This enables the smaller masks to survive.

[lk0429]

So the masks (segments) you get from SAM's SamAutomaticMaskGenerator are in order from small to large area?

[sangrockEG]

I'm not sure about the default order of SAM is small-to-large or not. Probably it's large-to-small but a bit noisy. But you can easily sort it. For our process, you should sort it to large-to-small order.

And to get the se map from the sorted list of masks, you can aggregate them into a one single mask, just like follows:

se_map = zeros_like(masks[0]) for i, mask in enumerate(masks): se_map[mask] = i+1

Here, large-to-small order sorting for the masks indeed prioritizes smaller mask. Because in the above for loop, when a pixel is shared by two masks, then it would be assigned as smaller mask.

[lk0429]

In your get_se_map.py code: " for i, mask in enumerate(reversed(masks)): temp[mask['segmentation']] = i " reversed(masks) assigns serial numbers to masks in reverse order. Are masks here sorted by area? If not, what rules do you use to assign the mask index to the se map? Ultimately, I really want to know if each mask is sorted by area in the se map result obtained from '/se/default/'? Is your preference for masks with small areas reflected in the above code? If not, please point it out. Thank you very much for your patience in answering my questions.

If I give priority to some masks that contain the whole object, won't it make the activation more complete?

[lk0429]

I have found that the masks in ‘./se/default/’ are not strictly in order of area size, while ‘predicted_iou’ seems to be in order from large to small.

[sangrockEG]

Sorry for the delayed response. I have not tried to give priority to some masks that contain the whole object, but it may lead to better performance.