RLE in Segementation counts

[import-jalashwa]

Hi,

Thanks for your amazing work on MASA !

I am trying to train MASA on my custom dataset. I have generated masks from SAM as instructed in the documentation for my custom images. While I am able to get the details for the annotations bbox, area I am slightly confused about: annotation['segmentation']. From a sample of the SA-1B dataset annotation file, i gather that annotation['segmentation'] should have size (basically the size of the image) and counts. I gather that the counts is the segmentation mask in the coco RLE format. Is it absolutely necessary for me to have my annotation['segmentation'] in the RLE format or can I use some other format as well ? If RLE is necessary, could you point me to tools that I can use to generate my masks in the RLE format

@siyuanliii

[siyuanliii]

Thanks for your question! If you want to train MASA on your custom image, and if you do not use advance augmentation like copy and paste, bbox is already enough. If run the amg script from SAM, annotation['segmentation'] should already contain RLE format annotations? What annotations format for the mask do you have, bit mask?

[import-jalashwa]

Thanks for getting back to me ! You're right, annotation["segmentation"] is indeed in the bit mask format

Here's a sample for your kind reference: {'segmentation': array([[False, False, False, ..., False, False, False], [False, False, False, ..., False, False, False], [False, False, False, ..., False, False, False], ..., [False, False, False, ..., False, False, False], [False, False, False, ..., False, False, False], [False, False, False, ..., False, False, False]]), 'area': 295938, 'bbox': [812, 466, 1107, 611], 'predicted_iou': 1.0232726335525513, 'point_coords': [[1410.0, 826.875]], 'stability_score': 0.9841829538345337, 'crop_box': [0, 0, 1920, 1080]}

Assuming i do not perform any advance augmentation, can convert_sam_2_cocofmt.py handle the bit mask as it is so that I can perform a basic training on custom dataset ?

[siyuanliii]

If you do not need any advance augmentation, then you can just run convert_sam_2_cocofmt.py and use the generated sa1b_coco_fmt_500k_bbox_anno.json to train your model

[siyuanliii]

I also attached an example of how to convert bit mask to RLE using pycocotools

import numpy as np
from pycocotools import mask as coco_mask

def mask_to_rle_coco(mask):
    """
    Converts a binary mask to COCO-style RLE encoding.

    Parameters:
    mask (np.ndarray): 2D binary mask (numpy array) where 1 represents the object and 0 represents the background.

    Returns:
    dict: RLE encoded mask in COCO format.
    """
    # Ensure mask is in uint8 format
    mask = mask.astype(np.uint8)

    # Encode mask using pycocotools
    rle = coco_mask.encode(np.asfortranarray(mask))

    return rle

# Example usage:
# Create a binary mask (example)
mask = np.array([
    [0, 0, 1, 1],
    [1, 1, 1, 0],
    [0, 0, 0, 0]
], dtype=np.uint8)

# Convert mask to RLE format
rle_encoding = mask_to_rle_coco(mask)
print("COCO-style RLE Encoding:", rle_encoding)