RLE -> Mask for semantic segmentation

[vkovac2]

Hello. I have a problem with loading the mask from RLE. So, I got the json file from labelstudio that had the rle format. I tried decoding it with all the open source codes and nothing works. I also tried using this :https://github.com/thi-ng/umbrella , but unfortunately it didnt work.

After decoding and exporting it, I tried loading it in python. The decoded result is of length 22251060, while my image is 2313 x 2405. The shapes do not match, so I cannot make it as a mask.

What is the correct way of changing the RLE to mask?

Thanks a lot

[vkovac2]

I used brush for semantic segmentation btw.

[zakajd]

I'm not a project developer, but here is a Python function which can be used to decode RLEs:

import numpy as np

def rle_decode(rle_str, shape, fill_value=1, dtype=int, relative=False):
    """
    Args:
        rle_str (str): rle string
        shape (Tuple[int, int]): shape of the output mask
        relative: if True, rle_str is relative encoded string
    """
    s = rle_str.strip().split(" ")
    starts, lengths = np.array([np.asarray(x, dtype=int) for x in (s[0:][::2], s[1:][::2])])
    mask = np.zeros(np.prod(shape), dtype=dtype)
    if relative:
        start = 0
        for index, length in zip(starts, lengths):
            start = start + index
            end = start + length
            mask[start:end] = fill_value
            start = end
        return mask.reshape(shape[::-1]).T
    else:
        starts -= 1
        ends = starts + lengths
        for lo, hi in zip(starts, ends):
            mask[lo:hi] = fill_value
        return mask.reshape(shape[::-1]).T

[vkovac2]

Dear @zakajd,

Thanks for replying. This function does not work for odd number length of RLE. Even if the length of RLE is an even number, although it runs, it does not give the correct output. I have tried using the JavaScript library that they provide for encoding-decoding, however even when i decode that RLE it seems extremely wrong..wrong size of dimensions. Seems that label studio uses a custom rle encoding.

Sincerely, Veljko KOVAC

[makseq]

@vkovac2 I've just released the brush converter to numpy & png formats. Please, check this PR from label-studio-converter: https://github.com/heartexlabs/label-studio-converter/pull/9

1 Clone this repo git@github.com:heartexlabs/label-studio-converter.git

2 cd label-studio-converter

3 activate PR branch: git checkout brush-converter

3 activate your virtualenv if you use it source <env>/bin/activate

4 install PR as package pip install -e .

5 start your label-studio label-studio start <my_project>

6 go to Export page in browser and check BRUSH_TO_NUMPY and BRUSH_TO_PNG options.

If you need more customized export then dive into brush.py: https://github.com/heartexlabs/label-studio-converter/pull/9/commits/8eb02c4bc2c4cf99c84c308868dc186ab61490b2#diff-8c13b5aae0f94c3f59fe7a98aca8501e

[vkovac2]

Worked for me. Added a comment on the code that i need to put in order to work. Thanks for adding this function.

[makseq]

Now you can do

pip uninstall label-studio
pip install label-studio==0.7.4

It contains fixed brushes with correct resize behavior, correct multiple brush results in one completion fix and export to PNG/Numpy. It will be appreciate if you check this release candidate version.

[faizanengineer]

I'm trying to decode my RLE string to draw a mask on the image, but it always draws a vertical line at the top right corner. I used the code below, but it seems like it was unable to decode the RLE correctly

RLE String in Json File = '{"rle":[121256,0,3,1,715,0,6,1,712,0,8,1,712,0,8,1,712,0,9,1,711,0,12,1,708,0,19,1,702,0,21,1,699,0,23,1,698,0,22,1,699,0,21,1,700,0,20,1,704,0,15,1,675,0,5,1,714,0,6,1,705,0,4,1,4,0,8,1,703,0,18,1,701,0,20,1,699,0,22,1,697,0,24,1,696,0,25,1,694,0,26,1,693,0,27,1,693,0,28,1,692,0,28,1,692,0,29,1,691,0,29,1,690,0,30,1,691,0,30,1,690,0,30,1,690,0,30,1,690,0,30,1,690,0,30,1,690,0,30,1,690,0,31,1,689,0,31,1,689,0,31,1,689,0,30,1,691,0,29,1,691,0,29,1,691,0,28,1,692,0,28,1,693,0,27,1,693,0,27,1,694,0,26,1,695,0,25,1,696,0,23,1,698,0,22,1,700,0,20,1,701,0,19,1,702,0,18,1,704,0,16,1,705,0,15,1,707,0,13,1,708,0,12,1,710,0,10,1,711,0,9,1,713,0,7,1,714,0,6,1,355398,0]}'

import json
import` numpy as np
import cv2

def rle_decode(rle_list, shape, fill_value=1, dtype=int, relative=False):
    rle_str = ' '.join(map(str, rle_list))
    s = rle_str.strip().split(" ")
    starts, lengths = np.array([np.asarray(x, dtype=int) for x in (s[0:][::2], s[1:][::2])])
    mask = np.zeros(np.prod(shape), dtype=dtype)
    if relative:
        start = 0
        for index, length in zip(starts, lengths):
            start = start + index
            end = start + length
            mask[start:end] = fill_value
            start = end
        return mask.reshape(shape[::-1]).T
    else:
        starts -= 1
        ends = starts + lengths
        for lo, hi in zip(starts, ends):
            mask[lo:hi] = fill_value
        return mask.reshape(shape[::-1]).T

with open('downloads/labels/2D82C4F8-F1F2-4042-AE45-C4988A55048A.json', 'r') as f:
    data = json.load(f)

rle_list = data['rle']
shape = (100, 100)  # Assuming the shape is known or provided

mask = rle_decode(rle_list, shape)

image = np.zeros(shape, dtype=np.uint8)

image[mask == 1] = 255  # White color for mask

cv2.imshow('Mask', image)
cv2.waitKey(0)
cv2.destroyAllWindows()

[shantanudev]

@faizanengineer Were you able to figure this out?

Update: nvm, I figured it out.

def rle2img(rle, height, width): image=brush.decode_rle(rle) decoded_rle_img=np.reshape(image, [height, width, 4])[:, :, 3] return decoded_rle_img

for task in tqdm(all_labels): annotation=project.get_task(task) brush_results=annotation['annotations'][0]['result'] masked_image=None for ind, b in enumerate(brush_results): rle=b['value']['rle'] height=b['original_height'] width=b['original_width'] if masked_image is None: masked_image=rle2img(rle,height,width) else: masked_image+=rle2img(rle,height,width) np.save(os.path.join('./mask_output',os.path.basename(annotation['data']['image']).split('.jpg')[0]+'.npy'),masked_image)

This was roughly my code logic. The documentation is pretty lacking here but this worked for my needs to decode the annotation back to the image. Hope this saves anyone else some annoying rabbit hole.

[faizanengineer]

Please try this out , hope it works for you

import os
import numpy as np
from tqdm import tqdm

def rle2img(rle, height, width):
    # Decode RLE to a binary mask
    decoded_mask = decode_rle(rle, height, width)
    return decoded_mask

# Assuming `all_labels` is a list of tasks and `project.get_task` is defined
for task in tqdm(all_labels):
    annotation = project.get_task(task)
    brush_results = annotation['annotations'][0]['result']
    masked_image = None

    for ind, b in enumerate(brush_results):
        rle = b['value']['rle']
        height = b['original_height']
        width = b['original_width']

        # Process each RLE mask
        current_mask = rle2img(rle, height, width)

        # Aggregate masks
        if masked_image is None:
            masked_image = current_mask
        else:
            masked_image += current_mask

    # Save the aggregated mask
    output_path = os.path.join('./mask_output', os.path.basename(annotation['data']['image']).split('.jpg')[0] + '.npy')
    np.save(output_path, masked_image)

[faizanengineer]

@shantanudev Please see the message above, I forgot to mention you