Import JSON with bonding box and segmentation masks

[alelordelo]

Hi, amazing app, super intuitive and polished! : )

I have a dataset with JSON files with classes, bonding boxes and segmentation masks (one png file for each mask).

It it possible to import to RectLabel? Can I import the masks as pixels? Converting to geometry seems not to seamless...

thanks!

[ryouchinsa]

Thanks for writing the issue.

Which JSON format are you using? We will update RectLabel to import the JSON file and png mask images.

[ryouchinsa]

Improved so that you can drag & drop grayscale mask images to the labels table. The new version 2023.05.29 was released.

[ryouchinsa]

Improved so that you can drag & drop the COCO RLE JSON file of the SA-1B dataset to the labels table to import mask annotations. The new version 2023.05.31 was released.

Segment Anything 1 Billion (SA-1B) is a dataset designed for training general-purpose object segmentation models from open world images. https://github.com/facebookresearch/segment-anything

[alelordelo]

@ryouchinsa , great just tested now!

Is it possible to add annotations (like a longer text description) to masks? I only see an option to add a label....

[alelordelo]

Also.., when you import the JSON, doesn't it import the labels?

[ryouchinsa]

In general, a COCO JSON file includes all images and the annotations. So that Export menu -> Import COCO JSON file is used to import.

For dragging & dropping the COCO JSON file to the labels table, currently the COCO RLE JSON format is corresponded. This format does not include label names, just include the mask images. https://github.com/facebookresearch/segment-anything

{
    "image":
    {
        "image_id": 1,
        "width": 1500,
        "height": 2060,
        "file_name": "sa_1.jpg"
    },
    "annotations": [
    {
        "bbox": [866.0, 946.0, 132.0, 199.0],
        "area": 14773,
        "segmentation":
        {
            "size": [2060, 1500],
            "counts": "TS_f15SP27K3N2iTNHWf1:bYN0Yf12cYN1\\f11mWN7SNKni11PVNS2OmMPj15aUN\\2;aMSj1m3`UNVL_j1m4N1O1O1O10000O10O10O100000000O10000O100O1O100O101O000000000O10O101N1N2N2O1O100O100\\KhUNT3Wj1jLnUNS3Tj1kLoUNR3Rj1mLTVNW3ci1hL`VNW3_i1hLdVNV3\\i1iLfVNV3Zi1jLgVNU3Yi1jLiVNV3Vi1jLlVNT3Ti1kLnVNT3Ri1lLoVNS3Qi1lLQWNS3oh1mLRWNR3nh1nLSWNQ3mh1nLVWNP3jh1PMWWNo2ih1QMXWNl2jh1TMWWNg2mh1XMUWNV1gNlN_j1NkVNS1jNkN]j12jVNQ1lNiN\\j16jVNm0mNjN[j19iVNk0nNjNZj1;iVNh0QOkNVj1=jVNe0TOjNTj1a0jVN3EXObi1e0YYNVOjf1j0\\4001O00001O00001O00001O10O01O001O1O1O1O1O2N1O1O1O1O1O1O100O101N10000O00100O1O1O100O1O1O0000lNRRNGQn17S1O2N1O101M4Mom^o0"
        },
        "predicted_iou": 0.9523417353630066,
        "point_coords": [
            [940.9375, 1034.5625]
        ],
        "crop_box": [622.0, 902.0, 567.0, 707.0],
        "id": 523353737,
        "stability_score": 0.9742233753204346
    },
    ...
}

[ryouchinsa]

To add a long description, you can use a text attribute.

We need some ideas to brush up the design when you type the long text and how to show the long text on the labels table.

[alelordelo]

In general, a COCO JSON file includes all images and the annotations. So that Export menu -> Import COCO JSON file is used to import.

For dragging & dropping the COCO JSON file to the labels table, currently the COCO RLE JSON format is corresponded. This format does not include label names, just include the mask images. https://github.com/facebookresearch/segment-anything

{
    "image":
    {
        "image_id": 1,
        "width": 1500,
        "height": 2060,
        "file_name": "sa_1.jpg"
    },
    "annotations": [
    {
        "bbox": [866.0, 946.0, 132.0, 199.0],
        "area": 14773,
        "segmentation":
        {
            "size": [2060, 1500],
            "counts": "TS_f15SP27K3N2iTNHWf1:bYN0Yf12cYN1\\f11mWN7SNKni11PVNS2OmMPj15aUN\\2;aMSj1m3`UNVL_j1m4N1O1O1O10000O10O10O100000000O10000O100O1O100O101O000000000O10O101N1N2N2O1O100O100\\KhUNT3Wj1jLnUNS3Tj1kLoUNR3Rj1mLTVNW3ci1hL`VNW3_i1hLdVNV3\\i1iLfVNV3Zi1jLgVNU3Yi1jLiVNV3Vi1jLlVNT3Ti1kLnVNT3Ri1lLoVNS3Qi1lLQWNS3oh1mLRWNR3nh1nLSWNQ3mh1nLVWNP3jh1PMWWNo2ih1QMXWNl2jh1TMWWNg2mh1XMUWNV1gNlN_j1NkVNS1jNkN]j12jVNQ1lNiN\\j16jVNm0mNjN[j19iVNk0nNjNZj1;iVNh0QOkNVj1=jVNe0TOjNTj1a0jVN3EXObi1e0YYNVOjf1j0\\4001O00001O00001O00001O10O01O001O1O1O1O1O2N1O1O1O1O1O1O100O101N10000O00100O1O1O100O1O1O0000lNRRNGQn17S1O2N1O101M4Mom^o0"
        },
        "predicted_iou": 0.9523417353630066,
        "point_coords": [
            [940.9375, 1034.5625]
        ],
        "crop_box": [622.0, 902.0, 567.0, 707.0],
        "id": 523353737,
        "stability_score": 0.9742233753204346
    },
    ...
}

Thanks, so batch import is only available for COCO RLE, not COCO? I prefer to have separate files instead of a merged one. But guess the only way then is to merge everything?

[ryouchinsa]

We will improve the "Import COCO RLE JSON files" to import the usual COCO JSON files. Each JSON file name should be the same as image file name. image.jpg -> image.json, etc.

[ryouchinsa]

• Improved so that you can drag & drop the COCO JSON file to the labels table.
• Improved so that "Import COCO JSON per image files" can import per image COCO JSON files.

The new version 2023.07.05 was released. Please let us know your opinion.

[ryouchinsa]

Currently If you need our support to fix this problem, please let us know.