instance mask label missing

[RhoHeeSeon]

Hello, Thank you for your great work !

I'm trying to make annotations.json for MetaGraspNet. However, I found that the simulation dataset does not have the labels 'instances_objects', 'instances_semantic', 'occlusion', 'occlusion_objects', and 'seg_masks_single' in the npz file. I looked to see if the instance segmentation mask information was in the hdf5 file, but it wasn't there either. Also I can not find the label 'bbox' of real dataset.

I downloaded the dataset from this path that you linked. Real, Sim How can I get instance mask labels of sim data and bbox labels of real data?

This is my result.

Thanks in advance !

[maximiliangilles]

Hi @RhoHeeSeon,

Q1: I just checked scene0. It seems to me like you are trying to open the "wrong" file. If you are interested in the labels you mentioned in your question., try to open the file "0.npz" - everything you need should be there. It is also a good idea, to run the visualized_2d.py script to visualize the ground truth.

Q2: With regard to your second question: BBox data can be obtained from visible segmentation masks, here is a code snippet.

def mask_to_bbox(instances_objects):

    Code from here https://discuss.pytorch.org/t/extracting-bounding-box-coordinates-from-mask/61179/4
    Input:
    mask: [height, width, num_instances]. Mask pixels are either 1 or 0.
    Output:
    Returns: bbox array [num_instances, (y1, x1, y2, x2)].
```
num_objects = len(np.unique(instances_objects)) - 1
masks = np.zeros((num_objects, instances_objects.shape[0], instances_objects.shape[1]))
for i in range(num_objects):
    masks[i] = instances_objects == (i+1)
masks = np.moveaxis(masks, 0, -1)

boxes = np.zeros([masks.shape[-1], 4], dtype=np.int32)
for i in range(masks.shape[-1]):
    m = masks[:, :, i]
    # Bounding box.
    horizontal_indicies = np.where(np.any(m, axis=0))[0]
    vertical_indicies = np.where(np.any(m, axis=1))[0]
    if horizontal_indicies.shape[0]:
        x1, x2 = horizontal_indicies[[0, -1]]
        y1, y2 = vertical_indicies[[0, -1]]
        # x2 and y2 should not be part of the box. Increment by 1.
        x2 += 1
        y2 += 1
    else:
        # No mask for this instance. Might happen due to
        # resizing or cropping. Set bbox to zeros
        x1, x2, y1, y2 = 0, 0, 0, 0
    boxes[i] = np.array([y1, x1, y2, x2])
return boxes.astype(np.int32)

Best,

Maximilian

[RhoHeeSeon]

I was able to solve the problem. Thank you so much for your fast help. :)

Best regards, Heeseon Rho