confused about rect labels

[cww97]

Hi, should be (score, width, height, depth, rotation, translation, object_id), where object id is set to -1 by default(not used)

Originally posted by @Fang-Haoshu in https://github.com/graspnet/graspnetAPI/issues/9#issuecomment-755859498

[cww97]

>>> f = '/home/weiwen/mnt0/mail2020/graspDL/datasets/graspNet-1b/scenes/scene_0000/kinect/rect/0000.npy'
>>> rect = np.load(f)
>>> rect.shape
(7645, 7)
>>> rect[0]
array([4.2113794e+02, 5.0137921e+02, 3.6851642e+02, 4.7148450e+02,
       2.6994640e+01, 5.0000000e-01, 1.4000000e+01], dtype=float32)
>>> rect[1]
array([421.13794 , 501.3792  , 366.85837 , 470.54257 ,  26.995132,
         0.8     ,  14.      ], dtype=float32)

the last object_id is not -1

[cww97]

however, I ran predictRGD.py at here

I get 255 '*.npz' files for each scene, clearly these are the predicted labels for each RGD image. but the rectagle labels in graspnet dataset seems ...are they share the same format ?

XuSheng told me his rect label format is (x1, y1, x2, y2, height, score, object_id)

[cww97]

I also wanna kown the 6d-labels format

>>> rect_graspnet = np.load('graspnet_dataset/grasp_label/000_labels.npz')
>>> rect_graspnet.files
['points', 'offsets', 'collision', 'scores']
>>> rect_graspnet.points
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'NpzFile' object has no attribute 'points'
>>> rect_graspnet['points']
array([[-0.0122905 ,  0.07282408,  0.10049217],
       [ 0.02331672,  0.07790881,  0.01449282],
       [ 0.027253  ,  0.07622192,  0.09785532],
       ...,
       [ 0.031894  ,  0.01771812,  0.07439509],
       [-0.02498909, -0.05372961, -0.11060694],
       [ 0.01005789,  0.07523085,  0.07048508]], dtype=float32)
>>> rect_graspnet['points'].shape
(3459, 3)
>>> rect_graspnet.files
['points', 'offsets', 'collision', 'scores']
>>> rect_graspnet['offsets'].shape
(3459, 300, 12, 4, 3)
>>> rect_graspnet['collision'].shape
(3459, 300, 12, 4)
>>> rect_graspnet['scores'].shape
(3459, 300, 12, 4)

[GouMinghao]

Please refer to the newly written docs: https://graspnetapi.readthedocs.io/en/latest/grasp_format.html

[cww97]

to compare

in this repo, I ran with your pretrained weights,

>>> a = np.load('grasp_multiObject_multiGrasp/predicted_rectangle_grasp/scene_0100/kinect/0000.npy')
>>> a.shape
(1556, 7)
>>> a[0]
array([ 4.86425720e+02,  4.92709625e+02,  4.86425720e+02,  4.51794342e+02,
        2.75182495e+01,  7.89444968e-02, -1.00000000e+00])

in graspnet, I download your rect labels

>>> b = np.load('graspnet_dataset/rect_labels/scene_0000/kinect/0000.npy')
>>> b.shape
(7645, 7)
>>> b[0]
array([4.2113794e+02, 5.0137921e+02, 3.6851642e+02, 4.7148450e+02,
       2.6994640e+01, 5.0000000e-01, 1.4000000e+01], dtype=float32)
>>> b[1]
array([421.13794 , 501.3792  , 366.85837 , 470.54257 ,  26.995132,
         0.8     ,  14.      ], dtype=float32)

a.shape[0] == 1556 but b.shape[0]==7645, is this because of the rect labels in graspnets dataset are generated from 6d format. so the number is larger a lot than a

[Fang-Haoshu]

>>> f = '/home/weiwen/mnt0/mail2020/graspDL/datasets/graspNet-1b/scenes/scene_0000/kinect/rect/0000.npy'
>>> rect = np.load(f)
>>> rect.shape
(7645, 7)
>>> rect[0]
array([4.2113794e+02, 5.0137921e+02, 3.6851642e+02, 4.7148450e+02,
       2.6994640e+01, 5.0000000e-01, 1.4000000e+01], dtype=float32)
>>> rect[1]
array([421.13794 , 501.3792  , 366.85837 , 470.54257 ,  26.995132,
         0.8     ,  14.      ], dtype=float32)

the last object_id is not -1

14 means the obj id in object zoos. But again, this field is not used and you can set to -1 when predicting.

[Fang-Haoshu]

however, I ran predictRGD.py at here

I get 255 '*.npz' files for each scene, clearly these are the predicted labels for each RGD image. but the rectagle labels in graspnet dataset seems ...are they share the same format ?

XuSheng told me his rect label format is (x1, y1, x2, y2, height, score, object_id)

Sorry, I don't get your question

[Fang-Haoshu]

to compare

in this repo, I ran with your pretrained weights,

>>> a = np.load('grasp_multiObject_multiGrasp/predicted_rectangle_grasp/scene_0100/kinect/0000.npy')
>>> a.shape
(1556, 7)
>>> a[0]
array([ 4.86425720e+02,  4.92709625e+02,  4.86425720e+02,  4.51794342e+02,
        2.75182495e+01,  7.89444968e-02, -1.00000000e+00])

in graspnet, I download your rect labels

>>> b = np.load('graspnet_dataset/rect_labels/scene_0000/kinect/0000.npy')
>>> b.shape
(7645, 7)
>>> b[0]
array([4.2113794e+02, 5.0137921e+02, 3.6851642e+02, 4.7148450e+02,
       2.6994640e+01, 5.0000000e-01, 1.4000000e+01], dtype=float32)
>>> b[1]
array([421.13794 , 501.3792  , 366.85837 , 470.54257 ,  26.995132,
         0.8     ,  14.      ], dtype=float32)

a.shape[0] == 1556 but b.shape[0]==7645, is this because of the rect labels in graspnets dataset are generated from 6d format. so the number is larger a lot than a

Do you mean why the prediction is much less than the ground-truth and expect the prediction to be the same number of labels?

[cww97]

however, I ran predictRGD.py at here I get 255 '*.npz' files for each scene, clearly these are the predicted labels for each RGD image. but the rectagle labels in graspnet dataset seems ...are they share the same format ? XuSheng told me his rect label format is (x1, y1, x2, y2, height, score, object_id)

Sorry, I don't get your question

I get a description like this when asking X1020

which is diff from (score, width, height, depth, rotation, translation, object_id) from you

[Fang-Haoshu]

however, I ran predictRGD.py at here I get 255 '*.npz' files for each scene, clearly these are the predicted labels for each RGD image. but the rectagle labels in graspnet dataset seems ...are they share the same format ? XuSheng told me his rect label format is (x1, y1, x2, y2, height, score, object_id)

Sorry, I don't get your question

I get a description like this when asking X1020

which is diff from (score, width, height, depth, rotation, translation, object_id) from you

Oh I see, yes, should be (center_x, center_y, open_x, open_y, height, score, object_id) They are now available in the newest docs: https://graspnetapi.readthedocs.io/en/latest/grasp_format.html#rectangle-grasp

[cww97]

thanks a lot, seems I can see evaluation results today~

[cww97]

seem next I should convert these into 6d grasps then evaluation

[Fang-Haoshu]

Cool