No grasp detected after masking

[MaximeBinot]

Hello,

I have succesfully installed all the requirements and tested the grasp detection demo with the example data and it works. Now I would like to test it with my own data so I replace the png files in example_data by my own 2 images (color and depth) captured by a realsense d435i (see below).

When I try to run the demo again with my own images I get this error: No grasp detected after masking ValueError: too many values to unpack (expected 2)

My first guess is that comes from the workspace parameters and the camera intrinsics but after trying a few things the result is still the same.

• I configured the workspace limits by using realsense-viewer and by noting the points x and y coordinates indicated by the cursor. I did the same thing for z by placing an object high enough to set a limit and by taking the workspace ground as a low limit. (Screenshot of the scene with realsense viewer and the coordinates at the cursor)

• I don't know whether it's the depth camera or the rgb camera intrinsics I should write in the code.
• Is there anything else I have to do to use this demo on my own data ?

I can provide more context if required but I am using near source version of the demo.py for grasp_detection.

Thanks in advance for your help !

[chenxi-wang]

When I try to run the demo again with my own images I get this error: No grasp detected after masking ValueError: too many values to unpack (expected 2)

This error occurs when predictions are not empty but:

• the widths of all predicted grasps exceed 0.1m
• (if you set top_down_grasp to True) there are no top-down grasps in predictions
• the translations of all predicted grasps are not in workspace you set

Maybe you can try disabling top_down_grasp?

I don't know whether it's the depth camera or the rgb camera intrinsics I should write in the code.

If you align depth camera to rgb, you should use rgb intrinsics, otherwise depth intrinsics.

[MaximeBinot]

I tried to disable top_down_grasp but I get the same error No grasp detected after masking ValueError: too many values to unpack (expected 2)

The python command is thus python demo.py --checkpoint_path log/checkpoint_detection.tar --debug

I have set the values of the workspace to large values ( xmin, xmax = -0.4, 0.4 ymin, ymax = -0.3, 0.3 zmin, zmax = 0.0, 1.0)

The object in the workspace is only 5cm on its biggest side so the width should never exceed 0.1m (see image in first post)

I entered my own camera intrinsics (computed thanks to pyrealsense2 library). The values seems legit as they are close to the ones in the demo.

Any other suggestions on what could cause this error ? Can my input image be responsible ? (I use 1280x720 images)

[chenxi-wang]

I just found a bug in the inference code and it may cause no prediction when there are only one or two objects. Later I will re-compile the code. Please wait for a moment.

[chenxi-wang]

Hi, you can try the new version now. I hope this will solve your problem.

[MaximeBinot]

It worked ! Thank you very much for this fix and for your rapid answer :smile:

I am testing on the strawberry I showed earlier in my screenshots and the best grasp turns out to be this one. It seems good but I noticed a problem with the alignment between depth and rgb image. (see below) Do you have any tips to do a good alignment in order to get good intrinsics parameters ?

Thanks again for you help !

[chenxi-wang]

It can be done with official APIs. You can refer to https://github.com/IntelRealSense/librealsense/blob/master/wrappers/python/examples/align-depth2color.py#L57-L82

[elenacliu]

I also met the error when there are only 1~2 objects if I set the --top_down option. But if there is actually no grasp output, how to write the code to avoid the error No grasp detected after masking ValueError: too many values to unpack (expected 2)?

I guess the bug may be here? https://github.com/graspnet/anygrasp_sdk/blob/27b09a9de6d631939da519daea4300480557ce25/grasp_detection/demo.py#L54

Or somewhere in your .so file?

[chenxi-wang]

@elenacliu For some of the empty-grasp cases, anygrasp may return 3 values (None, cloud, points.cuda()). That's just for testing before releasing the sdk and I'm considering unifying them in the next version. To avoid the error for now, you can pass the returned values to a variable first and check the tuple length.