About Hand and Object Segmentation

[aikuide]

Hi！ In the paper, it is mentioned that the perception module obtained RGBD images and segmented images to obtain segmented point clouds of objects and hands. I would like to know how to obtain segmented images?

[ychao-nvidia]

If you are referring to the simulation experiments, the segmentation images are directly rendered from the simulation engine (Bullet).

[aikuide]

Sorry, I may not be familiar with what you mean by rendering. Does it require neural network training on the dataset of hands and objects to achieve segmentation

[ychao-nvidia]

IIRC Bullet uses an OpenGL based renderer. This is classical rendering and does not require neural network.

[aikuide]

Thank you very much for your patient answer. So how can I obtain such segmented images in the paper

[ychao-nvidia]

The segmented image is obtained in this line. You should be able to check it if you run the demo.

[aikuide]

So what do you mean is that you can directly obtain such segmented images through rendering and then pass them directly to the perception module? In my understanding, real-time segmentation of hands and objects is also a difficult task

[ychao-nvidia]

You can directly obtain such segmented images through rendering and then pass them directly to the perception module?

Yes.

Real-time segmentation of hands and objects is also a difficult task.

It does not matter in simulation. In the real world, yes.

[aikuide]

Thank you for your patient answer