lvwj19 / PPR-Net-plus

PPR-Net++: Accurate 6D Pose Estimation in Stacked Scenarios
Apache License 2.0
36 stars 6 forks source link

Processing of real depth images for bin picking task #8

Closed richardjkirton closed 2 years ago

richardjkirton commented 2 years ago

Hi, thank you for the great work!

I am currently trying to implement PPR-Net by training it on simulated data and then perform a real-world bin-picking task with a robot.

I have trained the network and now wish to test how it performs with real depth images. How did you process these images to be inputted to the trained network? Did you save them as 16-bit png images and then convert them to h5 format the same way the training data was processed or did you directly use the point cloud data from the camera?

It would also be great if you perhaps had some sort of example code showing how you used the trained network to process single images.

Thanks again!

lvwj19 commented 2 years ago

Thanks for your attention. We do not collect depth images during application, but directly obtain the original point cloud data from the depth camera. In particular,We use ros to manage our recognition systems. One ros node is for the original point cloud acquisition from depth camera; one ros node is for the original point cloud preprocessing, such as three dimensional clipping and farthest point sampling; one ros node is for 6D pose estimation, in which the preprocessed point cloud is fed into PPR-Net. You need to set up the ROS environment based on your depth camera configuration. And we do not plan to release our application code, sorry. But the entire process is easily set up via ROS by yourself.

richardjkirton commented 2 years ago

Okay great, thank you for the information.

lvwj19 commented 2 years ago

The following code will help you set up ros nodes. Preprocessing code: you can refer to "PPR-Net-plus/convert_to_pointcloud/bunny/H5DataGenerator.py" Inference code:you can refer to "PPR-Net-plus/tools/IPABunny_msg/evaluate.py"

richardjkirton commented 2 years ago

Thanks!

Spj-Zhao commented 2 years ago

hi @richardjkirton ,have you realized the bin picking task using your own camera?