search

BerkeleyAutomation / dex-net

Repository for reading the Dex-Net 2.0 HDF5 database of 3D objects, parallel-jaw grasps, and robust grasp metrics
https://berkeleyautomation.github.io/dex-net/code.html
Other
296 stars 97 forks source link

generate_gqcnn_dataset.py not generating valid data #50

Open yruoxi opened 4 years ago

yruoxi commented 4 years ago

I am trying to reproduce the dexnet result for learning. I came up with a problem with dex-net/tools/generate_gqcnn_dataset.py and it does not quite work. Can you perhaps help me with that, or point me to someone who knows this subject matter or has encountered this error before?

Details are as follows

generated_depth

From the left is the depth_ims_tf from generate script, from the right is the reference from the pre-generated set, the left ones feels like just random noise without depth correctly rendered.

I turned on the vis debugging portion of the yaml config file, in which candidate grasp was displayed correctly, rendered_images are similar to the ones on the left, it does have something but nothing like an object. the grasps images seems to not be on any target objects. There is also some problem with vis3d as it needs a trimesh object. Here are the images below, success messages and fail messages.

output

gripper

depth_out

second_err

If i turned off the debug vis in yaml, the script will run to completion, but the generated data is unusable like the ones shown above. Also the code does not have any missing and unable to find packages in the output window.

Can you give some help on that, you insights will be a great help to me.

yruoxi commented 4 years ago

still seeking for help, @jeffmahler

benjertho commented 4 years ago

@yruoxi I am seeing this as well. Did you find a solution for this?

yruoxi commented 4 years ago

No, still seeking help, I don't know 3d rendering well enough to fix the bug.

benjertho commented 4 years ago

Try this fix out: https://github.com/benjertho/dex-net-public/commit/5e67ee089d068c11b400810e4dde19147bcc7e25

It looks like importing the grasping and visualization libraries was the problem.

yruoxi commented 4 years ago

Just tried it and it totally works, I did change the grasp2d and vis2d not found error but I didnt change the vis2d to plt. Also for the record, in gqcnn/grasping/grasp.py, import should also be changed to from gqcnn.grasping import Grasp2D.

Thanks so much for your help

anmakon commented 4 years ago

Hi @benjertho and @yruoxi, I'm seeing similar behaviour (depth and binary images consisting of random noise) as you've encountered before. I've changed all the library imports (grasp2d, vis2d) and tested it with both the dexnet_2_database and the example one, but no luck up to now. It doesn't throw any warnings or errors with the debugging mode turned off. Changing the visualisation from vis2d to matplotlib when debugging didn't really do the trick for me either. Unfortunately, I can't visualise the 3D environment with the object and gripper to check if anything is going wrong there. Do you remember changing anything else to get it working?