search

wkentaro / morefusion

MoreFusion: Multi-object Reasoning for 6D Pose Estimation from Volumetric Fusion, CVPR 2020
https://morefusion.wkentaro.com
Other
230 stars 47 forks source link

How to train on custom synthetic dataset? #68

Open mikkeljakobsen opened 4 years ago

mikkeljakobsen commented 4 years ago

Hi,

I would like to train MoreFusion on my own synthetic dataset of boxes. I've generated my dataset using BlenderProc and I have ground truth instance masks and 6D poses for the boxes. An example train image from my generated data is shown below: image I managed to train Mask R-CNN so it generalizes to real images to some extend. But I need to do full 6D pose estimation.

So my question is: do you think it would be possible to succesfully train MoreFusion on my own custom synthetic data as described above? And what would I need to do in order to adapt the training script to custom data?

Best regards, Mikkel

wkentaro commented 4 years ago

To train the 6D pose estimation model, we need the below information, which probably you can extract from Blender. https://github.com/wkentaro/morefusion/blob/5c1036c19db0a55d24314336b09c01de11284a9f/morefusion/datasets/rgbd_pose_estimation/ycb_video/dataset.py#L73-L80

For the actual training, you need to create your custom dataset class similar to the below. https://github.com/wkentaro/morefusion/blob/5c1036c19db0a55d24314336b09c01de11284a9f/morefusion/datasets/rgbd_pose_estimation/ycb_video/dataset.py#L9

And use it in the train.py script. https://github.com/wkentaro/morefusion/blob/master/examples/ycb_video/singleview_3d/train.py

alanxuefei commented 3 years ago

In ./singleview_3d/train.py, YCBVideoRGBDPoseEstimationDatasetReIndexed is used. YCBVideoRGBDPoseEstimationDatasetReIndexed uses the sample data 1 as below. in_data The above datasets is generated from the sample data 2 as below via MyRealRGBDPoseEstimationDataset rosdatacollection The sample data 2 may be collected via data_collection_server.py.

Questions:

hz-ants commented 3 years ago

@ mikkeljakobsen @alanxuefei Hi,have you completed the project task of multiple instances of similar objects?Can you share your successful experience,thank you!

mikkeljakobsen commented 3 years ago

@ mikkeljakobsen @alanxuefei Hi,have you completed the project task of multiple instances of similar objects?Can you share your successful experience,thank you!

Sorry, I didn't manage to train MoreFusion on my own data.