search

jimmyyhwu / pose-interpreter-networks

Real-time robotic object pose estimation with deep learning
MIT License
122 stars 27 forks source link

end_to_end_eval position estimation (own dataset) #29

Closed ron-uni closed 4 years ago

ron-uni commented 4 years ago

Hey there,

when I evaluate the data using eval.ipynb on the small images, which where used for training the pose estimation network, the position error is abot 1.5 cm. So that's good.

But when I use the end to end evaluation, the position error ist about 2 m. The error is only this big in the z coordinate. The x and y coordinates are fine.

Do you have any idea why it is estimating the position an the small pictures right, but on the masks it is getting from the segmentation network wrong.

Thanks in advance!

Kind regard!

jimmyyhwu commented 4 years ago

Might be helpful to debug using the other notebook, end_to_end_visualize.ipynb. I would check the quality of the segmentations.

ron-uni commented 4 years ago

Thanks for your fast answer!

I did this and the segmentations are fine.

ron-uni commented 4 years ago

How does it work, that you train the pose estimation network on pictures with a resolutuin of 240x320 but the estimation in the end_to_end_eval is later on 480x640. Are you scaling the output of the segmentation network down?

And is there anything special I have to consider, when I render the pictures for the pose estimation network?

jimmyyhwu commented 4 years ago

There is a layer in the end-to-end model which resizes down to 320x240, see here.

The pose interpreter network is trained and tested on 320x240. The segmentation network is trained and tested on 640x480. The models need to be tested with the same resolution they were trained with. Maybe that might be the source of your error?

ron-uni commented 4 years ago

Okay thanks!

No, I trained both networks with the right resolution.

I think my mistake is in rendering the images for the pose estimation network. I am rendering the images with unreal engine 4. How are you generating the positions randomly but in a deterministic manner? Which values are important to consider in this script https://github.com/jimmyyhwu/pose-interpreter-networks/blob/master/pose_estimation/dataset/generate_pose_lists.py, when I am generating the pose list.

jimmyyhwu commented 4 years ago

When generating random poses, the random seed is set to 0, see here.

ron-uni commented 4 years ago

It worked. When I generate the poses in the manner like you did in your script. The poses are estimated correctly. Thank you very much!