search

jimmyyhwu / pose-interpreter-networks

Real-time robotic object pose estimation with deep learning
MIT License
122 stars 27 forks source link

Problem about orientation_error of validation set is extremely high compare to train set. #17

Closed yikakabu closed 5 years ago

yikakabu commented 5 years ago

Hi! I used my own dataset which contains four objects to train the pose_interpreter_network and I found the orientation_error of val dosen't decline well when error of train set converged as the chart below.

火狐截图_2019-05-07T11-28-27 356Z

Consequently, the result of end_to_end_eval is unsatisfactory as well. Despite the positions of object are predicted correctly, most the orientations are wrong. Could you give me any suggestion? Thanks in advance.

jimmyyhwu commented 5 years ago

You could look at the images that get high orientation error on val. If any object has symmetry, you need to select a consistent "correct" orientation.

yikakabu commented 5 years ago

Thank you for your reply.

All objects of my dataset are symmetrical in different level. I don't know how to "select a consistent 'correct' orientation" as you said. Every mask image of the val_dataset created by generate_pose_lists.py only has one target pose.

According to the code, the ways to compute error of val set and train set are same. But the values of error are so different. And the val_data and train_data are rendered by the same method simultaneously. So I wonder why does this happen and how to fix it.

Here are my result of end_to_end_visualize.ipynb. 火狐截图_2019-05-10T09-33-58 958Z I really appreciate your help.

jimmyyhwu commented 5 years ago

What I mean by selecting a consistent "correct" orientation is that regression works only if there is one correct answer. If there are multiple correct answers, it will predict the mean of the correct answers.

Take a box, for example. Application of z-axis rotations of 0 degrees or 180 degrees is equivalent. However, if your dataset annotates that rotation with 0 half of the time, and 180 half of the time, then the model will learn to predict the mean of those two rotations, 90 degrees, which will be very far off from either 0 or 180.

For objects with symmetry, you will need to modify the dataset annotations. Make sure the quaternion orientation labels are unique, i.e., equivalent orientations do not have multiple correct labels in the dataset.

See also Section III-B in the paper. Resolving quaternion double cover was key to enabling the model to converge.

yikakabu commented 5 years ago

Thank you, I am suddenly enlightened.

jimmyyhwu commented 5 years ago

Closing due to inactivity.