search

NVlabs / CenterPose

Single-Stage Keypoint-based Category-level Object Pose Estimation from an RGB Image (ICRA 2022)
Other
285 stars 34 forks source link

Question about loss function #9

Closed yuchen-ji closed 2 years ago

yuchen-ji commented 2 years ago

I noticed that you use 2d keypoints & relative cuboid dimensions for supervision, could i also use the 6-DOF pose for surpervision? this 6-DOF loss could backward correctly？Does PNP algorithm affect back propagation？ thanks for your reply！

Uio96 commented 2 years ago

Yep. The 6-DoF pose is another choice. But its searching space is larger than our keypoint choice. As for PnP, we used a non-differentiable version while you may want to try some differentiable one, like this https://github.com/BoChenYS/BPnP.

For more reference, I would recommend this repo (https://github.com/YoungXIAO13/ObjectPoseEstimationSummary).

yuchen-ji commented 2 years ago

Yep. The 6-DoF pose is another choice. But its searching space is larger than our keypoint choice. As for PnP, we used a non-differentiable version while you may want to try some differentiable one, like this https://github.com/BoChenYS/BPnP.

For more reference, I would recommend this repo (https://github.com/YoungXIAO13/ObjectPoseEstimationSummary).

Thanks much for your reply! Your centerpose is great! If I want to add an additional loss term for 6-DoF. maybe process the heatmap which the centerpose output to 2d and 3d keypoints, then use bpnp to backward? ( I'm not sure whether these processes are differentiable

Uio96 commented 2 years ago

Yep. The 6-DoF pose is another choice. But its searching space is larger than our keypoint choice. As for PnP, we used a non-differentiable version while you may want to try some differentiable one, like this https://github.com/BoChenYS/BPnP. For more reference, I would recommend this repo (https://github.com/YoungXIAO13/ObjectPoseEstimationSummary).

Thanks much for your reply! Your centerpose is great! If I want to add an additional loss term for 6-DoF. maybe process the heatmap which the centerpose output to 2d and 3d keypoints, then use bpnp to backward? ( I'm not sure whether these processes are differentiable

Yep. You can have a try. The paper I mentioned earlier may give you some ideas.

yuchen-ji commented 2 years ago

Thank you. bpnp can replace pnp to make it differentiable. I have another confuse about the cofing for training, what's the meaning of self.max_obj = 10, here: https://github.com/NVlabs/CenterPose/blob/6c89d420b33bd01c14c13f509af08bfe3d8b2fe7/src/lib/datasets/dataset_combined.py#L128 and the '--K' in opts.py, here: https://github.com/NVlabs/CenterPose/blob/6c89d420b33bd01c14c13f509af08bfe3d8b2fe7/src/lib/opts.py#L120

Learningm commented 2 years ago

Thank you. bpnp can replace pnp to make it differentiable. I have another confuse about the cofing for training, what's the meaning of self.max_obj = 10, here:

https://github.com/NVlabs/CenterPose/blob/6c89d420b33bd01c14c13f509af08bfe3d8b2fe7/src/lib/datasets/dataset_combined.py#L128

and the '--K' in opts.py, here: https://github.com/NVlabs/CenterPose/blob/6c89d420b33bd01c14c13f509af08bfe3d8b2fe7/src/lib/opts.py#L120

max_obj means maximum object number in current frame. K means topK center selection during inference.

By the way, did you try inserting the BPNP module ?