search

antabangun / DualRefine

This repository contains the code for the paper "DualRefine: Self-Supervised Depth and Pose Estimation Through Iterative Epipolar Sampling and Refinement Toward Equilibrium", a self-supervised depth and pose estimation model. It refines depth and pose estimates by computing matching costs based on epipolar geometry and deep equilibrium model.
https://antabangun.github.io/projects/DualRefine/index.html
GNU General Public License v3.0
56 stars 5 forks source link

About re-train the model on the KITTI-odom dataset #2

Closed kevin1455 closed 1 year ago

kevin1455 commented 1 year ago

Hello, Thanks for your excellent work. I trained the KITTI-Odom dataset using the environment and train.sh script you provided. However, I found that the pose results from training were poor, as shown in the figure below. These results closely resemble the DualRefine-initial results presented in the paper. Nonetheless, the test results using the weight you provided are highly accurate. Could you please offer some suggestions? Thank you. image

antabangun commented 1 year ago

Hi, thank you for your interest in our work. Could you try plotting both the model's initial and refined pose outputs? You can get both by doing this.

disp_output, poses = self.depth_pose(...)
poses_initial, poses_refined = poses

The refined pose accuracy also depends on the initial estimates, so it may not give the best output if the initial estimates are insufficient. In most cases, the refinement does improve the overall accuracy, so I suspect the initial estimates itself may not be that good.

The provided weights are trained on the eigen split, which may have some shared images with the odom test set, so it may not be a fair comparison. Unfortunately, we don't have the weights that we trained on the odom training split anymore. So, I'll try to re-train the model on the odom training split again to see if I encounter the same issue and will try to keep you updated.

kevin1455 commented 1 year ago

Thanks a lot for your response. By using the functions you provided, I have obtained the results shown in the figure below(Re-train), and the refined pose is indeed outstanding. It appeared that the depth_pose.forward_pose function I previously employed seemed to output only the initial pose.

However, it is quite surprising that the initial pose from the weight you released is already quite good(Release). Could this possibly be attributed to the issue you mentioned regarding the potential overlap of samples between the training and test sets?

image

antabangun commented 1 year ago

That's great to see. Yes, the forward_pose is a forward function for the initial 'PoseNet' module, which, in hindsight, was not the best naming and could cause misunderstanding.

As for the released weights, the overlap of samples may have played a role. However, I also observed that the outputs from the initial pose estimates can vary greatly, and this might have been one of the better ones. I think that the training strategy for this may need to be revisited for more consistent results.

kevin1455 commented 1 year ago

Thanks for your reply and looking forward to your further progress.