Why not only use ORBSlam2? ICP tracker seems redundent.

[jeezrick]

Hello. During my experiments, I've noticed that in complex scenarios such as the TUM dataset or your hotel scene, the ICP tracking often fails. Consequently, the system resorts to plain ORB-SLAM tracking, skipping the 'track_with_icp_pose' function. This seems to increase the tracking time by about a quarter without any effect on final pose. Is this an accurate assessment?

I ran 1000 frames of hotel scene, and record the fail times:

TOTAL ICP failed portion: 923 / 1001 = 0.922077922077922

And

    def refine_icp_pose(self, pose_t1_t0, tracking_success):
        if tracking_success and self.orb_useicp:
            self.orb_backend.track_with_icp_pose(
                self.curr_frame["color_map_orb"],
                self.curr_frame["depth_map_orb"],
                pose_t1_t0.astype(np.float32),
                self.curr_frame["timestamp"],
            )
            time.sleep(0.005)
        else:
            self.orb_backend.track_with_orb_feature(
                self.curr_frame["color_map_orb"],
                self.curr_frame["depth_map_orb"],
                self.curr_frame["timestamp"],
            )
            self.failed_times += 1
            time.sleep(0.005)

why sleep for 5 ms? Isn't orbslam tracking live in the main thread?

[MisEty]

1. This is because in complex scenarios, there may be cases where the Gaussian optimization during the runtime of tracking has not been completed fully, leading to a higher chance of tracking failure. On the other hand, using only ICP on the Replica dataset can yield more accurate poses. In fact, we consider the ORB backend and the ICP frontend as two parts that can be selected based on specific circumstances. Parameters can be chosen based on the specific mapping situation.
2. This is because there may sometimes be synchronization conflicts in multi-threading. It is possible that removing this 5ms delay can also work. We haven't conducted a detailed test.

[jeezrick]

thanks for your reply, it's all clear now.