About the initialization of the pose for tracking optimization on Replica dataset

[pengchongH]

Hi! Thanks for your nice work!

I wonder that the reason why you choose multi-scale RGBD odometry to initialize the pose on Replica rather than using constant speed assumption which is used in other datasets. Could you please share some brief idea behind this choice?

[unique1i]

Hi @pengchongH. Constant speed assumption works nicely also on Replica dataset. The odometry initialization here gives slightly better performance.

[pengchongH]

Thanks for your reply. I have tried to use constant speed on the Replica dataset. However, it looks like the constant speed cannot work well in room0 and room1, but it works well in office0.

[unique1i]

I confirm the const speed initialization alone does lead to large pose error on room0 and room1. The is due to that a few times along the sequence there are much higher re-rendering error after initialization.

A mechanism is there to switch to odometer initialization in this case, please also enable help_camera_initialization in configs/Replica/replica.yaml when using const speed initialization. This helps get small pose error.

[pengchongH]

Thanks for your reply. I have tried this and it works well. I think this issue only occurs in room scenes, including room0-room2, in Replica, since you use const speed assumption for other datasets, and the const speed assumption also works well for office scenes in Replica. Based on this, I would like to know whether you have some brief idea why this kind of issue occurs.

[VladimirYugay]

Pose optimization is quite sensitive for initialization. We think that in the rooms since they fall to the local minima and can't recover, while odom initialization helps avoid them.

[VladimirYugay]

@pengchongH After investigation, we figured out that there was a bug in the release branch. This function works, but not entirely correct - we need to initialize rotation quaternions differently. We fixed it and are now testing it on all the datasets. We will push the fix to the main branch as soon as we're done testing.

[VladimirYugay]

Pushed to the main. Feel free to reopen if you have any further questions!

[pengchongH]

Thanks for your reply. I will try your updated codes. Really appreciate!