search

raulmur / ORB_SLAM2

Real-Time SLAM for Monocular, Stereo and RGB-D Cameras, with Loop Detection and Relocalization Capabilities
Other
9.3k stars 4.69k forks source link

Replaying euroc dataset with map points seems offseted #495

Closed codec-abc closed 6 years ago

codec-abc commented 6 years ago

Hi,

Firstly let me thanks all the people involved in this great project.

I try to replay the euroc data set (Machine Hall number 3) with the image sequence from cam0 in Unity. For that, I have played it using the provided sample in an Ubuntu VM and dump the poses at each frames and the final 3D coordinates of the map points. This gave me 2 files: poses.txt and mappoints.txt.

Then, in Unity at startup, I create a small sphere for each point of the map data and create a plane half transparent in front of the camera to display the background texture. Then, at each frame I update the background texture and apply the corresponding pose to the camera.

When replaying the image sequence it seems that the map points are near where they are supposed to be but not quite right as we can see in these 2 images:

window1

window2

Moreover I wrote a shader to remove the distortion from the image and processed the image set with OpenCV before replaying and the results are still not correct.

Do you have any idea about what explain what is happening?

Moreover I have set-up a repository so if someone is kind/adventurous enough he/she can load the project in Unity and experiment with it.

The project contains a single scene and a single script so it should be easy enough for anyone (even with no Unity experience) to toy around with the project.

codec-abc commented 6 years ago

I found the bugs. There was 2 of them. The first is that I didn't exported the poses and the 3d points of the map at the same time. Since I wasn't aware that there is an arbitrary scale factor on monocular SLAM that will vary even when replaying the exact same set of images. I didn't pay attention to that. But it does matter a lot and reexporting both files at the same time fix a really big part that made the offset. The other bug was in the distortion shader, for a reason it seems that I needed to "flip" the optical center for the y-axis.

Sorry to have opened an issue for that.

nehaprajapat commented 2 years ago

@codec-abc I am facing similar issue while transforming the camera poses obtained from slam to position coordinates in Unity vector space. How does scale factor plays a role in it and how can it be resolved?

codec-abc commented 2 years ago

The scale factor can be resolved if you identify a element of known size in the image or by using another way (stereo-vision, inertia sensor, etc...)