search

tum-vision / lsd_slam

LSD-SLAM
GNU General Public License v3.0
2.62k stars 1.23k forks source link

OpenCV calibration problem #250

Open antonays opened 7 years ago

antonays commented 7 years ago

Hello, I am trying to implement the algorithm on a raspberry pi, i am trying to do it with two types of cameras, one is the camera module of the raspberry pi, with the possibility to add wide lens, the other one is a usb camera attached to the raspberry pi. Unfortunately both the cameras are rolling shutter so i am expecting not perfect results.

However, My problem is a problem of calibration. Trying to calibrate with a ptam tool (ethzal-ptam), i am able (after a lot of hustle) to calibrate with a lows enough rms (below 0.5) and get the calibration parameters, which then work to some extend with the lsd-slam algorithm. The algorithm works, the tracking is good, however the map reconstruction is terrible, and i need to be very careful when moving the camera not to induce a fast rotation which almost always causes a loss of tracking. This was re-tried with many different configurations and trials - for example, with or without cropping, with or without skew, with different settings for the camera framerate, brighness, focus settings and many different things. NOTHING WORKS

The main problem is with the opencv calibration - performing the calibration itself with the ros calibration tool is very easy and immediate. I get the opencv parameters and use them for the lsd slam, but this just doesn't work, when running the algorithm with these parameters, the tracking is terrible - without moving the camera it will crush by itself after ~100 frames. Moving the camera will crash immediately, after 2-3 frames. Needless to say no tracking is done and nothing can be seen in the viewer. The opencv calibration parameters were double and triple checked, both in the ros calibrator and with the matlab calibration tool and they seem to be correct, but there is no way to make the lsd-slam work with them.

Following an existing issue, i was thinking that maybe the problem comes from an old version of opencv (the one that is built in to ubuntu 14.04), but updating opencv to version 3 and recompiling the lsd-slam didn't help at all - same problem remains.

Note that the viewer itself works well with the .bag files and is able to reconstruct whatever is in there.

Is there anybody who tried a similar implementation (cheap, rolling shutter camera with narrow lens) and got the algorithm to work well both in tracking and reconstruction?

romdos commented 7 years ago

Did you launch it with camera_info:=... parameter?

antonays commented 7 years ago

@romdos, i did try to launch with this information but it didn't solve my problems. In the end i couldn't get any reasonable results with a rolling shutter narrow angle camera, neither with a rolling shutter wide angle camera. It is also mentioned in the documentation that a rolling shutter will lead to inferior results. I ended up using an expensive ueye global shutter camera which i borrowed from a friend for the implementation.