live_slam and dataset_slam is not working

jferdelyi commented 9 years ago

Hello,

I'm working on Linux Ubuntu 12.04.5 LTS with ROS Fuerte.

I’m currently working on a drone project which consists on making the 3D reconstruction of the environment. I am now in the preliminary tests phase of the 3D reconstruction via multiple algorithms in order to determine which is the most relevant.

This said, while testing the LSD-SLAM algorithm, I encountered some problems and I can’t seem to be able to generate a good cloud of points. The camera has been recalibrated 3 times to be sure we got coherent values (because I thought that the problem could be a bad calibration), but the reconstruction is still bad.

I have a set of images taken by a drone with its OpenCV calibration parameters; you will also find the information in the calibImages.cfg file with its coefficients. All this taken in consideration, I get with following command :

rosrun lsd_slam_core dataset _files:=/home/djeef/Documents/TER/TER_img_drones _calib:=/home/djeef/Documents/calibImages.cfg _hz:=30

In the "best case" scenario :

Reading Calibration from file /home/djeef/Documents/calibImages.cfg ... found! found OpenCV camera model, building rectifier. Input resolution: 640 480 In: 738.035034 738.035034 309.750183 227.734238 -0.139448 0.208432 0.000000 0.000000 Out: Crop Output resolution: 640 480 Started mapping thread! Started constraint search thread! Started optimization thread found 999 image files in folder /home/djeef/Documents/TER/TER_img_drones! Doing Random initialization! started image display thread! Done Random initialization! TRACKING LOST for frame 15 (0.00% good Points, which is -nan% of available points, DIVERGED)! ...

Or in the worst case scenario :

Reading Calibration from file /home/djeef/Documents/calibImages.cfg ... found! found OpenCV camera model, building rectifier. Input resolution: 640 480 In: 738.035034 738.035034 309.750183 227.734238 -0.139448 0.208432 0.000000 0.000000 Out: Crop Output resolution: 640 480 Started mapping thread! Started constraint search thread! Started optimization thread found 999 image files in folder /home/djeef/Documents/TER/TER_img_drones! Doing Random initialization! started image display thread! Done Random initialization! warning: reciprocal tracking on new frame failed badly, added odometry edge (Hacky). warning: reciprocal tracking on new frame failed badly, added odometry edge (Hacky). int g2o::csparse_extension::cs_cholsolsymb(const csdi, double_, const csdis, double_, int*): cholesky failed! Cholesky failure, writing debug.txt (Hessian loadable by Octave) terminate called after throwing an instance of 'Sophus::ScaleNotPositive' what(): Sophus exception: Scale factor is not positive Abandon (core dumped)

I tested the algorithm with this calibration file, when it's on "crop" or "full" mode or the "e1 e2 e3 e4 0", I get the problem I described above. When I use "none" I get :

Reading Calibration from file /home/djeef/Documents/calibImages.cfg ... found! found OpenCV camera model, building rectifier. Input resolution: 640 480 In: 738.035034 738.035034 309.750183 227.734238 -0.139448 0.208432 0.000000 0.000000 NO RECTIFICATION Output resolution: 640 480 OpenCV Error: Assertion failed (func != 0) in transpose, file /tmp/buildd/ros-fuerte-opencv2-2.4.2-1precise-20130312-1306/modules/core/src/matrix.cpp, line 1885 terminate called after throwing an instance of 'cv::Exception' what(): /tmp/buildd/ros-fuerte-opencv2-2.4.2-1precise-20130312-1306/modules/core/src/matrix.cpp:1885: error: (-215) func != 0 in function transpose

Intrinsic parameters : Res : 640x480 cx : 309.750187 cy : 227.734240 fx : 738.0350082 fy : 738.0350082 k1 : -0.1394478 k2 : 0.208432 k3 : 0 p1 : 0 p2 : 0

My calibration file : 738.0350082 738.0350082 309.750187 227.734240 -0.1394478 0.208432 0 0 640 480 crop 640 480

I also tested with a monocular camera; it gives me a black screen. Here are its intrinsic parameters, I saw here (https://github.com/tum-vision/lsd_slam/issues/16) that the problem could be coming from the fact that the camera I'm using is an RGB camera and not a monochrome camera :

Intrinsic parameters : Res : 640x480 fx : 675.471 fy : 676.928 cx : 317.989
cy : 237.301 k1 : 0.152314 k2 : -1.26394 k3 : 2.4048 p1 : -0.000654936 p2 : 0.000568698

My calibration file : 675.471008 676.927979 317.898010 237.300995 0.152314 -1.263940 -0.000655 0.000569 640 480 crop 640 480

JakobEngel commented 9 years ago

hmm, can you upload the images somewhere so I can have a look at them?

jferdelyi commented 9 years ago

I would be more than happy to do so, but unfortunately the images I'm using are not my property and so I don't have the right to publish them. We managed to use the calibration by directly entering the D, K, R, P matrixes in the ROS node that publishes the video. However, when we tried to implement all this on the drone, we encoutered a problem with a dimension that you cannot multiply by 16 which makes us use the .cfg with the crop mode file but the algorithm wouldn't work correctly.

JakobEngel commented 9 years ago

hmm, the multiple-of-16 constraint is somewhat unfortunate anyway, and could be changed with some work. How do you crop the image? If you just crop of pixels on the right / bottom (no re-scaling, no cropping on top / left), the calibration parameters fx, fy, cx, cy stay the same. however as the parameters from the calibration file fromat used by LSD-SLAM are scaled with width and height, they have to be changed respectively. (i.e. multiply fx and cx with (oldWidth/newWidth), and the same with fy and cy).

QichaoXu commented 9 years ago

@mhkabir, Hello,

I'm working on ROS indigo + ubuntu 14.04 LTS, I have got results using dataset_slam, but the results seems not so good.

I use a monochrome global-shutter camera (exactly the same as the one you recommend in the homepage),

but I don't know how to use live_slam with this camera. I mean, there should be some commands to initiate it, how? Could you tell me the details?
more, could you tell me how you calibrate your camera? Because my calibration seems not efficient, which may be the reason why dataset_slam result is bad.

I think using the recommended camera could make things easy, as you have all worked with it successfully.

JakobEngel commented 9 years ago

@QichaoXu

for calibrating the camera I always used the PTAM cameracalibrator, e.g. from here: https://github.com/JakobEngel/ethzasl_ptam (fork from ETH's ROS-PTAM port), which works very well.
during live-usage, simply press 'r' to reset everything, and then slowly move the camera sideways (do not rotate it!), for initialization. Also, best start with a fixed exposure value, as aggressive auto-shutter corrections may cause problems sometimes.

QichaoXu commented 9 years ago

@JakobEngel ,thank you for your reply. I'm working on it.

karim-nemra commented 9 years ago

Hello

I am new in ROS,

I am using the LSD-SLAM, for live test I am using .bag file, is it possible to use an .avi video as input.

Does the *.bag file contain only the images?

Thank you very much for your answe

karim-nemra commented 9 years ago

Hello, Jakob Engel

I have the following message, after few munites. The same message with different squences

warning: reciprocal tracking on new frame failed badly, added odometry edge (Hacky). terminate called after throwing an instance of 'Sophus::ScaleNotPositive' what(): Sophus exception: Scale factor is not positive Aborted (core dumped)

QichaoXu commented 9 years ago

@JakobEngel

The results of live_slam running on my computer are always not so good, what could be the problem probably?

I thought there could be these two reasons?

1.My calibration file is: 1.1977124 1.1977124 0.49921875 0.498958333 0 640 480 1.1977124 1.1977124 0.49921875 0.498958333 0 640 480 could there any wrong in my calibration?

2.I have always wanna ask, could the processing speed of the computer have any bad influence on the result? If processing speed isn't fast enough to calculate information in each frame, could it effect the result?

What do you think? Hope you can give me some suggestions.

liqile commented 9 years ago

@QichaoXu hi, have you solved your problem? I faced with similar problem. how to run lsd with your own data?

中文翻译：您好，请问您的问题现在解决了吗，我现在不知道如何跑其他的数据。我现在想用lsd跑一下tum上rgb-d的其他数据，但是viewer总是啥都不显示

giancan commented 8 years ago

@JakobEngel Hi, are there any news on this topic? I get the same error and I would like to process some images extracted from a video. Should I change anything? Will it make any difference if I scale the images down and recalibrate the camera? Thanks

tum-vision / lsd_slam

live_slam and dataset_slam is not working #40