Room Example not working / ardrone camera

[Tgaaly]

I downloaded the room image sequence example and followed the instructions to run lsd_slam. I believe the problem is that lsd_slam takes the camera calibration from 2 places. one in the pinhole_example_calib.cfg and the other from /camera_info. I copied the values from the /camera_info topic and put them in the .cfg file but I get the following error:

Maybe the .cfg file is wrong. Has anyone been able to run this succesfully? I would be grateful for some help.

I also tried this on an ar_drone camera but does not work well. I calibrated and rectified using image_proc but nothing worked. It also ran way too slow so I had to disable the debug depth display window. Has anyone been able to use it on ardrone?

Reading Calibration from file pinhole_example_calib.cfg ... not found! Trying /home/me/catkin_ws/src/lsd_slam/lsd_slam_core/calib/pinhole_example_calib.cfg ... found! found ATAN camera model, building rectifier. /home/me/catkin_ws/src/lsd_slam/lsd_slam_core/calib/pinhole_example_calib.cfg Input resolution: 640 480 In: 254.326950 375.934387 267.381897 231.599091 0.000000 NO RECTIFICATION Output resolution: 640 480 Prepped Warp matrices Started optimization thread Started constraint search thread! Started mapping thread! Doing Random initialization! started image display thread! Done Random initialization! terminate called after throwing an instance of 'Sophus::ScaleNotPositive' what(): Sophus exception: Scale factor is not positive Aborted (core dumped)

[Tgaaly]

Ok I got the room example working. I re-installed lsd-slam and it worked. I had changed code in lsd-slam to read from the .cfg calib file. That was messing things up.

As for getting this running on the ardrone. I re-calibrated my camera. There was a problem with the size of the squares of the calibration checkered board. Now it seems to be working.

[magiccjae]

Hi @Tgaaly ,

I am trying to do LSD_SLAM with my ardrone 2.0. I calibrated the camera and generated a .yml file. However, as you know, the resolution of ardrone 2.0 is 640x360. So when I tried to run LSD live slam by subscribing the /ardrone/front/camera_info topic, it didn't work because the resolution was not the multiple of 16. How were you able to solve this problem? Does it have anything to do with .cfg file? Thanks in advance.

[JakobEngel]

Two options:

• use the .cfg file to automatically rescale the image to a multiple of 16 (hacky, but straight-forward): X X X X X 640 360 crop 640 368
• adapt the whole pyramid-downscaling thing to work with odd resolutions, e.g. by cropping the last line if the height is an odd number. That would be the proper way to do it, but slightly more effort.

[magiccjae]

Thanks @JakobEngel,,

Please clarify one more thing for me.

It seems like you are using the Calibration file for Pre-Rectified Images. Where do I get values for fx/width fy/height cx/width cy/height 0? What do these values represent? Thanks in advance !

[Tgaaly]

@nagneeve yup as @JakobEngel pointed out. I used the .cfg file and the crop option. I did not use the pre-rectification. I set those values to my camera calibration info.

For my ar.drone 2.0 I used the following:

580.389234 578.865313 313.683277 194.130114 -0.532927 0.311583 0.002103 0.002892 0.000000
640 360
crop
640 320

To run it I used the following command:

rosrun lsd_slam_core live_slam /image:=/ardrone/front/image_raw _/calib:=/home/<...>/catkin_ws/src/lsd_slam/lsd_slam_core/calib/pinhole_example_calib.cfg

[magiccjae]

@Tgaaly

As I tried to run LSD SLAM with the values you provided above, I get the following: TRACKING LOST for frame 8 (0.00% good Points, which is -nan% of available points, DIVERGED)

So it seems like I need to get my own values for .cfg file. Anyone can teach me how to get these values please?

[Annyv2]

@Tgaaly Hi! Were you able to make the LSD_SLAM work fast enough with the ar_drone? can you tell us what you did? I have the same problem and currently what I did was use this .cfg (I found it somewhere around here)

0.771557 1.368560 0.552779 0.444056 1.156010 640 360 crop 640 480

That's not technicly mine but it's the one that at least makes it work, even if it's slow. These are the actual values that I found on my own but it doesn't work if I use them (I get TRACKING LOST for frame 2 (0.00% good Points, which is -nan% of available points, DIVERGED)!)

322.274125 321.856470 316.199048 183.550804 -0.480174 0.171296 0.008339 0.001948 0.000000 640 360 crop 640 480

@nagneeve To get your .cfg file you can use something like this http://wiki.ros.org/camera_calibration/Tutorials/MonocularCalibration but @JakobEngel says somewhere that he uses this one https://github.com/JakobEngel/ethzasl_ptam @JakobEngel could that be causing my problem of tracking lost? should I reconfigure again? or should I use another .cfg?

edit: If I try to run the .cfg and set it to crop to 640 320 as you have it I get segmentation fault, why is this happening?

[Tgaaly]

Hey @Annyv2 - it was still slow with a small number of keyframes picked up. I was manually moving the ardrone by hand and in some cases it would create a good map but quite slowly and the quality was not very good. @JakobEngel has given some advise in #75 but I have yet to try them out. If anyone gets this working well on the ardrone, please let us know.

[Annyv2]

I think that the autor of this video https://www.youtube.com/watch?v=r9LegSK6MfU obtained a very good result, he told me this: "As I remember, we tried to avoid using PTAM cameracalibrator, so we used the image_proc to send the rect-stream into lsd_core node directly. (like this command: rosrun lsd_slam_core live_slam /image:=/image_rect _calib:=)"

I tried doing that because I also use image_proc to obtain the image_mono, but there's no calibration on ardrone/camera_info. When I calibrated the camera I tried to commit it to the ardrone but it wouldn't let me. Has someone been able to replicate this?

[haochihlin]

Hello everyone, I am the author of that video. Basing on the project memo which was written by me 7 months ago. (I am not sure if it is workable.) There are my steps:

1. Modify the resolution in "video.h" file of "Ardrone_autonomy" package. (I used 640 x 352)
2. Calibrate the front-camera of ARDrone by camera_calibration node (If you met some problems about commit failure, you can change parameters manually)
3. By using "image_proc" node, get the rect-stream. (e.g. you will find "image_rect" on topic list)
4. Run the lsd-core node. ( $ rosrun lsd_slam_core live_slam /image:=/image_rect _calib:= )

I wish you all success in your projects.

[magiccjae]

@jim1993 ,

I have been trying to get a good 3D point cloud through LSD_SLAM and sometimes I can get pretty decent maps by adjusting minUseGrad, CameraPixelNoise parameters. You said in your video that after mapping, the next step is to port into Rviz, Make plan and Execute plan. Could you please tell me little more detail about how to port into Rviz and use MoveIt? Thank you in advance !

[Phelipea]

Is that a way to edit that video.h file if i have the binary version installed? i did search for that file in /opt directory but it doesn't exists. I've tried to install through the source but it was getting Segmentation Fault, and i couldn't resolve that.

[Annyv2]

I got it to work by lowering the resolution so I guess it's my computer (right now it works decently with 400 x 288 or lower) but the pointcloud I get is... backwards, is there a way i can change this? I add a picture of what I get. @Phelipea Not sure if this could help you out, but you can also change the resolution if you use a calibration file and then after crop you put the desired resolution

[Phelipea]

I'm doing that as @Tgaaly suggested, using the pinhole_example_calib file, the image_rect topic, varying the parameters you suggested and tried some different resolutions but the mapping is still slow and the pointclouds matching are so confuse, can you show me your configs?

[Annyv2]

I'm not using the image_rect topic because I didn't "commit" when using cameracalibration, so the manual way of changing it is using the .cfg file I think. This is what I'm using, first I create the image mono using this:

ROS_NAMESPACE=ardrone rosrun image_proc image_proc

Then this

rosrun lsd_slam_core live_slam image:=/ardrone/image_mono _calib:=/home/anny/Calibracion/CameraCalibration.cfg

With calibration file

0.50355332 0.894045767 0.494061013 0.509863 0 640 360 crop 400 288

Or this with distortion vector

0.50355332 0.894045767 0.494061013 0.509863 -0.466013 0.176685 0.032364 -0.010129 0.000000 640 360 crop 400 288

My pointcloud is inverted haha but I think that I didn't change anything that could cause this (I also modified the noise parameters), has anyone else had this problem?

[Phelipea]

Thanks @Annyv2 !! The mapping is getting better but still to slow, is it a good option to write a note that crops or resize the image to a multiple of 16, instead of setting this in that calibration file? Will it improve the performance?

[Annyv2]

Do you obtain an inverted map like me? I think that @JakobEngel has said before that some parameters need to be configured to make it work better here #75. I also modified the camera noise parameters minUseGrad, CameraPixelNoise like @nagneeve (I made them bigger, I think it improved it a little bit)

[Phelipea]

I'll check those parameters, thanks again! My maps are inverted too, Do you know if it's possible to generate an octomap or how to save them as .pcd file directly? I'll use them for navigation =)

[Annyv2]

Oh, perhaps it's like that then. I haven't done that so I'm not really sure and I haven't tried using the m command that they explain on the readme either, but if you find a way or use the command let us know :D

[boonflies]

Has anybody had success with porting 3D pointcloud into Rviz, Make plan and Execute plan?