search

ethz-asl / ethzasl_ptam

Modified version of Parallel Tracking and Mapping (PTAM)
http://wiki.ros.org/ethzasl_ptam
235 stars 184 forks source link

Cameracalibrator crashed #4

Closed fkreuz closed 11 years ago

fkreuz commented 12 years ago

Hi,

I have a problem with the cameracalibrator. I ran it via the standard launch file, It worked. But after some point of time, it got crashed and showed an error message "Segmentation fault". Sometimes the period of crashing is long and I can manage to calibrate the camera, but other times it happened so fast. Do you have an explanation why it happens?

Another question regarding the stereo initialization phase for PTAM, did you always manually do this initialization before flying (as shown in here: http://www.youtube.com/watch?v=7nhlDK34k_c&list=UUrZt485fBMeNtCDlgEZAHVA&index=9&feature=plcp)?

Thank you before for your answer. Finley

stephanweiss commented 12 years ago

Dear Finley,

Thanks for pointing this out, I will try to upload a tutorial asap (it'll be August though).

Concerning the cameracalibrator crashes: This is a problem of cameracalibrator when there are too many FAST corners detected which do not belong to a checkerboard. We still have to track down the issue. For now you may use the following workaround: place the checkerboard on a homogenuous background (e.g. white table, unicolor floor etc) and try to only "look" at the checkerboard with the camera while calibrating.

Concerning the stereo initializaion: We initialize while flying. Try first to initialize it while holding the helicopter by hand - to get a feeling for good initialization motion. Then you can go for in-flight initialization either by hand or using the AutoInit option in the dynamic reconfigure GUI.

Hope this helps, in any case please feel free to contact us again.

Kind regards, Stephan

From: fkreuz [reply@reply.github.com] Sent: Thursday, June 14, 2012 12:59 PM To: Weiss Stephan Subject: [ethzasl_ptam] Cameracalibrator crashed (#4)

Hi,

I have a problem with the cameracalibrator. I ran it via the standard launch file, It worked. But after some point of time, it got crashed and showed an error message "Segmentation fault". Sometimes the period of crashing is long and I can manage to calibrate the camera, but other times it happened so fast. Do you have an explanation why it happens?

Another question regarding the stereo initialization phase for PTAM, did you always manually do this initialization before flying (as shown in here: http://www.youtube.com/watch?v=7nhlDK34k_c&list=UUrZt485fBMeNtCDlgEZAHVA&index=9&feature=plcp)?

Thank you before for your answer. Finley

Reply to this email directly or view it on GitHub: https://github.com/ethz-asl/ethzasl_ptam/issues/4

fkreuz commented 12 years ago

Hi Stephan,

thanks for your answers. I followed your advice for the cameracalibrator's problem, and it seems to work fine now.

Another question regarding the map. I've noticed from the sFly video that your PTAM implementation can covered large area, and also at the same time grows the number of features stored. For me, it seems that the number of features are limited, thus while moving to another new area it tends to get into the failure mode. Is it because you use fish-eyed camera whose has greater FOV which can maintain the number of detected features while moving? I'm using regular monocular camera.

One more thing, do you have suggestion which parameters I have to tune in order to increase its speed. I'm running your implementation with default settings on Intel Atom 1.6GHz computer, the pose estimation rate is only around 5 Hz.

Btw, sorry for mixing up all these questions in the cameracalibration topic. Thanks in advance for your answers.

Cheers, Finley

stephanweiss commented 12 years ago

Hi Finley,

I am happy to hear that the cameracalibrator now works for you. Thanks for the feedback.

The number of features is limited in the sFly video. More precisely, the number of keyframes (KFs) is limited and each KF has up to 300 features assigned. When moving to a new area a new KF with new features should be created. If the maximum number of KF is reached, the KF furthest away (Euclidean distance) from the current position (i.e. newest KF) is deleted along with its assigned features. Because of this, you should be fine also with a regular monocular camera (FOV of about 90° I assume?).

The failure mode you mention surprises me, I understand that PTAM is not generating a new KF when moving to a new area. This can have the following causes:

Improving speed: if you use the standard parameters and "MaxKF"=5 then it should run at 20Hz in tracking mode or 13Hz in constant KF generation mode (i.e. fast unidirectional movement exploring new area). However, 100% of the ATOM will be used. The frame capture routine used about 15% and PTAM the rest. Do you have any other processes running which may slow down PTAM?

Let me know how it goes.

Best Stephan

From: fkreuz [reply@reply.github.com] Sent: Thursday, August 02, 2012 5:59 PM To: Weiss Stephan Subject: Re: [ethzasl_ptam] Cameracalibrator crashed (#4)

Hi Stephan,

thanks for your answers. I followed your advice for the cameracalibrator's problem, and it seems to work fine now.

Another question regarding the map. I've noticed from the sFly video that your PTAM implementation can covered large area, and also at the same time grows the number of features stored. For me, it seems that the number of features are limited, thus while moving to another new area it tends to get into the failure mode. Is it because you use fish-eyed camera whose has greater FOV which can maintain the number of detected features while moving? I'm using regular monocular camera.

One more thing, do you have suggestion which parameters I have to tune in order to increase its speed. I'm running your implementation with default settings on Intel Atom 1.6GHz computer, the pose estimation rate is only around 5 Hz.

Btw, sorry for mixing up all these questions in the cameracalibration topic. Thanks in advance for your answers.

Cheers, Finley

Reply to this email directly or view it on GitHub: https://github.com/ethz-asl/ethzasl_ptam/issues/4#issuecomment-7459595

fkreuz commented 12 years ago

Hi Stephan,

regarding the failure mode, my camera calibration is correct, but it seemed that I didn't have enough contrast in the new area and also I moved the camera too fast. So, I tried to move it slowly and chose to use automatic exposure to maintain the contrast in the new area. And yeah it seems to work fine. But the problem is while flying the UAV quite low (below 5m), the image which is captured by the camera will be quite fast and jerky, thus leads to failure mode most of the time. I tried also to activate the auto-init function, but the result is quite the same. Do you have any advice on getting through this?

And as for the speed issue, I managed to run it at around 20Hz with standard parameters and MaxKF = 5 by reducing the camera framesize from 640x480 to 320x240. Btw what were the settings you use for the camera and which package was it? For me, I'm using the uvc_cam and gscam packages, but they use around 40% of the processor threads on 320x240 @ 24fps and around 90-100% for 640x480 @ 24fps.

Really appreciate your help till now, thanks a lot.

markusachtelik commented 12 years ago

Hi Finley,

for the first issue (flying low), what is your exposure time? From our experience, it should not be higher than 5ms, otherwise you get too much motion blur. I see you're using the uvc_cam package, so I assume you're using a webcam? Those usually have rolling shutters which distort your image especially when having large motion and flying low such that it won't be useful for any pose estimation. Also, to my knowledge, webcams send compressed video stream over the USB bus. That's another quality loss and would explain the high cpu usage (for de-compression). I would highly recommend using a board level usb camera such as the PointGrey FireFly or similar with global shutter. We currently have a b/w MatrixVision mvBlueFOX-MLC200w (http://www.matrix-vision.com/USB2.0-single-board-camera-mvbluefox-mlc.html) and a 2.8 mm lens from lensation (http://www.lensation.de/en/shop/detail/7-megapixel/flypage/406-lensagon-bm2820.html?sef=hcfp) which we're quite happy with at the moment.

Best, Markus

fkreuz commented 12 years ago

Hi Markus,

thanks for pointing out these things. Yes, I'm using a webcam, to be precise Logitech C905. For the exposure time, I don't know exactly how much is it. So, do you mean there's no chance using this package with only webcam in a low flight scenario? Thanks before.

Cheers, Finley

stephanweiss commented 12 years ago

Hi Finley,

Your chances of success with a webcam are definitely lower than with a board level usb camera, but not zero. But as Markus pointed out: we strongly suggest to use a better camera.

some thoughts:

I hope this helps - let us know how it is going. Best Stephan

From: fkreuz [notifications@github.com] Sent: Friday, August 10, 2012 5:45 PM To: ethz-asl/ethzasl_ptam Subject: Re: [ethzasl_ptam] Cameracalibrator crashed (#4)

Hi Markus,

thanks for pointing out these things. Yes, I'm using a webcam, to be precise Logitech C905. For the exposure time, I don't know exactly how much is it. So, do you mean there's no chance using this package with only webcam in a low flight scenario? Thanks before.

Cheers, Finley

— Reply to this email directly or view it on GitHubhttps://github.com/ethz-asl/ethzasl_ptam/issues/4#issuecomment-7648418.

fkreuz commented 12 years ago

Hi Stephan,

thanks a lot for your inputs. Just for summarizing, correct me if I got it wrong. As far as I understood from Markus and your explanation previously, the main problem using webcam camera for flying low is the image quality (either from rolling shutter effect, slow shutter speed, or due to compression). I'll try to consider the board level usb camera. But for the mean time, I will try to maximize this webcam first.

So, hypothetically if I increase the flying height it's going to somewhat improve, isn't it? What do you think about using front facing camera? The idea is when the distance to the surrounding environment is quite large then i think it will also reduce the motion blur.

Thanks before.

Best, Finley

stephanweiss commented 12 years ago

Hi Finley,

Yes, the main issue is the image quality with respect to robust feature extraction/matching. You can try the front looking cam and the larger distance. This will reduce motion blur caused by translation. Note, however, that on an MAV the blur caused by rotation is a significant portion - and this will increase with distance.

Best of luck and please keep us informed on how it is going. Stephan

From: fkreuz [notifications@github.com] Sent: Monday, August 13, 2012 11:32 AM To: ethz-asl/ethzasl_ptam Subject: Re: [ethzasl_ptam] Cameracalibrator crashed (#4)

Hi Stephan,

thanks a lot for your inputs. Just for summarizing, correct me if I got it wrong. As far as I understood from Markus and your explanation previously, the main problem using webcam camera for flying low is the image quality (either from rolling shutter effect, slow shutter speed, or due to compression). I'll try to consider the board level usb camera. But for the mean time, I will try to maximize this webcam first.

So, hypothetically if I increase the flying height it's going to somewhat improve, isn't it? What do you think about using front facing camera? The idea is when the distance to the surrounding environment is quite large then i think it will also reduce the motion blur.

Thanks before.

Best, Finley

— Reply to this email directly or view it on GitHubhttps://github.com/ethz-asl/ethzasl_ptam/issues/4#issuecomment-7687126.