search

kou1okada / libelas

LIBELAS: Library for Efficient Large-scale Stereo Matching (Unofficial fork)
http://www.cvlibs.net/software/libelas/
46 stars 24 forks source link

improve the result #2

Closed zhaozhongch closed 2 years ago

zhaozhongch commented 5 years ago

Thanks for your work but I didn't get desired result from this lib. I have two pgm images as following left_1403637130538319104 right_1403637130538319104 (The upload pictures are png version seems github issue cannot allow me to upload pgm version but in the code I use .pgm picture) The result is like the follwing left_1403637130538319105_disp right_1403637130538319104_disp I just use the original demo main.cpp. Anyway to improve it? Thanks!

zhaozhongch commented 5 years ago

Thanks for your reply. I'll try to play with the parameter for a while

kou1okada commented 5 years ago

Wait, sorry. May be my post, which was already removed, is not correct.

kou1okada commented 5 years ago

The cause of the problem was not only translation but also rotation.

kou1okada commented 5 years ago

The larger part of stereo matching algorithms are assuming the epipolar constraint along the y-axis. Under this assumption, the same coordinate values of the y-axis on left and right images must ride on same epipolar line. LIBELAS is also assumed it. But your images seem to have translation and rotation. Also the negative values for the disparity are not preferred for many of the implementations of stereo matching algorithms. For more details, read original thesis and Wikipedia.

By the way, for locations closed to the camera, I could not be matched images by binocular stereopsis even with my bare eyes. Thus, it should be not easy to match them by the stereo matching algorithms.

My result is following:

# Download images
wget https://user-images.githubusercontent.com/25599902/60465177-bc597d00-9c0d-11e9-9b0a-8bb7de82535a.png -O left.png
wget https://user-images.githubusercontent.com/25599902/60465186-bfed0400-9c0d-11e9-9886-1812a262e27e.png -O right.png
# Fix rotation and parallel translation for the right image with convert command of ImageMagick.
convert -verbose right.png -rotate 0.35 -crop 752x480+0+0 -distort Affine '0,0 -12,-14' right-fixed.png
# Convert source images from png to pgm.
for i in {left,right}*.png; do convert -verbose $i ${i%.*}.pgm; done
# Get disparity maps with LIBELAS.
./elas left.pgm right-fixed.pgm
# Convert result images from pgm to png. 
for i in {left,right}*_disp.pgm; do convert -verbose $i ${i%.*}.png; done

left.png left right-fixed.png right-fixed left_disp.png left_disp right-fixed_disp.png right-fixed_disp

kou1okada commented 5 years ago

Hmm,,, It's so sensitive to deviations from the epipolar constraint.

zhaozhongch commented 5 years ago

Hi thank you very much for solving this. My data comes from a dataset called euroc dataset https://projects.asl.ethz.ch/datasets/doku.php?id=kmavvisualinertialdatasets This dataset has image in cam0 folder and cam1 folder if you download the dataset ASL format, which corresponding to the left_camera and right_camera. I download the Machine Hall 3 one. I just pick one image in the cam0 folder and then pick another image with the same timestamp in the cam1 folder. considering you say the picture is too close to the camera. I choose another set of picture from theMachine Hall 3. Which is the following left_1403637159038319104 right_1403637159038319104 The result using the elas is elas Considering the first example this looks better. I also tried to use opencv's build-in way to compute the disparity map, the code comes from https://docs.opencv.org/3.1.0/d3/d14/tutorial_ximgproc_disparity_filtering.html The result is sgbm_wls_onfig From this two view seems the result from opencv is slightly better. The important thing is that using libelas it costs about 200ms the get the disparity map but the opencv one costs 20 to 25 ms. This is my result so far, I'll again adjust the parameter to see if I can get better result Thanks!

kou1okada commented 5 years ago

This should be improved the result by calibration, too. It seems better conditions than first case.

oleg-alexandrov commented 3 years ago

I played quite a bit with both LIBELAS and OpenCV's SGBM algorithm. LIBELAS does well, but the way it takes its inputs and outputs is a bit awkward. The biggest limitation I see is the constraint that in LIBEAS the disparity must be positive. I solve this by artificially padding the left image on the left and the right image on the right with zeros with a value big enough that LIBELAS does compute a positive disparity, then I undo this in the disparity returned from LIBELAS.

Also, LIBELAS does not handle very gracefully the case when the input data has missing values, say a certain portion of the scene is seen only in one image and I fill with zeros the corresponding area for the second image. It can return a junk disparity, but which can be filtered out as its values have a jump compared to regular disparities.

Anyhow, I made a fork of this repo and tweaked the main program to deal with these issues. I also made this program take the parameters LIBELAS uses later as command-line arguments, and read and write its images as TIF with float values rather than using 8-bit images on input and output.

My fork is at https://github.com/NeoGeographyToolkit/libelas

kou1okada commented 3 years ago

Your patch seems to be a user friendly front end for LIBELAS. Thank you for letting us know.