search

fixstars / libSGM

Stereo Semi Global Matching by cuda
Apache License 2.0
596 stars 187 forks source link

I got a problem when i make the project! #63

Open CZH-cpu opened 3 years ago

CZH-cpu commented 3 years ago

I have already installed the CUDA 7.5 ,opencv 3.4.1 ,cmake 3.5.1 and all of these is on ubuntu 16.04. when i do the order "make" ,lots of errors happened.

/media/czh/12f4dee2-c297-4aa8-b2e3-455294a518e7/libSGM-master/src/oblique_path_aggregation.cu(38): error: Within a device/global function, only shared variables may be marked "static" detected during instantiation of "void sgm::path_aggregation::enqueue_aggregate_upleft2downright_path(sgm::cost_type , const sgm::feature_type , const sgm::feature_type *, int, int, unsigned int, unsigned int, int, cudaStream_t) [with MAX_DISPARITY=64U]" (203): here

/media/czh/12f4dee2-c297-4aa8-b2e3-455294a518e7/libSGM-master/src/oblique_path_aggregation.cu(39): error: Within a device/global function, only shared variables may be marked "static" detected during instantiation of "void sgm::path_aggregation::enqueue_aggregate_upleft2downright_path(sgm::cost_type , const sgm::feature_type , const sgm::feature_type *, int, int, unsigned int, unsigned int, int, cudaStream_t) [with MAX_DISPARITY=64U]" (203): here

/media/czh/12f4dee2-c297-4aa8-b2e3-455294a518e7/libSGM-master/src/oblique_path_aggregation.cu(40): error: Within a device/global function, only shared variables may be marked "static" detected during instantiation of "void sgm::path_aggregation::enqueue_aggregate_upleft2downright_path(sgm::cost_type , const sgm::feature_type , const sgm::feature_type *, int, int, unsigned int, unsigned int, int, cudaStream_t) [with MAX_DISPARITY=64U]" (203): here

/media/czh/12f4dee2-c297-4aa8-b2e3-455294a518e7/libSGM-master/src/oblique_path_aggregation.cu(42): error: Within a device/global function, only shared variables may be marked "static" detected during instantiation of "void sgm::path_aggregation::enqueue_aggregate_upleft2downright_path(sgm::cost_type , const sgm::feature_type , const sgm::feature_type *, int, int, unsigned int, unsigned int, int, cudaStream_t) [with MAX_DISPARITY=64U]" (203): here

/media/czh/12f4dee2-c297-4aa8-b2e3-455294a518e7/libSGM-master/src/oblique_path_aggregation.cu(43): error: Within a device/global function, only shared variables may be marked "static" detected during instantiation of "void sgm::path_aggregation::enqueue_aggregate_upleft2downright_path(sgm::cost_type , const sgm::feature_type , const sgm::feature_type *, int, int, unsigned int, unsigned int, int, cudaStream_t) [with MAX_DISPARITY=64U]" (203): here

5 errors detected in the compilation of "/tmp/tmpxft_0000230b_00000000-7_oblique_path_aggregation.cpp1.ii". CMake Error at sgm_generated_oblique_path_aggregation.cu.o.cmake:266 (message): Error generating file /media/czh/12f4dee2-c297-4aa8-b2e3-455294a518e7/libSGM-master/src/CMakeFiles/sgm.dir//./sgm_generated_oblique_path_aggregation.cu.o

src/CMakeFiles/sgm.dir/build.make:133: recipe for target 'src/CMakeFiles/sgm.dir/sgm_generated_oblique_path_aggregation.cu.o' failed make[2]: [src/CMakeFiles/sgm.dir/sgm_generated_oblique_path_aggregation.cu.o] Error 1 CMakeFiles/Makefile2:85: recipe for target 'src/CMakeFiles/sgm.dir/all' failed make[1]: [src/CMakeFiles/sgm.dir/all] Error 2 Makefile:127: recipe for target 'all' failed make: *** [all] Error 2

Please help me.

atakagi-fixstars commented 3 years ago

Hi, @CZH-cpu

Maybe CUDA Toolkit is too old to compile.
Try to update CUDA Toolkit. The latest is 11.2. https://developer.nvidia.com/cuda-downloads

Regards,

CZH-cpu commented 3 years ago

Thanks for answering, after that i succeed in installing ubuntu18.04 , cuda10.0 and the libSGM project. By the way my opencv version is 3.1.4. Then i tried to run the sample execution and i got some new problem. First of all, i want to know the position to put these images, I named my images as left_image_0000 and right_image_0000 and I put them at /libSGM/build/sample/image/ which i think it should be,then i run the code "./stereo_movie leftimage%01d.pgm rightimage%01d.pgm" at /libSGM/build/sample/muvie/ and the terminal told me imread failed.(the photo are colored), I do not know the wrong part. Second since you said the sample images are available in the "Daimler Urban Scene Segmentation Benchmark Dataset 2014" , when i tried to download the images ,it just told me the no found of anything so if you have the images for testing, can you provide for me ?

Hope for you answering again , thanks a lot.

sotsuka-fixstars commented 3 years ago

@CZH-cpu Thank you for trying to use libSGM.

I named my images as left_image_0000 and right_image_0000 and I put them at /libSGM/build/sample/image/ which i think it should be,then i run the code "./stereo_movie leftimage%01d.pgm rightimage%01d.pgm" at /libSGM/build/sample/muvie/ and the terminal told me imread failed.

I beg your pardon. According to README.md, stereo_movie sample find files by 0-indexed, but it should be 1-indexed.
Moreover, if a number of left-right image pairs is 1 and format string is used, stereo_movie cannot run correctly.
We will fix it.

  1. Path

A location which has images can be anywhere, but it should be current directory if we use ./<image_name>. In this case, our current directory is /path/to/libSGM/build/sample/movie/, and image files exist in ../image/ or /path/to/libSGM/build/sample/image/.
Following page may be helpful. https://www.geeksforgeeks.org/absolute-relative-pathnames-unix/

  1. Format String

left_image_%01d.pgm will be formatted to left_image_1.pgm, left_image_2.pgm, ... left_image_<N>.pgm where N is number of files. This is because movie sample uses std::snprintf (in definition of format_string).
Please see http://www.cplusplus.com/reference/cstdio/printf/ for more details.
We can also use string not including format string for 1-pair of images.

  1. Image Format

All samples assume that input image are grayscale (CV_8U, CV_16U).
Please use patch if we want to input colored images.

diff --git a/sample/movie/stereosgm_movie.cpp b/sample/movie/stereosgm_movie.cpp
index 062451b..b4bf6b8 100644
--- a/sample/movie/stereosgm_movie.cpp
+++ b/sample/movie/stereosgm_movie.cpp
@@ -60,8 +60,8 @@ int main(int argc, char* argv[])

        const int first_frame = 1;

-       cv::Mat I1 = cv::imread(format_string(argv[1], first_frame), -1);
-       cv::Mat I2 = cv::imread(format_string(argv[2], first_frame), -1);
+       cv::Mat I1 = cv::imread(format_string(argv[1], first_frame), cv::IMREAD_GRAYSCALE);
+       cv::Mat I2 = cv::imread(format_string(argv[2], first_frame), cv::IMREAD_GRAYSCALE);
        const int disp_size = argc >= 4 ? std::stoi(argv[3]) : 128;

        ASSERT_MSG(!I1.empty() && !I2.empty(), "imread failed.");
@@ -90,8 +90,8 @@ int main(int argc, char* argv[])

        for (int frame_no = first_frame;; frame_no++) {

-               I1 = cv::imread(format_string(argv[1], frame_no), -1);
-               I2 = cv::imread(format_string(argv[2], frame_no), -1);
+               I1 = cv::imread(format_string(argv[1], frame_no), cv::IMREAD_GRAYSCALE);
+               I2 = cv::imread(format_string(argv[2], frame_no), cv::IMREAD_GRAYSCALE);
                if (I1.empty() || I2.empty()) {
                        frame_no = first_frame;
                        continue;

Therefore, we can run stereo_movie sample like following in this mentioned case.

shingo.otsuka@sgm01:~/work/libSGM/build$ make
[ 63%] Built target sgm
[ 72%] Built target stereo_test
[ 81%] Built target stereo_movie
[ 90%] Built target stereo_reprojection
[100%] Built target stereo_benchmark
shingo.otsuka@sgm01:~/work/libSGM/build$ cd sample/image/
shingo.otsuka@sgm01:~/work/libSGM/build/sample/image$ ls
CMakeFiles  cmake_install.cmake  right_image_0000.pgm
Makefile    left_image_0000.pgm  stereo_test
shingo.otsuka@sgm01:~/work/libSGM/build/sample/image$ cd ../movie/
shingo.otsuka@sgm01:~/work/libSGM/build/sample/movie$ ./stereo_movie ../image/left_image_0000.pgm ../image/right_image_0000.pgm

Second since you said the sample images are available in the "Daimler Urban Scene Segmentation Benchmark Dataset 2014" , when i tried to download the images ,it just told me the no found of anything so if you have the images for testing, can you provide for me ?

There are no description about redistribution in "Daimler Urban Segmentation Dataset 2014", so it is difficult to provide dataset.
Instead, we prepared download instruction by following.

CZH-cpu commented 3 years ago

Thanks for your help!

After that I run the sample successfully, but I still be confused by some questions.

  1. Since all the things I can do is at .../build/sample/movie, do I have any other ways use libSGM? In short, I want to know what else can I do and I want to know more functions of libSGM.

  2. I run the sample by the images provided on the http://www.6d-vision.com/scene-labeling. I found it use 1000 images in only 14 seconds, so I want to know how many images do I need every second is appropriate.

  3. In README.md part, there is a Test Execution. I don't know what it's for, can you provide me a better explanation?

Hope for your replying again, thanks a lot!

sotsuka-fixstars commented 3 years ago
  1. Since all the things I can do is at .../build/sample/movie, do I have any other ways use libSGM? In short, I want to know what else can I do and I want to know more functions of libSGM.

The function of libSGM is that fast outputting disparity map from stereo images using NVIDIA GPU.
If we earned disparity map, we can use it for Object Detection, 3D reconstruction and other applications.
libSGM has several parameters, a wrapper for inputting opencv images (cv::Mat, cv::GpuMat) so that we can use Semi-Global Matching easily.

  1. I run the sample by the images provided on the http://www.6d-vision.com/scene-labeling. I found it use 1000 images in only 14 seconds, so I want to know how many images do I need every second is appropriate.

We can use sample/benchmark to find out how many images do libSGM can processes every second in a configuration.

  1. In README.md part, there is a Test Execution. I don't know what it's for, can you provide me a better explanation?

Tests are mostly prepared for developers.
If tests exist, we can

CZH-cpu commented 3 years ago

Thanks for replying!

After run the sample by the images from http://www.6d-vision.com/scene-labeling , I'm trying to use my own , but some errors happened.

Here is my initial image. IMG_8969

At first I try to divide the image and transform them into grayscale , the terminal told me that

cuda error /media/czh/8cdf51d2-2831-4793-9fd9-5a20cbf0b245/libSGM-master/src/device_buffer.hpp : 69 out of memory

so I think maybe the image is to big for my computer , than I compressed my images and run the libSGM successfully , but the depth map is wrong , it's all blue .

I don't know how to transform the image correctly , so can you tell me the right way and transform my initial image for me?

Thanks a lot , hope for you replying again.

atakagi-fixstars commented 3 years ago

Hi, @CZH-cpu

In stereo matching, an input image pair need to be "rectified" (i.e. corresponding points must be on the same y-axis), but your images are not. I manually rectified (shifted the right image up by 6px), then got reasonable result.

# Using ImageMagick
convert 109918086-68ac0a80-7cf1-11eb-8c0d-b55c26ff85c1.PNG -crop 1920x1080+0+0 -gravity NorthWest left.i63.png
convert 109918086-68ac0a80-7cf1-11eb-8c0d-b55c26ff85c1.PNG -crop 1920x1080+1920+0 -gravity NorthWest right.i63.png
convert left.i63.png -gravity North -crop 1920x1074+0+0 left.i63.6.png
convert right.i63.png -gravity North -crop 1920x1074+0+6 right.i63.6.png
convert left.i63.6.png -resize 1280x720 left.i63.6.small.png
convert right.i63.6.png -resize 1280x720 right.i63.6.small.png
./sample/image/stereo_test ./left.i63.6.small.png ./right.i63.6.small.png

disparity_color i63 6 small

CZH-cpu commented 3 years ago

Hi, @CZH-cpu

In stereo matching, an input image pair need to be "rectified" (i.e. corresponding points must be on the same y-axis), but your images are not. I manually rectified (shifted the right image up by 6px), then got reasonable result.

# Using ImageMagick
convert 109918086-68ac0a80-7cf1-11eb-8c0d-b55c26ff85c1.PNG -crop 1920x1080+0+0 -gravity NorthWest left.i63.png
convert 109918086-68ac0a80-7cf1-11eb-8c0d-b55c26ff85c1.PNG -crop 1920x1080+1920+0 -gravity NorthWest right.i63.png
convert left.i63.png -gravity North -crop 1920x1074+0+0 left.i63.6.png
convert right.i63.png -gravity North -crop 1920x1074+0+6 right.i63.6.png
convert left.i63.6.png -resize 1280x720 left.i63.6.small.png
convert right.i63.6.png -resize 1280x720 right.i63.6.small.png
./sample/image/stereo_test ./left.i63.6.small.png ./right.i63.6.small.png

disparity_color i63 6 small

Thanks for you answer!

  1. After that I tried the codes. When it comes to convert right.i63.png -gravity North -crop 1920x1074+0+6 right.i63.6.png , some errors happened. It told me convert-im6.q16: geometry does not contain imager.png' @warning/transform.c/CropImage/666.`. It makes me confused, please tell me how to fix it.

  2. Since my image didn't be rectified, I was wondering how did you find this point , can you tell me the process ?

  3. After so many trying for libSGM, it makes me have more interests in the code. Can you tell me the right way to read and understand a big code project? I want to know how all of these wonderful things achieved by codes. And also, it 's hard for me to use cmake to compile my even simple codes, can you provide me some method for learning? Since cuda and C++ are also applied in this project, can you also guide me the way for learning?

With my best wishes! Expect you replying again!

CZH-cpu commented 3 years ago

Thanks for you answer!

After that I tried the codes. When it comes to convert right.i63.png -gravity North -crop 1920x1074+0+6 right.i63.6.png , some errors happened. It told me convert-im6.q16: geometry does not contain image r.png' @warning/transform.c/CropImage/666.`. It makes me confused, please tell me how to fix it.

Since my image didn't be rectified, I was wondering how did you find this point , can you tell me the process ?

After so many trying for libSGM, it makes me have more interests in the code. Can you tell me the right way to read and understand a big code project? I want to know how all of these wonderful things achieved by codes. And also, it 's hard for me to use cmake to compile my even simple codes, can you provide me some method for learning? Since cuda and C++ are also applied in this project, can you also guide me the way for learning?

With my best wishes! Expect you replying again!

atakagi-fixstars commented 3 years ago

It told me convert-im6.q16: geometry does not contain image r.png' @warning/transform.c/CropImage/666.`.

Try adding -page +0+0.

convert right.i63.png -page +0+0 -gravity North -crop 1920x1074+0+6 right.i63.6.png

Since my image didn't be rectified, I was wondering how did you find this point , can you tell me the process ?

There is a vehicle-like object at the center of image. It seems the right one slightly lower than left.

Can you tell me the right way to read and understand a big code project?

In my personal opinion, it requires a certain amount of experience of software development.

And also, it 's hard for me to use cmake to compile my even simple codes, can you provide me some method for learning?

CMake is difficult for me too 🤣

Hou about reading CMake Tutorial? https://cmake.org/cmake/help/latest/guide/tutorial/index.html

Good luck.

CZH-cpu commented 3 years ago

Thank you for your reply

After obtaining the depth map, we should do 3D reconstruction. Does libsgm have the function of 3D reconstruction? If yes, please show me how to operate. If not, can you recommend some open source algorithms to me?

Looking forward to your reply again!

atakagi-fixstars commented 3 years ago

@CZH-cpu

Does libsgm have the function of 3D reconstruction?

Though it's not libsgm's function, the sample/reprojection perform 3D reconstruction. See the sample code.

CZH-cpu commented 3 years ago

Thank you for your reply, but now I have a new problem

In my recent study, I found that both the formation of disparity map and 3D reconstruction need the internal and external parameters of the camera, so I am very curious. Why can the executable file provided in / sample form disparity map without these parameters?

At the same time, I am also curious about how the internal and external parameters of the camera are involved in the calculation and program. If you can, can you provide me with some tutorials?

Thank you very much for your busy schedule to answer my question, looking forward to your reply again!