search

raulmur / ORB_SLAM2

Real-Time SLAM for Monocular, Stereo and RGB-D Cameras, with Loop Detection and Relocalization Capabilities
Other
9.37k stars 4.69k forks source link

Patching ORB SLAM2 to OpenCV 3.1.0. #183

Open OtacilioNeto opened 7 years ago

OtacilioNeto commented 7 years ago

I'm patching ORB SLAM2 to use OpenCV 3.1.0. The patch is almost done except by a issue that until now I'm unable to fix alone. I'm using clang 3.9 on FreeBSD 11. All times, when I try compile ORB SLAM I getting this error:

root@nostromo:/usr/ports/science/orbslam2 # make ===> Building for OrbSlam2-1.0.0 /usr/local/bin/cmake -H/usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b -B/usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b --check-build-system CMakeFiles/Makefile.cmake 0 /usr/local/bin/cmake -E cmake_progress_start /usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b/CMakeFiles /usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b/CMakeFiles/progress.marks /usr/bin/make -f CMakeFiles/Makefile2 all --- CMakeFiles/ORB_SLAM2.dir/all --- /usr/bin/make -f CMakeFiles/ORB_SLAM2.dir/build.make CMakeFiles/ORB_SLAM2.dir/depend --- CMakeFiles/ORB_SLAM2.dir/depend --- cd /usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b && /usr/local/bin/cmake -E cmake_depends "Unix Makefiles" /usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b /usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b /usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b /usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b /usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b/CMakeFiles/ORB_SLAM2.dir/DependInfo.cmake --color= /usr/bin/make -f CMakeFiles/ORB_SLAM2.dir/build.make CMakeFiles/ORB_SLAM2.dir/build --- CMakeFiles/ORB_SLAM2.dir/src/System.cc.o --- --- CMakeFiles/ORB_SLAM2.dir/src/Tracking.cc.o --- [ 3%] Building CXX object CMakeFiles/ORB_SLAM2.dir/src/Tracking.cc.o /usr/bin/c++ -DCOMPILEDWITHC11 -DORB_SLAM2_EXPORTS -I/usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b -I/usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b/include -I/usr/local/include/eigen3 -I/usr/local/lib/cmake/Pangolin/../../../include -isystem /usr/local/include -isystem /usr/local/include/opencv -O2 -pipe -fstack-protector -fno-strict-aliasing -Wall -O3 -march=native -std=c++11 -O2 -pipe -fstack-protector -fno-strict-aliasing -fPIC -o CMakeFiles/ORB_SLAM2.dir/src/Tracking.cc.o -c /usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b/src/Tracking.cc --- CMakeFiles/ORB_SLAM2.dir/src/System.cc.o --- [ 6%] Building CXX object CMakeFiles/ORB_SLAM2.dir/src/System.cc.o /usr/bin/c++ -DCOMPILEDWITHC11 -DORB_SLAM2_EXPORTS -I/usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b -I/usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b/include -I/usr/local/include/eigen3 -I/usr/local/lib/cmake/Pangolin/../../../include -isystem /usr/local/include -isystem /usr/local/include/opencv -O2 -pipe -fstack-protector -fno-strict-aliasing -Wall -O3 -march=native -std=c++11 -O2 -pipe -fstack-protector -fno-strict-aliasing -fPIC -o CMakeFiles/ORB_SLAM2.dir/src/System.cc.o -c /usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b/src/System.cc --- CMakeFiles/ORB_SLAM2.dir/src/Tracking.cc.o --- In file included from /usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b/src/Tracking.cc:22: In file included from /usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b/include/Tracking.h:25: In file included from /usr/local/lib/cmake/Pangolin/../../../include/opencv2/core/core.hpp:48: In file included from /usr/local/lib/cmake/Pangolin/../../../include/opencv2/core.hpp:3165: /usr/local/lib/cmake/Pangolin/../../../include/opencv2/core/utility.hpp:362:47: error: reinterpret_cast from 'nullptrt' to 'int ' is not allowed operation(_reinterpret_cast<_Tp_>(0), reinterpretcast<int>(NULL)); ^~~~~~~~ --- CMakeFiles/ORB_SLAM2.dir/src/System.cc.o --- In file included from /usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b/src/System.cc:23: In file included from /usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b/include/System.h:27: In file included from /usr/local/lib/cmake/Pangolin/../../../include/opencv2/core/core.hpp:48: In file included from /usr/local/lib/cmake/Pangolin/../../../include/opencv2/core.hpp:3165: /usr/local/lib/cmake/Pangolin/../../../include/opencv2/core/utility.hpp:362:47: error: reinterpret_cast from 'nullptrt' to 'int ' is not allowed operation(_reinterpret_cast<_Tp_>(0), reinterpretcast<int>(NULL)); ^~~~~~~~ 1 error generated. *\ [CMakeFiles/ORB_SLAM2.dir/src/System.cc.o] Error code 1

make[3]: stopped in /usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b --- CMakeFiles/ORB_SLAM2.dir/src/Tracking.cc.o --- 1 error generated. *\ [CMakeFiles/ORB_SLAM2.dir/src/Tracking.cc.o] Error code 1

make[3]: stopped in /usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b 2 errors

make[3]: stopped in /usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b *\ [CMakeFiles/ORB_SLAM2.dir/all] Error code 2

make[2]: stopped in /usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b 1 error

make[2]: stopped in /usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b *\ [all] Error code 2

make[1]: stopped in /usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b 1 error

make[1]: stopped in /usr/ports/science/orbslam2/work/ORB_SLAM2-1a3bc4b ===> Compilation failed unexpectedly. Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to the maintainer. *\ Error code 1

Stop. make: stopped in /usr/ports/science/orbslam2

Someone can give me a hint about it?

xuyongzhi commented 7 years ago

I also want to use opencv3.1, but I do not figure out the key difficults of transforming from 2 to 3 in ORB_SLAM2?

OtacilioNeto commented 7 years ago

Yesterday I did a full compile of ORB SLAM 2 with OpenCV 3.1 but using a ugly workaround to this issue. And also, when using gcc 4.8 this issue is not triggered, only with clang.

OtacilioNeto commented 7 years ago

The main issue is that ORBSLAM comes with a version of DBoW2 that uses OpenCV 2.4.X. I did a modification to use https://github.com/dorian3d/DBoW2/releases/tag/v1.1-nonfree. Another minors modifications about sdt:: are necessary.

AlejandroSilvestri commented 7 years ago

I simply replaced

#find_package(OpenCV 2.4.3 REQUIRED)
find_package(OpenCV 3.1.0 REQUIRED)

in CMakeLists.txt, and worked.

Now I compile orb-slam2 in eclipse cdt, modified to use the webcam, working ok with opencv 3.1.0.

ghost commented 7 years ago

Is orbslam faster with opencv 3? Or more accurate?

OtacilioNeto commented 7 years ago

I do not have tested speed or precision because my kitti dataset is in a computer in the shop for repair. This patch is not for Opencv3 only, but also for the latest version of Pangolin (v0.5) and DBoW2 (v1.1-nonfree) that needs OpenCV 3 with nonfree modules enabled.

OtacilioNeto commented 7 years ago

About only change find_package(OpenCV 2.4.3 REQUIRED) to find_package(OpenCV 3.1.0 REQUIRED) I think that it work for you because you are using a old version of Pangolin (v0.3) or DBoW2. When using DBoW2 v1.1-nonfree and Pangolin v0.5, in my system, I got a lot of issues related with namespace std. Because of this I did this patch.

OtacilioNeto commented 7 years ago

My patch is in this pull request https://github.com/raulmur/ORB_SLAM2/pull/184

AlejandroSilvestri commented 7 years ago

I don't think it's faster with opencv 3.1.0. orb-slam only uses FAST algorithm from opencv (besides opencv classes like Mat, Point, and so on), it seems unchanged from 2.4.

OtacilioNeto commented 7 years ago

Yes. I think that the main purpose to update to OpenCV 3.1.0 is that the OpenCV 3 series has many new algorithms that don't will be backported to series 2.4.X. And for many systems is not possible easily maintain two versions of OpenCV. FreeBSD, for example, has started the migration to OpenCV 3.1.0.

cambazz commented 7 years ago

Hello, I am interested on this too. I am on ubuntu 16 with ros-kinetic, and I followed the build instructions and this is where I got stuck - so basically ubuntu 16 with ros-kinetic has opencv 3 and the build script searches for opencv 2.4.3

CMake Error at CMakeLists.txt:31 (find_package): Could not find a configuration file for package "OpenCV" that is compatible with requested version "2.4.3". The following configuration files were considered but not accepted: /opt/ros/kinetic/share/OpenCV-3.1.0-dev/OpenCVConfig.cmake, version: 3.1.0

can we use this patch and if so how?

best regards, ~c

AlejandroSilvestri commented 7 years ago

Like I said above, in CmakeLists.txt replace

find_package(OpenCV 2.4.3 REQUIRED)

by

find_package(OpenCV 3.1.0 REQUIRED)
redheli commented 7 years ago

@AlejandroSilvestri Thank you. compiled ORB_SLAM2 with opencv3.1 and run ros node in ros-kinect.

AlejandroSilvestri commented 7 years ago

I think Cmakelists.txt should replace 

find_package(OpenCV 2.4.3 REQUIRED)

by

find_package(OpenCV 2.4.3)
if(NOT OpenCV)
  find_package(OpenCV 3.0.1 REQUIRED)
endif()

How can we propose this update to Raul Mur?

OtacilioNeto commented 7 years ago

I think that using a pull request? For me it is not clear that the only requested modification is this because the orbslam uses pangolin. Update the version of opencv used by orbslam and not update the version used by pangolin will force the system to uses two versions of opencv and many systems do not support this. I think that now the the best think is Raul Mur say what he think about this. So, we can do a more realistic and directed work.

AlejandroSilvestri commented 7 years ago

This modification worked for me, with only opencv 3.1 in my system. I don't think Pangolin uses opencv.

AlejandroSilvestri commented 7 years ago

OpenCV is used only in ORB_SLAM2 namespace (only in src and includes folders), not in any thirdparty software.

orb-slam2 uses:

All of these are compatible with opencv 2.4.3+ and 3.x I thank any contributions to this list.

AlejandroSilvestri commented 7 years ago

It's done!

Raúl Mur commited the patch 2 days ago, adding compatibility with opencv 3 and Eigen 3.3.

ameysutavani commented 7 years ago

@AlejandroSilvestri

Now I compile orb-slam2 in eclipse cdt, modified to use the webcam, working ok with opencv 3.1.0.

https://github.com/raulmur/ORB_SLAM2/issues/183#issuecomment-256766686

I am also trying to run orb-slam2 (monocular) with a webcam. Can you tell what code changes to make for that?

AlejandroSilvestri commented 7 years ago

Well, many find it easier with ROS, which I'm not using.

In my code main.cc (which correspond to monocular example in orb-slam2) there's a main loop feeding images to orb-slam with this line:

SLAM.TrackMonocular(im,0);

Im is the actual image. Instead of reading it from a file, I take it from the webcam with videoEntrada->read(im);

I use opencv videoCapture to open the webcam or video file and grab images.

ameysutavani commented 7 years ago

@AlejandroSilvestri I too have found ROS to be only helpful when it is setup correctly; else it just induces additional debugging in the process.

I was able to get ORBSLAM2 running using webcam with your additions, thanks.