search

rpng / open_vins

An open source platform for visual-inertial navigation research.
https://docs.openvins.com
GNU General Public License v3.0
2.17k stars 636 forks source link

TrackAruco fails with OpenCV 4.7.0 #316

Closed gorghino closed 11 months ago

gorghino commented 1 year ago

Hi! I built OpenCV 4.7 from the sources (with contrib) but OpenVINS build fails if ENABLE_ARUCO_TAGS is ON:

nvidia@ubuntu:~/wks_openVINS$ colcon build --cmake-args -DCMAKE_BUILD_TYPE=Debug --event-handlers console_cohesion+ --packages-select ov_core ov_init ov_msckf ov_eval # ROS2 with verbose output
Starting >>> ov_core 
[Processing: ov_core]                             
[Processing: ov_core]                                     
[Processing: ov_core]                                       
--- output: ov_core                                         
-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Found CUDA: /usr/local/cuda-11.4 (found suitable exact version "11.4") 
-- Found OpenCV: /usr/local (found suitable version "4.7.0", minimum required is "4") 
-- Found Boost: /usr/lib/aarch64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake (found version "1.71.0") found components: system filesystem thread date_time 
-- OPENCV: 4.7.0 | BOOST: 1.71.0
-- PYTHON VERSION: 3.8.10
-- PYTHON INCLUDE: /usr/include/python3.8
-- PYTHON LIBRARIES: /usr/lib/aarch64-linux-gnu/libpython3.8.so
-- Found Python3: /usr/bin/python3.8 (found version "3.8.10") found components: Interpreter 
-- ROS *2* version found, building ROS2.cmake
-- Found rclcpp: 16.0.3 (/home/nvidia/ros2_humble/install/share/rclcpp/cmake)
-- Found rosidl_generator_c: 3.1.4 (/home/nvidia/ros2_humble/install/share/rosidl_generator_c/cmake)
-- Found rosidl_adapter: 3.1.4 (/home/nvidia/ros2_humble/install/share/rosidl_adapter/cmake)
-- Found rosidl_generator_cpp: 3.1.4 (/home/nvidia/ros2_humble/install/share/rosidl_generator_cpp/cmake)
-- Using all available rosidl_typesupport_c: rosidl_typesupport_fastrtps_c;rosidl_typesupport_introspection_c
-- Using all available rosidl_typesupport_cpp: rosidl_typesupport_fastrtps_cpp;rosidl_typesupport_introspection_cpp
-- Found rmw_implementation_cmake: 6.1.1 (/home/nvidia/ros2_humble/install/share/rmw_implementation_cmake/cmake)
-- Found rmw_fastrtps_cpp: 6.2.2 (/home/nvidia/ros2_humble/install/share/rmw_fastrtps_cpp/cmake)
-- Found OpenSSL: /usr/lib/aarch64-linux-gnu/libcrypto.so (found version "1.1.1f")  
-- Found FastRTPS: /home/nvidia/ros2_humble/install/include  
-- Using RMW implementation 'rmw_fastrtps_cpp' as default
-- Found cv_bridge: 3.2.1 (/home/nvidia/ros2_humble/install/share/cv_bridge/cmake)
-- Configuring done
-- Generating done
-- Build files have been written to: /home/nvidia/wks_openVINS/build/ov_core
[ 10%] Building CXX object CMakeFiles/ov_core_lib.dir/src/cpi/CpiV1.cpp.o
[ 10%] Building CXX object CMakeFiles/ov_core_lib.dir/src/dummy.cpp.o
[ 21%] Building CXX object CMakeFiles/ov_core_lib.dir/src/cpi/CpiV2.cpp.o
[ 21%] Building CXX object CMakeFiles/ov_core_lib.dir/src/sim/BsplineSE3.cpp.o
[ 26%] Building CXX object CMakeFiles/ov_core_lib.dir/src/track/TrackAruco.cpp.o
[ 31%] Building CXX object CMakeFiles/ov_core_lib.dir/src/track/TrackBase.cpp.o
[ 36%] Building CXX object CMakeFiles/ov_core_lib.dir/src/track/TrackDescriptor.cpp.o
In file included from /home/nvidia/wks_openVINS/src/open_vins/ov_core/src/track/TrackAruco.cpp:22:
/home/nvidia/wks_openVINS/src/open_vins/ov_core/src/track/TrackAruco.h: In constructor ‘ov_core::TrackAruco::TrackAruco(std::unordered_map<long unsigned int, std::shared_ptr<ov_core::CamBase> >, int, bool, ov_core::TrackBase::HistogramMethod, bool)’:
/home/nvidia/wks_openVINS/src/open_vins/ov_core/src/track/TrackAruco.h:58:77: error: no match for ‘operator=’ (operand types are ‘cv::Ptr<cv::aruco::Dictionary>’ and ‘cv::aruco::Dictionary’)
   58 |     aruco_dict = cv::aruco::getPredefinedDictionary(cv::aruco::DICT_6X6_1000);
      |                                                                             ^
In file included from /usr/local/include/opencv4/opencv2/core/cvstd.hpp:81,
                 from /usr/local/include/opencv4/opencv2/core/base.hpp:58,
                 from /usr/local/include/opencv4/opencv2/core.hpp:53,
                 from /usr/local/include/opencv4/opencv2/objdetect/aruco_dictionary.hpp:7,
                 from /usr/local/include/opencv4/opencv2/objdetect/aruco_detector.hpp:7,
                 from /usr/local/include/opencv4/opencv2/aruco.hpp:7,
                 from /home/nvidia/wks_openVINS/src/open_vins/ov_core/src/track/TrackAruco.h:26,
                 from /home/nvidia/wks_openVINS/src/open_vins/ov_core/src/track/TrackAruco.cpp:22:
/usr/local/include/opencv4/opencv2/core/cvstd_wrapper.hpp:121:10: note: candidate: ‘cv::Ptr<T>& cv::Ptr<T>::operator=(const cv::Ptr<T>&) [with T = cv::aruco::Dictionary]’
  121 |     Ptr& operator=(const Ptr& o) { std::shared_ptr<T>::operator =(o); return *this; }
      |          ^~~~~~~~
/usr/local/include/opencv4/opencv2/core/cvstd_wrapper.hpp:121:31: note:   no known conversion for argument 1 from ‘cv::aruco::Dictionary’ to ‘const cv::Ptr<cv::aruco::Dictionary>&’
  121 |     Ptr& operator=(const Ptr& o) { std::shared_ptr<T>::operator =(o); return *this; }
      |                    ~~~~~~~~~~~^
/usr/local/include/opencv4/opencv2/core/cvstd_wrapper.hpp:122:38: note: candidate: ‘template<class Y> cv::Ptr<T>& cv::Ptr<T>::operator=(const cv::Ptr<Y>&) [with Y = Y; T = cv::aruco::Dictionary]’
  122 |     template<typename Y> inline Ptr& operator=(const Ptr<Y>& o) { std::shared_ptr<T>::operator =(o); return *this; }
      |                                      ^~~~~~~~
/usr/local/include/opencv4/opencv2/core/cvstd_wrapper.hpp:122:38: note:   template argument deduction/substitution failed:
In file included from /home/nvidia/wks_openVINS/src/open_vins/ov_core/src/track/TrackAruco.cpp:22:
/home/nvidia/wks_openVINS/src/open_vins/ov_core/src/track/TrackAruco.h:58:77: note:   ‘cv::aruco::Dictionary’ is not derived from ‘const cv::Ptr<_Tp>’
   58 |     aruco_dict = cv::aruco::getPredefinedDictionary(cv::aruco::DICT_6X6_1000);
      |                                                                             ^
/home/nvidia/wks_openVINS/src/open_vins/ov_core/src/track/TrackAruco.h:59:51: error: ‘create’ is not a member of ‘cv::aruco::DetectorParameters’
   59 |     aruco_params = cv::aruco::DetectorParameters::create();
      |                                                   ^~~~~~
make[2]: *** [CMakeFiles/ov_core_lib.dir/build.make:146: CMakeFiles/ov_core_lib.dir/src/track/TrackAruco.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [CMakeFiles/Makefile2:141: CMakeFiles/ov_core_lib.dir/all] Error 2
make: *** [Makefile:146: all] Error 2
---
--- stderr: ov_core
In file included from /home/nvidia/wks_openVINS/src/open_vins/ov_core/src/track/TrackAruco.cpp:22:
/home/nvidia/wks_openVINS/src/open_vins/ov_core/src/track/TrackAruco.h: In constructor ‘ov_core::TrackAruco::TrackAruco(std::unordered_map<long unsigned int, std::shared_ptr<ov_core::CamBase> >, int, bool, ov_core::TrackBase::HistogramMethod, bool)’:
/home/nvidia/wks_openVINS/src/open_vins/ov_core/src/track/TrackAruco.h:58:77: error: no match for ‘operator=’ (operand types are ‘cv::Ptr<cv::aruco::Dictionary>’ and ‘cv::aruco::Dictionary’)
   58 |     aruco_dict = cv::aruco::getPredefinedDictionary(cv::aruco::DICT_6X6_1000);
      |                                                                             ^
In file included from /usr/local/include/opencv4/opencv2/core/cvstd.hpp:81,
                 from /usr/local/include/opencv4/opencv2/core/base.hpp:58,
                 from /usr/local/include/opencv4/opencv2/core.hpp:53,
                 from /usr/local/include/opencv4/opencv2/objdetect/aruco_dictionary.hpp:7,
                 from /usr/local/include/opencv4/opencv2/objdetect/aruco_detector.hpp:7,
                 from /usr/local/include/opencv4/opencv2/aruco.hpp:7,
                 from /home/nvidia/wks_openVINS/src/open_vins/ov_core/src/track/TrackAruco.h:26,
                 from /home/nvidia/wks_openVINS/src/open_vins/ov_core/src/track/TrackAruco.cpp:22:
/usr/local/include/opencv4/opencv2/core/cvstd_wrapper.hpp:121:10: note: candidate: ‘cv::Ptr<T>& cv::Ptr<T>::operator=(const cv::Ptr<T>&) [with T = cv::aruco::Dictionary]’
  121 |     Ptr& operator=(const Ptr& o) { std::shared_ptr<T>::operator =(o); return *this; }
      |          ^~~~~~~~
/usr/local/include/opencv4/opencv2/core/cvstd_wrapper.hpp:121:31: note:   no known conversion for argument 1 from ‘cv::aruco::Dictionary’ to ‘const cv::Ptr<cv::aruco::Dictionary>&’
  121 |     Ptr& operator=(const Ptr& o) { std::shared_ptr<T>::operator =(o); return *this; }
      |                    ~~~~~~~~~~~^
/usr/local/include/opencv4/opencv2/core/cvstd_wrapper.hpp:122:38: note: candidate: ‘template<class Y> cv::Ptr<T>& cv::Ptr<T>::operator=(const cv::Ptr<Y>&) [with Y = Y; T = cv::aruco::Dictionary]’
  122 |     template<typename Y> inline Ptr& operator=(const Ptr<Y>& o) { std::shared_ptr<T>::operator =(o); return *this; }
      |                                      ^~~~~~~~
/usr/local/include/opencv4/opencv2/core/cvstd_wrapper.hpp:122:38: note:   template argument deduction/substitution failed:
In file included from /home/nvidia/wks_openVINS/src/open_vins/ov_core/src/track/TrackAruco.cpp:22:
/home/nvidia/wks_openVINS/src/open_vins/ov_core/src/track/TrackAruco.h:58:77: note:   ‘cv::aruco::Dictionary’ is not derived from ‘const cv::Ptr<_Tp>’
   58 |     aruco_dict = cv::aruco::getPredefinedDictionary(cv::aruco::DICT_6X6_1000);
      |                                                                             ^
/home/nvidia/wks_openVINS/src/open_vins/ov_core/src/track/TrackAruco.h:59:51: error: ‘create’ is not a member of ‘cv::aruco::DetectorParameters’
   59 |     aruco_params = cv::aruco::DetectorParameters::create();
      |                                                   ^~~~~~
make[2]: *** [CMakeFiles/ov_core_lib.dir/build.make:146: CMakeFiles/ov_core_lib.dir/src/track/TrackAruco.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [CMakeFiles/Makefile2:141: CMakeFiles/ov_core_lib.dir/all] Error 2
make: *** [Makefile:146: all] Error 2
---
Failed   <<< ov_core [1min 48s, exited with code 2]

Summary: 0 packages finished [1min 49s]
  1 package failed: ov_core
  1 package had stderr output: ov_core
  3 packages not processed

I think it's a problem related to the aruco library that is not in contrib anymore. I think it's in opencv2/objdetect module now. Adding

#include <opencv2/objdetect/aruco_detector.hpp>
#include <opencv2/objdetect/aruco_dictionary.hpp>

in TrackAruco.h doesn't fix the problem.

goldbattle commented 1 year ago

I need to spend some time to test out both Ubuntu 22.02 and the newer OpenCV to get it to support all the version. If you are able to address it, feel free to post the solution (or link to it).

gorghino commented 1 year ago

Hi @goldbattle , I'm using Ubuntu 20.04 but I'll tell you if I find the solution. Thank you!

ZibYang commented 1 year ago

The following solution works for me: start from ov_core/track/TrackAruco.h line 106

  1. change cv::Ptr<cv::aruco::Dictionary> aruco_dict; to cv::aruco::Dictionary aruco_dict;
  2. change cv::Ptr<cv::aruco::DetectorParameters> aruco_params; to cv::aruco::DetectorParameters aruco_params;
  3. add cv::aruco::ArucoDetector aruco_detector; below

Also in ov_core/track/TrackAruco.cpp line 97 change cv::aruco::detectMarkers(img0, aruco_dict, corners[cam_id], ids_aruco[cam_id], aruco_params, rejects[cam_id]); to aruco_detector.detectMarkers(img0, corners[cam_id], ids_aruco[cam_id], rejects[cam_id]);

gorghino commented 1 year ago

Hi @ZibYang Your fix removed Ptr errors but not this one:

Starting >>> ov_core 
Starting >>> ov_data                  
Finished <<< ov_data [1.45s]                                                        
--- stderr: ov_core                              
In file included from /home/nvidia/ws_openVINS/src/open_vins/ov_core/src/track/TrackAruco.cpp:22:
/home/nvidia/ws_openVINS/src/open_vins/ov_core/src/track/TrackAruco.h: In constructor ‘ov_core::TrackAruco::TrackAruco(std::unordered_map<long unsigned int, std::shared_ptr<ov_core::CamBase> >, int, bool, ov_core::TrackBase::HistogramMethod, bool)’:
/home/nvidia/ws_openVINS/src/open_vins/ov_core/src/track/TrackAruco.h:59:51: error: ‘create’ is not a member of ‘cv::aruco::DetectorParameters’
   59 |     aruco_params = cv::aruco::DetectorParameters::create();
      |                                                   ^~~~~~
make[2]: *** [CMakeFiles/ov_core_lib.dir/build.make:146: CMakeFiles/ov_core_lib.dir/src/track/TrackAruco.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:141: CMakeFiles/ov_core_lib.dir/all] Error 2
make: *** [Makefile:146: all] Error 2
---
Failed   <<< ov_core [24.7s, exited with code 2]

Summary: 1 package finished [25.9s]
  1 package failed: ov_core
  1 package had stderr output: ov_core
  3 packages not processed
goldbattle commented 1 year ago

I believe you can just remove the line and it will use the default constructor: https://github.com/opencv/opencv/blob/86fa0308fc17f982cf9aa291b5e53e7b9e71e0be/modules/objdetect/include/opencv2/objdetect/aruco_detector.hpp#L23-L59

It looks like you will need to make a ArucoDetector object (untested code, just a guess here):

aruco::Dictionary aruco_dict = aruco::getPredefinedDictionary(aruco::DICT_6X6_1000);
aruco::DetectorParameters aruco_params;
aruco_params.cornerRefinementMethod = cv::aruco::CORNER_REFINE_SUBPIX;
aruco_detector = aruco::ArucoDetector(aruco_dict, aruco_params);

and use it as:

aruco_detector.detectMarkers(img0, corners[cam_id], ids_aruco[cam_id], rejects[cam_id]);
ZibYang commented 1 year ago

Hi @ZibYang Your fix removed Ptr errors but not this one:

Starting >>> ov_core 
Starting >>> ov_data                  
Finished <<< ov_data [1.45s]                                                        
--- stderr: ov_core                              
In file included from /home/nvidia/ws_openVINS/src/open_vins/ov_core/src/track/TrackAruco.cpp:22:
/home/nvidia/ws_openVINS/src/open_vins/ov_core/src/track/TrackAruco.h: In constructor ‘ov_core::TrackAruco::TrackAruco(std::unordered_map<long unsigned int, std::shared_ptr<ov_core::CamBase> >, int, bool, ov_core::TrackBase::HistogramMethod, bool)’:
/home/nvidia/ws_openVINS/src/open_vins/ov_core/src/track/TrackAruco.h:59:51: error: ‘create’ is not a member of ‘cv::aruco::DetectorParameters’
   59 |     aruco_params = cv::aruco::DetectorParameters::create();
      |                                                   ^~~~~~
make[2]: *** [CMakeFiles/ov_core_lib.dir/build.make:146: CMakeFiles/ov_core_lib.dir/src/track/TrackAruco.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:141: CMakeFiles/ov_core_lib.dir/all] Error 2
make: *** [Makefile:146: all] Error 2
---
Failed   <<< ov_core [24.7s, exited with code 2]

Summary: 1 package finished [25.9s]
  1 package failed: ov_core
  1 package had stderr output: ov_core
  3 packages not processed

Sorry about that. You should also change line 59 at ov_core/track/TrackAruco.h from aruco_params = cv::aruco::DetectorParameters::create(); to aruco_params = cv::aruco::DetectorParameters(); And I believe that error will disappear~ enjoy your day!

ps. My reference is from: https://docs.opencv.org/4.x/d5/dae/tutorial_aruco_detection.html