search

yan99033 / CNN-SVO

Real-time motion from structure
331 stars 91 forks source link

run roslaunch in online model #3

Closed zxp771 closed 5 years ago

zxp771 commented 5 years ago

Hi Thanks for sharing! I have a problem when I run the kittiOnline00-02Online.launch. I have tested the library is successfully built by running the inference_monodepth executable. but when I run the .launch file it will show this error. [ INFO] [1546542125.339557424]: Found parameter: svo/cam_model, value: Pinhole [ INFO] [1546542125.339840655]: Found parameter: svo/cam_width, value: 1232 [ INFO] [1546542125.340118023]: Found parameter: svo/cam_height, value: 376 [ INFO] [1546542125.340465069]: Found parameter: svo/cam_fx, value: 718.856 [ INFO] [1546542125.340864351]: Found parameter: svo/cam_fy, value: 718.856 [ INFO] [1546542125.341153789]: Found parameter: svo/cam_cx, value: 607.193 [ INFO] [1546542125.341452130]: Found parameter: svo/cam_cy, value: 185.216 [ INFO] [1546542125.341726113]: Found parameter: svo/cam_d0, value: 0 [ INFO] [1546542125.342016760]: Found parameter: svo/cam_d1, value: 0 [ INFO] [1546542125.342266660]: Found parameter: svo/cam_d2, value: 0 [ INFO] [1546542125.342588759]: Found parameter: svo/cam_d3, value: 0 ================================================================================REQUIRED process [svo-1] has died! process has died [pid 7880, exit code -11, cmd /home/zxp/catkin_ws/devel/lib/svo_ros/vo_monodepth_online name:=svo log:=/home/zxp/.ros/log/d649ff14-0f77-11e9-bb00-f43909319a96/svo-1.log]. log file: /home/zxp/.ros/log/d649ff14-0f77-11e9-bb00-f43909319a96/svo-1*.log Initiating shutdown!

Could you give me some suggestion to figure out the problem? Thanks

yan99033 commented 5 years ago

In general, 'exit code -11' is raised because the program is trying to access an invalid pointer address. There could be a number of possible explanations for the 'exit code -11':

  1. The program is not reading the images properly. Check if the colour images are provided to the program
  2. Incompatible OpenCV version. Perform the following steps and see if it solves the error code i. Git pull the cv_bridge repo to your catkin workspace (e.g., ~/catkin_ws) ii. Check out the matching ROS version (e.g., Kinetic) iii. do a catkin_make
  3. Disable g2o (incompatible Eigen version can result in program termination)

If none of the suggestions works, you will need to debug the program using Valgrind or other memcheck tools. To do so, you need to modify the roslaunch file such that it launches the memcheck during the program execution. <node pkg="svo_ros" type="vo_monodepth_online" name="svo" clear_params="true" output="screen" required="true" launch-prefix="valgrind"> However, sometimes you cannot catch the error because SVO is such a huge program, that the debugging slows down the system tremendously.

Good luck!

zxp771 commented 5 years ago

Hi,@yan99033 Thanks for your help. I use the cv_bridge in the ROS env. could you tell me which version the opencv you use? and I already disable the G2O in my code. I will try other methods you suggested. Thanks!

yan99033 commented 5 years ago

image

That's what I got from dpkg -l | grep opencv

yan99033 commented 5 years ago

You can also check the issues in the original SVO github page.

zxp771 commented 5 years ago

Hi @yan99033 I check my opencv it's the same. I think the error may be caused by others. Thanks for your help and sharing! I will try those methods.

yan99033 commented 5 years ago

Hi @yan99033 I check my opencv it's the same. I think the error may be caused by others. Thanks for your help and sharing! I will try those methods.

I would suggest you build and run the original SVO code on your machine. If the original SVO is running properly, your problem is most likely due to incompatible OpenCV version; you will need to build the _cvbridge ROS package locally.

'exit code -11' is a common issue in the original SVO code.

zxp771 commented 5 years ago

Hi @yan99033

I pull the cv_bridge repo in my workspace. but it still shows the same error: REQUIRED process [svo-1] has died! process has died [pid 2602, exit code -11, cmd /home/zxp/catkin_ws/devel/lib/svo_ros/vo_monodepth_online name:=svo log:=/home/zxp/.ros/log/0284e0a6-1051-11e9-b873-f43909319a96/svo-1.log]. log file: /home/zxp/.ros/log/0284e0a6-1051-11e9-b873-f43909319a96/svo-1*.log Initiating shutdown!

zxp771 commented 5 years ago

Hi @yan99033 Here is my result from dpkg -l | grep opencv image

zxp771 commented 5 years ago

Hi @yan99033 I added the launch-prefix="gdb -ex run --args" to the tag in the launchfile. and got the bt (backtrace) results like this: Thread 1 "vo_monodepth_on" received signal SIGSEGV, Segmentation fault. 0x0000000000000000 in ?? () (gdb) bt

0 0x0000000000000000 in ()

1 0x00007ffff780013b in cv::initUndistortRectifyMap(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, cv::InputArray const&, cv::Size, int, cv::_OutputArray const&, cv::_OutputArray const&) ()

at /home/zxp/catkin_ws/devel/lib/libopencv_imgproc.so.2.4

2 0x00007ffff643e773 in vk::PinholeCamera::PinholeCamera(double, double, double, double, double, double, double, double, double, double, double) () at /home/zxp/catkin_ws/devel/lib/libvikit_common.so

3 0x0000000000411de9 in vk::camera_loader::loadFromRosNs(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, vk::AbstractCamera*&) ()

4 0x000000000041327c in svo::VoNode::VoNode() ()

5 0x000000000040e34d in main ()

(gdb) [svo-1] killing on exit

could you give me some suggestion to fix these error? Thanks!

yan99033 commented 5 years ago

From what I can see, it is related to OpenCV version conflict. ROS Kinetic is using OpenCV 3.3 and your code is built with OpenCV 2.4.

at /home/zxp/catkin_ws/devel/lib/libopencv_imgproc.so.2.4

Did you do a clean catkin_make (i.e., delete all the files in ~/catkin_ws/devel and run catkin_make) before you run the program?

zxp771 commented 5 years ago

Hi @yan99033 Yes, I clean catkin_make before I rerun it every time. Do you mean It's not a compatible Opencv version what I used(2.4)? Should I use the kinetic's original Opencv(3.3.1)? Thanks for your patience to give me such useful advice!

zxp771 commented 5 years ago

Hi @yan99033 I tried the kinetic opencv. but the compile is failed like this: Errors << svo_ros:make /home/zxp/catkin_svo/logs/svo_ros/build.make.000.log /home/zxp/catkin_svo/devel/.private/svo/lib/libsvo.so: undefined reference to cv::_InputArray::_InputArray(cv::MatExpr const&)' collect2: error: ld returned 1 exit status make[2]: *** [/home/zxp/catkin_svo/devel/.private/svo_ros/lib/svo_ros/vo] Error 1 make[1]: *** [CMakeFiles/vo.dir/all] Error 2 make[1]: *** Waiting for unfinished jobs.... /home/zxp/catkin_svo/devel/.private/svo/lib/libsvo.so: undefined reference tocv::_InputArray::_InputArray(cv::MatExpr const&)' collect2: error: ld returned 1 exit status make[2]: [/home/zxp/catkin_svo/devel/.private/svo_ros/lib/svo_ros/vo_monodepth_online] Error 1 make[1]: [CMakeFiles/vo_monodepth_online.dir/all] Error 2 /home/zxp/catkin_svo/devel/.private/svo/lib/libsvo.so: undefined reference to `cv::_InputArray::_InputArray(cv::MatExpr const&)' collect2: error: ld returned 1 exit status make[2]: [/home/zxp/catkin_svo/devel/.private/svo_ros/lib/svo_ros/vo_monodepth_offline] Error 1 make[1]: [CMakeFiles/vo_monodepth_offline.dir/all] Error 2 make: *** [all] Error 2 It seems still caused by opencv?

zxp771 commented 5 years ago

Hi @yan99033 I tried the opencv in kinetic. I added the launch-prefix="gdb -ex run --args" to the tag in the launchfile. and got the bt (backtrace) results like this: Thread 1 "vo_monodepth_on" received signal SIGSEGV, Segmentation fault. 0x00007ffff6e4b1b2 in cv::InputArray::getMat(int) const () from /opt/ros/kinetic/lib/x86_64-linux-gnu/libopencv_core3.so.3.3 (gdb) bt '#0 0x00007ffff6e4b1b2 in cv::InputArray::getMat(int) const () at /opt/ros/kinetic/lib/x86_64-linux-gnu/libopencv_core3.so.3.3

1 0x00007ffff1bc6611 in cv::initUndistortRectifyMap(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, cv::InputArray const&, cv::Size, int, cv::_OutputArray const&, cv::_OutputArray const&) ()

at /opt/ros/kinetic/lib/x86_64-linux-gnu/libopencv_imgproc3.so.3.3

2 0x00007ffff60fc2e3 in vk::PinholeCamera::PinholeCamera(double, double, double, double, double, double, double, double, double, double, double) ()

at /home/zxp/catkin_ws/devel/lib/libvikit_common.so

3 0x0000000000411d39 in vk::camera_loader::loadFromRosNs(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, vk::AbstractCamera*&) ()

4 0x00000000004131cc in svo::VoNode::VoNode() ()

5 0x000000000040e29d in main ()

It's the same error like in version of Opencv in 2.4 version.

yan99033 commented 5 years ago

Looks like the error occurs in the following lines of code in CNN-SVO/rpg_vikit/vikit_common/src/pinhole_camera.cpp

cvK_ = (cv::Mat_<float>(3, 3) << fx_, 0.0, cx_, 0.0, fy_, cy_, 0.0, 0.0, 1.0); cvD_ = (cv::Mat_<float>(1, 5) << d_[0], d_[1], d_[2], d_[3], d_[4]); cv::initUndistortRectifyMap(cvK_, cvD_, cv::Mat_<double>::eye(3,3), cvK_, cv::Size(width_, height_), CV_16SC2, undist_map1_, undist_map2_);

I am not sure what is the exact problem. Since it is related to cv::InputArray::getMat(int), I am guessing that the problem comes from the bold arguments provided to the function:

cv::initUndistortRectifyMap(cv::_InputArray const&, cv::_InputArray const&, cv::_InputArray const&, cv::InputArray const&, cv::Size, int, cv::_OutputArray const&, cv::_OutputArray const&) ()

Try to modify these two lines

cvK_ = (cv::Mat_<float>(3, 3) << fx_, 0.0, cx_, 0.0, fy_, cy_, 0.0, 0.0, 1.0); cvD_ = (cv::Mat_<float>(1, 5) << d_[0], d_[1], d_[2], d_[3], d_[4]);

to

cv::Mat cvK_ = cv::Mat::eye(3,3,CV_32F); cvK_.at<float>(0,0) = fx_; cvK_.at<float>(1,1) = fy_; cvK_.at<float>(0,2) = cx_; cvK_.at<float>(1,2) = cy_;

cv::Mat cvD_(5,1,CV_32F); cvD_.at<float>(0) = d_[0]; cvD_.at<float>(1) = d_[1]; cvD_.at<float>(2) = d_[2]; cvD_.at<float>(3) = d_[3]; cvD_.at<float>(4) = d_[4];

and see what happens.

Again, the problem is related to the original SVO code. You might be able to find the similar issues in the original SVO github page.

zxp771 commented 5 years ago

Hi @yan99033 Thanks for your advice. I modified the code but still have nothing to change. I find the original SVO project https://github.com/uzh-rpg/rpg_svo and original vikit they use. but got nothing useful. Thanks for your help again.

zxp771 commented 5 years ago

Hi @yan99033 I find the problem and finally, I can run the test dataset. I'm wondering that if you have verified your method on the original dataset(test_rig3.launch + airground_rig_s3_2013-03-18_21-38-48.bag)?

yan99033 commented 5 years ago

Hi @yan99033 I find the problem and finally, I can run the test dataset. I'm wondering that if you have verified your method on the original dataset(test_rig3.launch + airground_rig_s3_2013-03-18_21-38-48.bag)?

Good for you! BTW, can you describe how did you solve the problem?

Our method has only been verified on KITTI and Oxford Robotcar datasets.

Thanks.

zxp771 commented 5 years ago

Hi @yan99033 Thanks for your reply. As you said before the error is caused by the Opencv.(kinetic opencv and the original opencv may have some conflict.) I go to this( https://github.com/ethz-asl/opencv2_catkin )to get the convenience wrapper that handles the include and link for opencv automatically for catkin packages. I put it into the workspace(catkin_ws/src) and rebuild the project after clear the old build and devel folder. finally, it can run the dataset by use roslaunch. Now I can run the roslaunch file and the project can work well. (sometime the algorithm may get lose like this: image I don't know if is fine because I met a new problem which will be motioned behind.) When I want to open the rviz to see the visualization processing and the result it will show this error: image image

I check the visualizer code but have nothing to do. Do you know what happened?

yan99033 commented 5 years ago

Hi @yan99033 Thanks for your reply. As you said before the error is caused by the Opencv.(kinetic opencv and the original opencv may have some conflict.) I go to this( https://github.com/ethz-asl/opencv2_catkin )to get the convenience wrapper that handles the include and link for opencv automatically for catkin packages. I put it into the workspace(catkin_ws/src) and rebuild the project after clear the old build and devel folder. finally, it can run the dataset by use roslaunch. Now I can run the roslaunch file and the project can work well. (sometime the algorithm may get lose like this: image I don't know if is fine because I met a new problem which will be motioned behind.) When I want to open the rviz to see the visualization processing and the result it will show this error: image image

I check the visualizer code but have nothing to do. Do you know what happened?

@zxp771 Did you use RGB images? Because OpenCV raised an assertion error in cvtColor, and I guess that has something to do with greyscale image and colour image dimensions.

Also, you could be feeding the wrong image directory; therefore the images cannot be read.

zxp771 commented 5 years ago

Hi @yan99033

Yes, I used the Kitti dataset and of course, they are RGB images. Do you mean that the rviz may convert the image again?

yan99033 commented 5 years ago

Hi @yan99033

Yes, I used the Kitti dataset and of course, they are RGB images. Do you mean that the rviz may convert the image again?

@zxp771 The cvtColor is used in the original SVO code too. I would suggest you build the original SVO code first, making sure that it is running properly. Or else it is very hard to tell whether or not the problem is related to the modified SVO code.

It is hard to pinpoint the problem by just looking at the screenshots.

ouyangandy commented 5 years ago

Can I run this code with my own camera?

yan99033 commented 5 years ago

@ouyangandy Yes, you can. You just do it as if you are running SVO with your own camera, but you need to install the monodepth-cpp library in order to run the online mode