search

yankailab / OpenKAI

OpenKAI: A modern framework for unmanned vehicle and robot control
GNU Affero General Public License v3.0
229 stars 93 forks source link

Error building for TX2 #13

Closed kyuhyong closed 6 years ago

kyuhyong commented 7 years ago

I followed everything in how to build documentation.

  1. Installation of jetson inferrence was just fine
  2. Have some issue with /usr/include/opencv2/gpu/gpu.hpp so I had to add following in /src/Base/cv.h using namespace std;
    just before #ifdef USE_CUDA
  3. Still build error /home/nvidia/src/OpenKAI/src/Application/../Config/../SLAM/_ORB_SLAM2.h:13:20: fatal error: System.h: No such file or directory I simply cannot find where the System.h file located. How can I locate this file correctly? Thanks,

Kyu

kyuhyong commented 7 years ago

In an attempt to investigate whether the error was due to have not build ORB_SLAM2, I git downloaded the project on https://github.com/raulmur/ORB_SLAM2 And build was just fine. I found the System.h file inside the ORB_SLAM2 under /include however this file is not generic use as it requires followings too.

include

include

include<opencv2/core/core.hpp>

include "Tracking.h"

include "FrameDrawer.h"

include "MapDrawer.h"

include "Map.h"

include "LocalMapping.h"

include "LoopClosing.h"

include "KeyFrameDatabase.h"

include "ORBVocabulary.h"

include "Viewer.h"

This is very frustrating for me. Could you further explain what is needed for resolving this issue? Or what the System.h file supposed to be? Thanks,

Kyu

yankailab commented 7 years ago

Hi Kyu, the integration for orb slam is still under development, I will make a build option to separate orb Slam if you dont need it.

Meanwhile, I need a bit more time to fully support orb slam if you do need it.

kyuhyong commented 7 years ago

Hi. Thanks for your answer! I don't need to run SLAM at this moment and just to see how it performs with my drone setup. So there is a way to OFF building ORB_SLAM? I will check if I can do it in the cmakefile too and see if it works. Kyu

kyuhyong commented 7 years ago

I commented out #include "../SLAM/_ORB_SLAM2.h" in /Config/Module.h and now start to build and got following another error.

/home/nvidia/src/OpenKAI/src/Vision/_Flow.cpp:39:12: error: ‘createOptFlow_Farneback_GPU’ is not a member of ‘cv::superres’
  m_pFarn = superres::createOptFlow_Farneback_GPU();

I think this is due to incorrect OpenCV version I have on my tx2 Following is what I get about my opencv version.

$ dpkg -l | grep libopencv
ii  libopencv4tegra                             2.4.13-17-g5317135                            arm64        Tegra optimized Open Computer Vision Library
ii  libopencv4tegra-dev                         2.4.13-17-g5317135                            arm64        Development files for Tegra optimized Open Source Computer Vision Library
ii  libopencv4tegra-repo                        2.4.13-17-g5317135                            arm64        Repository with Tegra optimized Open Source Computer Vision Library
$ pkg-config --modversion opencv
3.2.0

Is OpenKai requires specific opencv version? Thanks,

Kyu

GPrathap commented 7 years ago

Yes, some of the modules are required OpenCV 3.x contribute modules as well. In order to build OpenCV 3.x, you can follow this[1]. Since it is required to build with contribute modules[2], there is a few command line options are to be added in build script which is mentioned in [1]. Commandline options can be found in [2].

  1. https://github.com/jetsonhacks/buildOpenCVTX2
  2. https://github.com/opencv/opencv_contrib
SteveJos commented 7 years ago

Hello @kyuhyong and @kyuhyong, were you able to run openKAI on TX2? I managed to build (version without ORBSLAM) but I dont get the video from ZED in the openKAI window. What do I have to edit in the kiss file in order to get the ZED configured?

Thank you in advance!

kyuhyong commented 7 years ago

I was able to compile the code but now I get link error as following. [ 1%] Linking CXX executable OpenKAI /usr/bin/ld: warning: libopencv_core.so.2.4, needed by /usr/lib/gcc/aarch64-linux-gnu/5/../../../../lib/libopencv_gpu.so, may conflict with libopencv_core.so.3.2 /usr/bin/ld: CMakeFiles/OpenKAI.dir/src/Vision/_ZED.cpp.o: undefined reference to symbol '_ZN2cv3gpu6GpuMat6createEiii' //usr/lib/libopencv_core.so.2.4: error adding symbols: DSO missing from command line collect2: error: ld returned 1 exit status CMakeFiles/OpenKAI.dir/build.make:2540: recipe for target 'OpenKAI' failed make[2]: *** [OpenKAI] Error 1 CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/OpenKAI.dir/all' failed make[1]: *** [CMakeFiles/OpenKAI.dir/all] Error 2 Makefile:83: recipe for target 'all' failed make: *** [all] Error 2 I think this error is due to ZED SDK is built with opencv 2.4 while I was trying to use opencv 3.2 I copied some library files into ZED's /lib but no success yet.

@SteveJos how did you solve this issue? Thanks,

GPrathap commented 7 years ago

Yeap still zed is being used OpenCV for Tegra 2.13.x version in tx2. Here is the trick which gets thing working.

  1. reinstall OpenCV for Tegra. to do this hope you can locate libopencv4tegra-.....*.deb in your tk2 home folder. 
    
sudo dpkg -i libopencv4tegra-.....*.deb 
sudo apt-get update
sudo apt-get install libopencv4tegra libopencv4tegra-dev

More information: http://elinux.org/Jetson/Installing_OpenCV

2. open up go to build directory of OpenKAI, then type 
 `ccmake ..`

Note: if you don't have ccmake in your tx2 you can install it with `sudo apt-get install cmake-curses-gui`. if this is not working, ask from google how to install "CMake GUI for Linux". 

3. Set opencv location to build folder where opencv 3.2.x is installed. 
4.Then type c and g to generate cmake configuration files. 
4. Then to build OpenKAI, make -j3 in build directory of OpenKAI.
5. Then have a fun with OpenKAI.

`
SteveJos commented 7 years ago

@GPrathap Thank you for your solution. Unfortunately I dont see any option to change the path of opencv3.2. Furthermore did you try to compile and install OpenCV 3.x from source? Or does OpenKAI work just with the opencv4tegra stuff?

I get the following runtime error (after a succesful build):

` ZED (Init) >> Depth mode: Performance ZED (Init) >> Video mode: VGA@30

OpenCV Error: Gpu API call (invalid device function) in call, file /hdd/buildbot/slave_jetson_tx_3/35-O4T-L4T-R24/opencv/modules/gpu/include/opencv2/gpu/device/detail/transform_detail.hpp, line 320 terminate called after throwing an instance of 'cv::Exception' what(): /hdd/buildbot/slave_jetson_tx_3/35-O4T-L4T-R24/opencv/modules/gpu/include/opencv2/gpu/device/detail/transform_detail.hpp:320: error: (-217) invalid device function in function call

Aborted (core dumped) `

SteveJos commented 7 years ago

And if I run the latest code with apCopter_zed.kiss I still get a black window with FPS information etc. but no video recognition.

GPrathap commented 7 years ago

In here change the OpenCV include and lib directory into opencv 3.2.x build/lib and build/include directories. image

You are getting this exception because of wrong liking is being happended. See this issue[https://github.com/opencv/opencv/issues/4849]. If you follow exactly what I suggest earlier you wont face any problem.

SteveJos commented 7 years ago

@GPrathap all right thank you for the hint disabling "USE_OPENCV4TEGRA" option! Then I can change the opencv directories. But as I understood one should compile with the opencv4tegra libs? Otherwise I end up with this compile error:

error

How did you exactly install/compile which version of opencv? It really gives me a headache :/

GPrathap commented 7 years ago

Please don't disable OPENCV4TEGRA. The error you are getting because of tracking API in OpenCV comes as an external module. You have to build OpenCV with this as well (https://github.com/opencv/opencv_contrib).

Please if you need any further instruction, please send an email or call via google hangout. I won't reply on the thread anymore.

yankailab commented 7 years ago

Compile with both OpenCV4Tegra and OpenCV3 on the same system is not a good idea it would mess up and make thing complicated.

So the problem seems to be that OpenKAI does not compile with stocked OpenCV4Tegra because it doesnt has contrib part.

I will update soon and make a build switch to separate the contrib needed part. OpenKAI should be compiled with OpenCV4Tegra out of box, it seems sometime ago I merged contrib part by mistake. Will do it soon.

kyuhyong commented 7 years ago

@yankailab Thank you for your reply. I look forward to have the update.

yankailab commented 7 years ago

@kyuhyong I updated CMakeList for TX2, it should work with stock OpenCV4Tegra 2.4.13 from JetPack3.1, please find here for detailed install process: http://www.openkai.xyz/tx1build.html

kyuhyong commented 7 years ago

@yankailab Thanks, I finally can build OpenKAI on TX2 with clean flashed Jetpack3.0. (I'm confused because the instruction still says its 3.0. Is Jetpack 3.1 correct?)

I still have problem running tho Followings are what I got.

$ sudo ./OpenKAI ../kiss/apCopter.kiss

I0908 05:09:07.506253 4162 main.cpp:11] Kiss file:../kiss/apCopter.kiss I0908 05:09:07.507812 4162 Startup.cpp:60] OpenKAI: pApp->v("winMouse", &m_winMouse) E0908 05:09:07.507860 4162 Startup.cpp:162] OpenKAI: Create instance failed: apView I0908 05:09:07.507884 4162 _VisionBase.cpp:56] brio: pK->v("angleV", &m_angleV) I0908 05:09:07.507897 4162 _VisionBase.cpp:57] brio: pK->v("angleH", &m_angleH) I0908 05:09:07.507908 4162 _VisionBase.cpp:58] brio: pK->v("bGimbal", &m_bGimbal) I0908 05:09:07.507921 4162 _VisionBase.cpp:59] brio: pK->v("isoScale", &m_isoScale) I0908 05:09:07.507933 4162 _VisionBase.cpp:61] brio: pK->v("orientation", (int*)&m_orientation) I0908 05:09:07.507946 4162 _VisionBase.cpp:62] brio: pK->v("depthNormFrom", &m_depthNormInt.x) I0908 05:09:07.507956 4162 _VisionBase.cpp:63] brio: pK->v("depthNormTo", &m_depthNormInt.y) I0908 05:09:07.507969 4162 _VisionBase.cpp:73] brio: pK->v("bHSV", &bParam) I0908 05:09:07.507980 4162 _VisionBase.cpp:78] brio: pK->v("bDepthNorm", &bParam) I0908 05:09:07.507993 4162 _Camera.cpp:41] brio: pK->v("bCrop", &m_bCrop) I0908 05:09:07.508005 4162 _Camera.cpp:50] brio: pK->v("bCalibration", &m_bCalibration) I0908 05:09:07.508016 4162 _Camera.cpp:51] brio: pK->v("bFisheye", &m_bFisheye) I0908 05:09:07.508028 4162 _Camera.cpp:52] brio: pK->v("calibFile", &calibFile) I0908 05:09:07.508038 4162 _Camera.cpp:94] brio: Initialized E0908 05:09:07.508123 4162 Startup.cpp:162] OpenKAI: Create instance failed: orbSlam2 I0908 05:09:07.508255 4162 _ThreadBase.cpp:43] apState: pK->v("FPS", &FPS) HIGHGUI ERROR: V4L2: Pixel format of incoming image is unsupported by OpenCV E0908 05:09:07.640410 4163 _Camera.cpp:109] brio: Cannot open camera:0

Looks like I need to upgrade openCV? Maybe I have to start another thread.

kyuhyong commented 7 years ago

Sorry, I forgot to remove the onboard camera on TX2 but I still getting another error message.

$ ./OpenKAI ../kiss/apCopter.kiss I0911 00:14:38.644196 2223 main.cpp:11] Kiss file:../kiss/apCopter.kiss I0911 00:14:38.646327 2223 Startup.cpp:60] OpenKAI: pApp->v("winMouse", &m_winMouse) E0911 00:14:38.646390 2223 Startup.cpp:162] OpenKAI: Create instance failed: apView I0911 00:14:38.646425 2223 _VisionBase.cpp:56] brio: pK->v("angleV", &m_angleV) I0911 00:14:38.646446 2223 _VisionBase.cpp:57] brio: pK->v("angleH", &m_angleH) I0911 00:14:38.646466 2223 _VisionBase.cpp:58] brio: pK->v("bGimbal", &m_bGimbal) I0911 00:14:38.646483 2223 _VisionBase.cpp:59] brio: pK->v("isoScale", &m_isoScale) I0911 00:14:38.646502 2223 _VisionBase.cpp:61] brio: pK->v("orientation", (int*)&m_orientation) I0911 00:14:38.646520 2223 _VisionBase.cpp:62] brio: pK->v("depthNormFrom", &m_depthNormInt.x) I0911 00:14:38.646538 2223 _VisionBase.cpp:63] brio: pK->v("depthNormTo", &m_depthNormInt.y) I0911 00:14:38.646555 2223 _VisionBase.cpp:73] brio: pK->v("bHSV", &bParam) I0911 00:14:38.646572 2223 _VisionBase.cpp:78] brio: pK->v("bDepthNorm", &bParam) I0911 00:14:38.646595 2223 _Camera.cpp:41] brio: pK->v("bCrop", &m_bCrop) I0911 00:14:38.646611 2223 _Camera.cpp:50] brio: pK->v("bCalibration", &m_bCalibration) I0911 00:14:38.646630 2223 _Camera.cpp:51] brio: pK->v("bFisheye", &m_bFisheye) I0911 00:14:38.646646 2223 _Camera.cpp:52] brio: pK->v("calibFile", &calibFile) I0911 00:14:38.646662 2223 _Camera.cpp:94] brio: Initialized E0911 00:14:38.646771 2223 Startup.cpp:162] OpenKAI: Create instance failed: orbSlam2 I0911 00:14:38.646970 2223 _ThreadBase.cpp:43] apState: pK->v("FPS", &FPS) HIGHGUI ERROR: V4L/V4L2: VIDIOC_S_CROP HIGHGUI ERROR: V4L/V4L2: VIDIOC_S_CROP OpenCV Error: Gpu API call (invalid device function) in call, file /hdd/buildbot/slave_jetson_tx_3/35-O4T-L4T-R24/opencv/modules/gpu/include/opencv2/gpu/device/detail/transform_detail.hpp, line 320 terminate called after throwing an instance of 'cv::Exception' what(): /hdd/buildbot/slave_jetson_tx_3/35-O4T-L4T-R24/opencv/modules/gpu/include/opencv2/gpu/device/detail/transform_detail.hpp:320: error: (-217) invalid device function in function call

Aborted (core dumped)

yankailab commented 7 years ago

hello, JetPack 3.1 for Jetson TX2 still have some issues in camera input and uart i/o etc., it is recommeded to use one earlier version JetPack 3.0 at the moment.

kyuhyong commented 7 years ago

@yankailab It's weird because I built OpenKAI on TX2 with Jetpack 3.0 and still not working. Below is what I got when tried to run.

$ ./OpenKAI ../kiss/apCopter.kiss I0914 03:07:22.062855 2276 main.cpp:11] Kiss file:../kiss/apCopter.kiss I0914 03:07:22.066059 2276 Startup.cpp:60] OpenKAI: pApp->v("winMouse", &m_winMouse) E0914 03:07:22.066121 2276 Startup.cpp:162] OpenKAI: Create instance failed: apView I0914 03:07:22.066155 2276 _VisionBase.cpp:56] brio: pK->v("angleV", &m_angleV) I0914 03:07:22.066176 2276 _VisionBase.cpp:57] brio: pK->v("angleH", &m_angleH) I0914 03:07:22.066195 2276 _VisionBase.cpp:58] brio: pK->v("bGimbal", &m_bGimbal) I0914 03:07:22.066213 2276 _VisionBase.cpp:59] brio: pK->v("isoScale", &m_isoScale) I0914 03:07:22.066231 2276 _VisionBase.cpp:61] brio: pK->v("orientation", (int*)&m_orientation) I0914 03:07:22.066249 2276 _VisionBase.cpp:62] brio: pK->v("depthNormFrom", &m_depthNormInt.x) I0914 03:07:22.066267 2276 _VisionBase.cpp:63] brio: pK->v("depthNormTo", &m_depthNormInt.y) I0914 03:07:22.066284 2276 _VisionBase.cpp:73] brio: pK->v("bHSV", &bParam) I0914 03:07:22.066301 2276 _VisionBase.cpp:78] brio: pK->v("bDepthNorm", &bParam) I0914 03:07:22.066321 2276 _Camera.cpp:41] brio: pK->v("bCrop", &m_bCrop) I0914 03:07:22.066339 2276 _Camera.cpp:50] brio: pK->v("bCalibration", &m_bCalibration) I0914 03:07:22.066356 2276 _Camera.cpp:51] brio: pK->v("bFisheye", &m_bFisheye) I0914 03:07:22.066372 2276 _Camera.cpp:52] brio: pK->v("calibFile", &calibFile) I0914 03:07:22.066390 2276 _Camera.cpp:94] brio: Initialized E0914 03:07:22.066498 2276 Startup.cpp:162] OpenKAI: Create instance failed: orbSlam2 I0914 03:07:22.066715 2276 _ThreadBase.cpp:43] apState: pK->v("FPS", &FPS) HIGHGUI ERROR: V4L/V4L2: VIDIOC_S_CROP HIGHGUI ERROR: V4L/V4L2: VIDIOC_S_CROP ^C SIGINT: Complete OpenCV Error: Gpu API call (driver shutting down) in mallocPitch, file /home/ubuntu/build-opencv/opencv/modules/dynamicuda/include/opencv2/dynamicuda/dynamicuda.hpp, line 1134 terminate called after throwing an instance of 'cv::Exception' what(): /home/ubuntu/build-opencv/opencv/modules/dynamicuda/include/opencv2/dynamicuda/dynamicuda.hpp:1134: error: (-217) driver shutting down in function mallocPitch

Aborted (core dumped)

yankailab commented 7 years ago

@kyuhyong , it looks you haven't installed OrbSlam2, if you don't need OrbSlam2, could you try to set "bInst":0 in "orbSlam2" in apCopter.kiss like below:

{ "name":"orbSlam2", "class":"_ORB_SLAM2", "FPS":30, "bInst":0, <-----------------------------------set it to 0 "Window":"apView", "_VisionBase":"brio", "width":640, "height":360, "bViewer":1, }

yankailab commented 7 years ago

@kyuhyong , another possible solution is to check the device ID of the camera instance setting in the .kiss file, e.g.

{ "name":"brio", "class":"Camera", "FPS":30, "bInst":1, "Window":"apView", "deviceID":0, <-- 0 means "/dev/video0", check if it exists in terminal by "ll /dev" command "width":1280, "height":720, "bFlip":0, "bGray":1, "calibFile":"/home/ubuntu/dev/workspace/OpenKAI/OpenKAI/data/calibration/brio_1280x720.xml", }

SteveJos commented 7 years ago

@kyuhyong @yankailab with the current release of OpenKAI I also get the error:

E1012 15:02:40.288002 28214 Startup.cpp:162] OpenKAI: Create instance failed: apView

It is independent of ZED or other cameras. ORB-SLAM and LSD are disabled. What could be the reason?

yankailab commented 6 years ago

@SteveJos , apView create instance will be failed if "bWindow":0 is set. Set "bWindow":1 to solve it. I add an output for this to make it clearer on launch.

yankailab commented 6 years ago

Added an install script for TX2 (tested on JetPack3.2) https://github.com/yankailab/OpenKAI/blob/master/sh/Install/OpenKAI_install_TX2.sh