search

FORTH-ModelBasedTracker / MocapNET

We present MocapNET, a real-time method that estimates the 3D human pose directly in the popular Bio Vision Hierarchy (BVH) format, given estimations of the 2D body joints originating from monocular color images. Our contributions include: (a) A novel and compact 2D pose NSRM representation. (b) A human body orientation classifier and an ensemble of orientation-tuned neural networks that regress the 3D human pose by also allowing for the decomposition of the body to an upper and lower kinematic hierarchy. This permits the recovery of the human pose even in the case of significant occlusions. (c) An efficient Inverse Kinematics solver that refines the neural-network-based solution providing 3D human pose estimations that are consistent with the limb sizes of a target person (if known). All the above yield a 33% accuracy improvement on the Human 3.6 Million (H3.6M) dataset compared to the baseline method (MocapNET) while maintaining real-time performance
https://www.youtube.com/watch?v=Jgz1MRq-I-k
Other
858 stars 137 forks source link

Build error on Jetson Nano #20

Closed beemsoft closed 4 years ago

beemsoft commented 4 years ago

Hi,

I'm trying to install MocapNET on a Jetson Nano. When running the initialize.sh script, I'm getting the following error:

/home/hans/MocapNET/dependencies/RGBDAcquisition/tools/AmMatrix/matrix4x4Tools.c:1062:10: fatal error: xmmintrin.h: No such file or directory
 #include <xmmintrin.h>
          ^~~~~~~~~~~~~
compilation terminated.
src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir/build.make:374: recipe for target 'src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir/__/__/dependencies/RGBDAcquisition/tools/AmMatrix/matrix4x4Tools.c.o' failed
make[2]: *** [src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir/__/__/dependencies/RGBDAcquisition/tools/AmMatrix/matrix4x4Tools.c.o] Error 1
CMakeFiles/Makefile2:100: recipe for target 'src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir/all' failed
make[1]: *** [src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

According to https://forums.developer.nvidia.com/t/fatal-error-emmintrin-h-no-such-file-or-directory-compilation-terminated/46577, ARM CPUs do not support AMD x86_64 SSE/SSE2.

Do you know how this could be fixed on the Jetson Nano?

Greetings, Hans Beemsterboer

AmmarkoV commented 4 years ago

Hello, I will use precompiler to disable this part of the code on ARM machines, overall it is not essential for compilation, since there is also a version of the call that can be optimized for ARM so I will promptly fix it...!

If you are able to compile tensorflow and opencv ( which you can in a jetson ) the rest of the code should work fine without sse..!

AmmarkoV commented 4 years ago

If you can run the ./update.sh script or manually git pull the dependencies/RGBDAcquisition repository and recompile the issue should be corrected..

beemsoft commented 4 years ago

Hi, Thanks for this fix! I could continue the build, but now I'm getting the following error:

[ 10%] Built target GroundTruthDumper Scanning dependencies of target MocapNETLib [ 11%] Building CXX object src/MocapNET1/MocapNETLib/CMakeFiles/MocapNETLib.dir/mocapnet.cpp.o [ 11%] Building CXX object src/MocapNET1/MocapNETLib/CMakeFiles/MocapNETLib.dir/__/__/Tensorflow/tf_utils.cpp.o [ 11%] Linking CXX shared library ../../../../libMocapNETLib.so ../../../../dependencies/libtensorflow/lib/libtensorflow.so: error adding symbols: File in wrong format collect2: error: ld returned 1 exit status src/MocapNET1/MocapNETLib/CMakeFiles/MocapNETLib.dir/build.make:100: recipe for target '../libMocapNETLib.so' failed make[2]: *** [../libMocapNETLib.so] Error 1 CMakeFiles/Makefile2:150: recipe for target 'src/MocapNET1/MocapNETLib/CMakeFiles/MocapNETLib.dir/all' failed make[1]: *** [src/MocapNET1/MocapNETLib/CMakeFiles/MocapNETLib.dir/all] Error 2 Makefile:83: recipe for target 'all' failed make: *** [all] Error 2

Tensorflow version: 1.15.2 OpenCV version: 4.1.1

AmmarkoV commented 4 years ago

Hello, this is a linker error that has to do with the build of the tensorflow library..

What I imagine that happened is that the ./initialize.sh script tried to check if you have tensorflow installed in your system by probing /usr/local/lib/libtensorflow.so ( as seen in line 183 ) but Jetson probably has a different library organization, so what the MocapNET script did is that it didn't found it there and went and downloaded a user only version of the library ( the x86_64 version as seen in line 192 ) and put it in dependencies/libtensorflow.

My code compiles ok in the ARM environment but when ARM code is linked against an x86_64 version of tensorflow this is impossible..

So.. How to fix!

There are two things to do :

If you don't have tensorflow built for your ARM system

A) If you can find a precompiled Jetson tensorflow library binary like the ones that may be provided here https://docs.nvidia.com/deeplearning/frameworks/install-tf-jetson-platform/index.html then do this. However please note that you don't need the python stuff you need the C libraries and the C includes ( you can take a look on what you need by seeing what you already have in dependencies/libtensorflow/include, you just need to change the .so libraries with their ARM counterpart )

screen-2020-06-23-10-02-17

B) If you can't find them for your platform you will need to compile tensorflow from scratch since there are no official binaries provided from google. The precompiled versions are only x86/x64 . Documentation on Building Tensorflow for C can be found here Any Version over 1.8 up to 1.15.2 should work fine, the tricky part is compiling it :P

You should keep in mind that the build system of tensorflow is pretty terrible I think this talk "How To Make Package Managers Cry" nails it :) ..

I have created a script to automate it on my x64 machines that might be useful for you https://github.com/AmmarkoV/MyScripts/blob/master/Tensorflow/tensorflowBuild.sh

If you manage to build it, or through another way already have tensorflow built for your ARM system, you should either replace or remove the downloaded version of the library and use the ARM version

So either replace MocapNET/dependencies/libtensorflow with your ARM stuff

OR

entirely remove the directory MocapNET/dependencies/libtensorflow and then provide the correct path to CMake so it will link your ARM code with the ARM version of tensorflow..!

For example you can

` sudo apt-get install cmake-qt-gui

cd MocapNET/build

cmake-gui ..

`

and change the paths to point to your system location as seen in this image to use your ARM based tensorflow library screen-2020-06-23-09-45-45

beemsoft commented 4 years ago

Hi, thanks for the clear instructions for how to approach this. Apparently, other people are also still busy with this: https://forums.developer.nvidia.com/t/tensorflow-libs-for-c-c-integration-on-jetson-nano/71706

For the moment, I'm switching back to my x86_64 laptop. I'm eager to do a test with MocapNET. Hopefully, I can pick up this platform issue later on.

AmmarkoV commented 4 years ago

I wish I could do more to help but unfortunately I don't have a Jetson available, it would be interesting for me as well to get a benchmark on such a system. :) I am pretty sure the method will work >40fps since all the CUDA stuff is available for the bundled 2D estimator which is the heaviest part of the RGB->3D pipeline. Also for building it on ARM, I know that there are ARM versions of tensorflow for raspberry pi so it should definitely be working on the Jetson. It is just that it is a much more exotic piece of hardware and that translates to a smaller development community.

Also as I already discussed with some other guys in this issues section keep in mind that I am waiting for some peer reviews, I have ~1 year of work that is waiting to get commited here that will be a pretty substantial improvement in pose estimation quality!

i.e. From this source video to this BVH

The current version here is not too bad but the new version is definitely better, the good thing is that the BVH files are exactly the same so it will be easy to upgrade once it lands here..

beemsoft commented 4 years ago

The upcoming update looks very good, thanks for the preview! At the moment, I'm testing OpenPose JSON output and let MocapNET generate the BVH file.

AmmarkoV commented 4 years ago

Due to the new release having reworked a lot of the back-end of the project this issue is outdated.

The main issue with Jetson support is NVIDIA CUDA/tensorflow support, OpenCV2 can be properly compiled in this platform, and MocapNET source is "compileable" on most ARM devices when the INTEL_OPTIMIZATIONS CMake flag is not enabled ( albeit with slightly worse performance ). I am thus closing this issue. Feel free to reopen a new issue if you try deploying on Jetson with the new version and you face problems!