search

FORTH-ModelBasedTracker / MocapNET

We present MocapNET, a real-time method that estimates the 3D human pose directly in the popular Bio Vision Hierarchy (BVH) format, given estimations of the 2D body joints originating from monocular color images. Our contributions include: (a) A novel and compact 2D pose NSRM representation. (b) A human body orientation classifier and an ensemble of orientation-tuned neural networks that regress the 3D human pose by also allowing for the decomposition of the body to an upper and lower kinematic hierarchy. This permits the recovery of the human pose even in the case of significant occlusions. (c) An efficient Inverse Kinematics solver that refines the neural-network-based solution providing 3D human pose estimations that are consistent with the limb sizes of a target person (if known). All the above yield a 33% accuracy improvement on the Human 3.6 Million (H3.6M) dataset compared to the baseline method (MocapNET) while maintaining real-time performance
https://www.youtube.com/watch?v=Jgz1MRq-I-k
Other
836 stars 135 forks source link

Errors when trying OpenCVTest and MocapNET2LiveWebcamDemo in Windows Subsystem for Linux #48

Closed Aurosutru closed 3 years ago

Aurosutru commented 3 years ago

MocapNet is installed in Ubuntu 2020.4 in WSL but when trying OpenCVTest the following errors occurs:

tm@LAPTOP-DGSMQ87U:/mnt/d/MocapNET-master$ ./OpenCVTest --from /shuffle.webm --novisualization (OpenCVTest:2395): GStreamer-CRITICAL **: 10:37:02.419: gst_element_make_from_uri: assertion 'gst_uri_is_valid (uri)' failed [ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (713) open OpenCV | GStreamer warning: Error opening bin: no source element for URI "/shuffle.webm" [ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created [ERROR:0] global ../modules/videoio/src/cap.cpp (116) open VIDEOIO(CV_IMAGES): raised OpenCV exception: OpenCV(4.2.0) ../modules/videoio/src/cap_images.cpp:253: error: (-5:Bad argument) CAP_IMAGES: can't find starting number (in the name of file): /shuffle.webm in function 'icvExtractPattern' Trying to open /shuffle.webm Openning camera failed tm@LAPTOP-DGSMQ87U:/mnt/d/MocapNET-master$

Are these errors non-critical? Another test also apparently fails:

tm@LAPTOP-DGSMQ87U:/mnt/d/MocapNET-master$ ./MocapNET2LiveWebcamDemo --from shuffle.webm CPU : Intel(R) Core(TM) i7-10750H CPU @ 2.60GHz pcilib: Cannot open /proc/bus/pci lspci: Cannot find any working access method. GPU : Trying to open source (shuffle.webm) Using BVH codebase version 0.44 buildOpenGLProjectionForIntrinsics: img(1920x1080) fx 582.18, fy 582.53, cx 960.00, cy 540.00 Number of Values Per Frame: 498 Frames: 1(1) / Frame Time : 0.0400 BVH subsystem initialized using dataset/headerWithHeadAndOneMotion.bvh that contains 1 frames ( 498 motions each ) Initializing IK code.. Body Problem : The IK problem we want to solve has 2 groups of subproblems It is also ultimately divided into 7 kinematic chains Chain 0 has 10 parts : jID(hip/0)->mID(0 to 2) jID(hip/0)->mID(3 to 5) jID(neck/3)->EndEffector jID(head/5)->EndEffector jID(rShldr/105)->EndEffector jID(lShldr/136)->EndEffector jID(rThigh/167)->EndEffector jID(lThigh/190)->EndEffector jID(eye.l/70)->EndEffector jID(eye.r/78)->EndEffector Chain 1 has 4 parts : jID(chest/2)->mID(9 to 11) jID(neck/3)->EndEffector jID(rShldr/105)->EndEffector jID(lForeArm/137)->EndEffector Chain 2 has 5 parts : jID(neck/3)->mID(12 to 14) jID(neck1/4)->mID(15 to 17) jID(head/5)->mID(18 to 20) jID(eye.l/70)->EndEffector jID(eye.r/78)->EndEffector Chain 3 has 3 parts : jID(rShldr/105)->mID(237 to 239) jID(rForeArm/106)->mID(240 to 242) jID(rHand/107)->EndEffector Chain 4 has 3 parts : jID(lShldr/136)->mID(315 to 317) jID(lForeArm/137)->mID(318 to 320) jID(lHand/138)->EndEffector Chain 5 has 5 parts : jID(rThigh/167)->mID(393 to 395) jID(rShin/168)->mID(396 to 398) jID(rFoot/169)->mID(399 to 401) jID(EndSite_toe1-2.R/172)->EndEffector jID(EndSite_toe5-3.R/188)->EndEffector Chain 6 has 5 parts : jID(lThigh/190)->mID(447 to 449) jID(lShin/191)->mID(450 to 452) jID(lFoot/192)->mID(453 to 455) jID(EndSite_toe1-2.L/195)->EndEffector jID(EndSite_toe5-3.L/211)->EndEffector BVH code initalization successfull.. Trying to set thread realtime prio = 99 Failed setting thread realtime priority LoadGraph dataset/combinedModel/mobnet2_tiny_vnect_sm_1.9k.pb using TensorFlow Version: 2.3.1 Error importing graph definition.. (3) Invalid Argument in graph.. Can't load graph dataset/combinedModel/mobnet2_tiny_vnect_sm_1.9k.pb tm@LAPTOP-DGSMQ87U:/mnt/d/MocapNET-master$ Your version is up to date

The file dataset/combinedModel/mobnet2_tiny_vnect_sm_1.9k.pb is a 261KB file in the dataset/combinedModel/ folder.

Any suggestions?

I have installed OpenPose in Win 10 and would like to use its .json output to create .bvh mocap files for Blender using MocapNET.

AmmarkoV commented 3 years ago

You tried running using 

./OpenCVTest --from /shuffle.webm

you should remove the slash on the /shuffle.webm

try 

./OpenCVTest --from shuffle.webm

When you provide a path with the / character in a linux filesystem it is assumed the file is in the root directory..! The other output warnings are development output and you can ignore them..

Aurosutru commented 3 years ago

I updated to 2.1 and as suggested tried ./OpenCVTest --from shuffle.webm

tm@LAPTOP-DGSMQ87U:/mnt/d/MocapNET-master$ ./OpenCVTest --from shuffle.webm Trying to open shuffle.webm
Unable to init server: Could not connect: Connection refused (feed:157): Gtk-WARNING **: 08:23:06.940: cannot open display:

The file shuffle.webm is in the same folder as OpenCVTest.

Is there a doc explaining how use the command line to convert OpenPose JSON files to CSV using convertOpenPoseJSONToCSV and then to 3D BVH files using the MocapNET2CSV binary, as mentioned in Readme.md?

AmmarkoV commented 3 years ago

Hello, the Gtk-WARNING **: 08:23:06.940: cannot open display means you are running on a headless display, I thought on the Linux subsystem for windows the display worked however it might not be the case..

Does 

./MocapNET2LiveWebcamDemo --from shuffle.webm --novisualization

work for you ? It should produce an out.bvh file after running..!

For the OpenPose JSON to CSV procedure I have included a script however a documentation file should be added, it is a good idea and I will do it because it is a very common question.

Aurosutru commented 3 years ago

Hi,

Tried the --novisualization suggestion but result appears the same. Sorry, I am not able to paste the terminal results in these messages with proper line endings, making things a jumble. This appears to be a Linux Subsystem feature that drops line endings.

tm@LAPTOP-DGSMQ87U:/mnt/d/MocapNET-master$ ./MocapNET2LiveWebcamDemo --from shuffle.webm --novisualization CPU : Intel(R) Core(TM) i7-10750H CPU @ 2.60GHz pcilib: Cannot open /proc/bus/pci lspci: Cannot find any working access method. GPU : Visualization disabled Trying to open source (shuffle.webm) Using BVH codebase version 0.44 buildOpenGLProjectionForIntrinsics: img(1920x1080) fx 582.18, fy 582.53, cx 960.00, cy 540.00 Number of Values Per Frame: 498 Frames: 1(1) / Frame Time : 0.0400 BVH subsystem initialized using dataset/headerWithHeadAndOneMotion.bvh that contains 1 frames ( 498 motions each ) Initializing IK code.. Body Problem : The IK problem we want to solve has 2 groups of subproblems It is also ultimately divided into 7 kinematic chains Chain 0 has 10 parts : jID(hip/0)->mID(0 to 2) jID(hip/0)->mID(3 to 5) jID(neck/3)->EndEffector jID(head/5)->EndEffector jID(rShldr/105)->EndEffector jID(lShldr/136)->EndEffector jID(rThigh/167)->EndEffector jID(lThigh/190)->EndEffector jID(eye.l/70)->EndEffector jID(eye.r/78)->EndEffector Chain 1 has 4 parts : jID(chest/2)->mID(9 to 11) jID(neck/3)->EndEffector jID(rShldr/105)->EndEffector jID(lForeArm/137)->EndEffector Chain 2 has 5 parts : jID(neck/3)->mID(12 to 14) jID(neck1/4)->mID(15 to 17) jID(head/5)->mID(18 to 20) jID(eye.l/70)->EndEffector jID(eye.r/78)->EndEffector Chain 3 has 3 parts : jID(rShldr/105)->mID(237 to 239) jID(rForeArm/106)->mID(240 to 242) jID(rHand/107)->EndEffector Chain 4 has 3 parts : jID(lShldr/136)->mID(315 to 317) jID(lForeArm/137)->mID(318 to 320) jID(lHand/138)->EndEffector Chain 5 has 5 parts : jID(rThigh/167)->mID(393 to 395) jID(rShin/168)->mID(396 to 398) jID(rFoot/169)->mID(399 to 401) jID(EndSite_toe1-2.R/172)->EndEffector jID(EndSite_toe5-3.R/188)->EndEffector Chain 6 has 5 parts : jID(lThigh/190)->mID(447 to 449) jID(lShin/191)->mID(450 to 452) jID(lFoot/192)->mID(453 to 455) jID(EndSite_toe1-2.L/195)->EndEffector jID(EndSite_toe5-3.L/211)->EndEffector BVH code initalization successfull.. Trying to set thread realtime prio = 99 Failed setting thread realtime priority LoadGraph dataset/combinedModel/mobnet2_tiny_vnect_sm_1.9k.pb using TensorFlow Version: 2.3.1 Error importing graph definition.. (3) Invalid Argument in graph.. Can't load graph dataset/combinedModel/mobnet2_tiny_vnect_sm_1.9k.pb tm@LAPTOP-DGSMQ87U:/mnt/d/MocapNET-master$ The updated version is 2.1

Trying another approach, I have installed Ubuntu 2020.4 in a VM and hope to have the advantage of seeing graphical output from MocapNET but there are some problems with building Tensorflow. If they don't get resolved I'll start a new issue for that. Thanks for the help.

Aurosutru commented 3 years ago

Looks like I might need to update again, though I did respond to what seemed an update suggestion to Ver. 2.1 previously.

AmmarkoV commented 3 years ago

Hello so this is a different error now so at least we are making some progress :) .. From the new output you sent me the error part is the final lines which read : 

LoadGraph dataset/combinedModel/mobnet2_tiny_vnect_sm_1.9k.pb using TensorFlow Version: 2.3.1 
Error importing graph definition.. (3) Invalid Argument in graph.. 
Can't load graph dataset/combinedModel/mobnet2_tiny_vnect_sm_1.9k.pb

So the error is emitted from within Tensorflow and it appears for some weird reason that the Tensorflow 2.3.1 runtime cannot load the 2D joint estimation model in your particular setup

This could be caused by the dataset/combinedModel/mobnet2_tiny_vnect_sm_1.9k.pb file being somehow corrupted ( broken download ? ), or an unknown (to me) compatibility issue with TF2.3.1 and windows (?)..

What I want you to try is to delete the potentially corrupted file and the 2.3.1 dependencies/libtensorflow directory and try rerunning the initialize.sh this time selecting the 1.1x version of tensorflow..

after doing a cd and entering the root directory of your MocapNET copy try

rm dataset/combinedModel/mobnet2_tiny_vnect_sm_1.9k.pb
rm -rf dependencies/libtensorflow
./initialize.sh

when prompted with the question 

         Do you want to use Tensorflow 1.x instead of 2.x ? 
         The project is compatible with both but if you have an older GPU it might be better for you 
         to stick with Tensorflow 1.x 

         (Y/N)?

type Y and press enter

the script should automatically rebuild and you can once again attempt running 

./MocapNET2LiveWebcamDemo --from shuffle.webm --novisualization

What I am trying to do is switch you to the 1.1x Tensorflow version and refresh the dataset/combinedModel/mobnet2_tiny_vnect_sm_1.9k.pb.. Hopefully after the above steps doing should yield you the out.bvh result

As a final check you can check the MD5 hashes of your files

md5sum dataset/combinedModel/mobnet2_tiny_vnect_sm_1.9k.pb 
md5sum dataset/combinedModel/openpose_model.pb 
md5sum dataset/combinedModel/vnect_sm_pafs_8.1k.pb

If the output is it means that your downloaded models are ok 

45bb04bc459af758696b8de1c386bacc  dataset/combinedModel/mobnet2_tiny_vnect_sm_1.9k.pb
87a383dc4d8bf6232ac746eaa9ac6e08  dataset/combinedModel/openpose_model.pb
040608c01366ccdecae88929a8f9e340  dataset/combinedModel/vnect_sm_pafs_8.1k.pb

Other than the potential slowdown the VM could also be an option however people have reported MocapNET working under the linux subsystem for windows so this is why I initially proposed this..!

Aurosutru commented 3 years ago

OK, checked the md5sum of each of the three suggested files and only the openpose .pb file matched. So I removed the other two, issued the rm -rf dependencies/libtensorflow command and ran initialize.sh. Now the build process hangs when it reaches 11%, similar but not identical to the problem when building Tensorflow I reported yesterday in issue #50.

All three files now match with md5sum.

Here is the WSL terminal output:

Do you want to use Tensorflow 1.x instead of 2.x ? The project is compatible with both but if you have an older GPU it might be better for you to stick with Tensorflow 1.x (Y/N)?y Selected Tensorflow version cpu/1.14.0 Found a local tensorflow installation, not altering anything RGBDAcquisition appears to already exist .. Now to try and build MocapNET.. Build directory already exists, but since the initialize.sh script was called this means the user wants to also initialize the build directory, so doing this to prevent problems like https://github.com/FORTH-ModelBasedTracker/MocapNET/issues/19 -- The C compiler identification is GNU 9.3.0 -- The CXX compiler identification is GNU 9.3.0 -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done SSE2 detected and will be used.. Using locally found libtensorflow v1 C-API BVH Code found and will be used.. OpenGL support will not be compiled in -- Could NOT find OpenCV (missing: OpenCV_DIR) -- Found OpenCV: /usr (found version "4.2.0") OpenCV code found and will be used.. /usr/lib/x86_64-linux-gnu/cmake/opencv4 -- Configuring done -- Generating done -- Build files have been written to: /mnt/d/MocapNET-master/build Scanning dependencies of target GroundTruthDumper [ 0%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/src/Applications/BVHTester/main.c.o [ 0%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/src/Library/TrajectoryParser/InputParser_C.c.o [ 1%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/src/Library/MotionCaptureLoader/bvh_loader.c.o [ 1%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/src/Library/MotionCaptureLoader/calculate/bvh_to_tri_pose.c.o [ 1%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/src/Library/MotionCaptureLoader/calculate/bvh_transform.c.o [ 2%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/src/Library/MotionCaptureLoader/calculate/bvh_project.c.o [ 2%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/src/Library/MotionCaptureLoader/import/fromBVH.c.o [ 2%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/src/Library/MotionCaptureLoader/export/bvh_to_trajectoryParserTRI.c.o [ 3%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/src/Library/MotionCaptureLoader/export/bvh_to_trajectoryParserPrimitives.c.o [ 3%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/src/Library/MotionCaptureLoader/export/bvh_export.c.o [ 3%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/src/Library/MotionCaptureLoader/export/bvh_to_bvh.c.o [ 4%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/src/Library/MotionCaptureLoader/export/bvh_to_svg.c.o [ 4%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/src/Library/MotionCaptureLoader/export/bvh_to_csv.c.o [ 4%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/src/Library/MotionCaptureLoader/edit/bvh_randomize.c.o [ 5%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/src/Library/MotionCaptureLoader/edit/bvh_filter.c.o [ 5%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/src/Library/MotionCaptureLoader/edit/bvh_rename.c.o [ 5%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/src/Library/MotionCaptureLoader/edit/bvh_merge.c.o [ 5%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/src/Library/MotionCaptureLoader/edit/bvh_cut_paste.c.o [ 6%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/src/Library/MotionCaptureLoader/edit/bvh_interpolate.c.o[ 6%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/src/Library/MotionCaptureLoader/edit/bvh_remapangles.c.o[ 6%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/src/Library/MotionCaptureLoader/ik/bvh_inverseKinematics.c.o [ 7%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/src/Library/MotionCaptureLoader/ik/hardcodedProblems_inverseKinematics.c.o [ 7%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/src/Library/MotionCaptureLoader/metrics/bvh_measure.c.o [ 7%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/src/Library/MotionCaptureLoader/tests/test.c.o [ 8%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/tools/Calibration/calibration.c.o [ 8%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/tools/Calibration/transform.c.o [ 8%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/tools/Calibration/undistort.c.o [ 9%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/tools/AmMatrix/matrix3x3Tools.c.o [ 9%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/tools/AmMatrix/matrix4x4Tools.c.o [ 9%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/tools/AmMatrix/matrixOpenGL.c.o [ 9%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/tools/AmMatrix/matrixCalculations.c.o [ 10%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/tools/AmMatrix/solveLinearSystemGJ.c.o [ 10%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/tools/AmMatrix/solveHomography.c.o [ 10%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/tools/AmMatrix/quaternions.c.o [ 11%] Building C object src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/tools/AmMatrix/simpleRenderer.c.o [ 11%] Linking C executable ../../../GroundTruthDumper /usr/bin/ld: CMakeFiles/GroundTruthDumper.dir///dependencies/RGBDAcquisition/opengl_acquisition_shared_library/opengl_depth_and_color_renderer/src/Applications/BVHTester/main.c.o: in function main': main.c:(.text.startup+0x617): undefined reference tobvh_print_C_Header' collect2: error: ld returned 1 exit status make[2]: [src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir/build.make:594: ../GroundTruthDumper] Error 1 make[1]: [CMakeFiles/Makefile2:369: src/GroundTruthGenerator/CMakeFiles/GroundTruthDumper.dir/all] Error 2 make: *** [Makefile:84: all] Error 2 tm@LAPTOP-DGSMQ87U:/mnt/d/MocapNET-master$

Aurosutru commented 3 years ago

Didn't find an md5sum for the original downloaded .zip so I downloaded it again and found it was different in size than the earlier one. Using the new MocapNet-master.zip the build process completed, but running the ./MocapNET2LiveWebcamDemo --from shuffle.webm --novisualization command produced the following:

/usr/bin/ld: ../../../dependencies/libtensorflow/lib/libtensorflow.so: .dynsym local symbol at index 2618 (>= sh_info of 2) [100%] Built target WebcamAnd2DJoints tm@LAPTOP-DGSMQ87U:/mnt/d/NewMocapNET-master/MocapNET-master$ ./MocapNET2LiveWebcamDemo --from shuffle.webm --novisualization CPU : Intel(R) Core(TM) i7-10750H CPU @ 2.60GHz pcilib: Cannot open /proc/bus/pci lspci: Cannot find any working access method. GPU : Visualization disabled Trying to open source (shuffle.webm) Using BVH codebase version 0.5 buildOpenGLProjectionForIntrinsics: img(1920x1080) fx 582.18, fy 582.53, cx 960.00, cy 540.00 Number of Values Per Frame: 498 Frames: 1(1) / Frame Time : 0.0400 BVH subsystem initialized using dataset/headerWithHeadAndOneMotion.bvh that contains 1 frames ( 498 motions each ) Initializing IK code.. Body Problem : The IK problem we want to solve has 2 groups of subproblems It is also ultimately divided into 7 kinematic chains Chain 0 has 10 parts : jID(hip/0)->mID(0 to 2) jID(hip/0)->mID(3 to 5) jID(neck/3)->EndEffector jID(head/5)->EndEffector jID(rShldr/105)->EndEffector jID(lShldr/136)->EndEffector jID(rThigh/167)->EndEffector jID(lThigh/190)->EndEffector jID(eye.l/70)->EndEffector jID(eye.r/78)->EndEffector Chain 1 has 4 parts : jID(chest/2)->mID(9 to 11) jID(neck/3)->EndEffector jID(rShldr/105)->EndEffector jID(lForeArm/137)->EndEffector Chain 2 has 5 parts : jID(neck/3)->mID(12 to 14) jID(neck1/4)->mID(15 to 17) jID(head/5)->mID(18 to 20) jID(eye.l/70)->EndEffector jID(eye.r/78)->EndEffector Chain 3 has 3 parts : jID(rShldr/105)->mID(237 to 239) jID(rForeArm/106)->mID(240 to 242) jID(rHand/107)->EndEffector Chain 4 has 3 parts : jID(lShldr/136)->mID(315 to 317) jID(lForeArm/137)->mID(318 to 320) jID(lHand/138)->EndEffector Chain 5 has 5 parts : jID(rThigh/167)->mID(393 to 395) jID(rShin/168)->mID(396 to 398) jID(rFoot/169)->mID(399 to 401) jID(EndSite_toe1-2.R/172)->EndEffector jID(EndSite_toe5-3.R/188)->EndEffector Chain 6 has 5 parts : jID(lThigh/190)->mID(447 to 449) jID(lShin/191)->mID(450 to 452) jID(lFoot/192)->mID(453 to 455) jID(EndSite_toe1-2.L/195)->EndEffector jID(EndSite_toe5-3.L/211)->EndEffector BVH code initalization successfull.. Trying to set thread realtime prio = 99 Failed setting thread realtime priority Graph dataset/combinedModel/mobnet2_tiny_vnect_sm_1.9k.pb could not be found tm@LAPTOP-DGSMQ87U:/mnt/d/NewMocapNET-master/MocapNET-master$ Your version is up to date

AmmarkoV commented 3 years ago

Hello first of all and as also this is a common issue with your VM installation, you shouldn't use the .zip files downloaded from github but rather the whole repository since it features automatic updates etc..

So instead of manually downloading the zip instead first try to install git 

sudo apt-get install git

and then after "cd" to a directory which you plan on having your code ( for example Documents ) and you can 

cd ~/Documents/
git clone https://github.com/FORTH-ModelBasedTracker/MocapNET
cd MocapNET
./initialize.sh

This will not only take care of the build dependencies etc, you will also be able to update by issuing 

./update.sh

Your first build problem it is this issue caused by recent changes on the BVH repository

If you didn't use the update.sh script to automatically update all parts of the repository (which could be the case since you manually download zip files) then consider

cd dependencies/RGBDAcquisition
git pull
cd ..
cd ..
cd build
make

to manually update the BVH code and do a rebuild

For the second error

Graph dataset/combinedModel/mobnet2_tiny_vnect_sm_1.9k.pb could not be found

so apparently the .pb file for the 2D estimator is not downloaded in the correct directory ?

I suppose if you use git clone to get the repository and run the initialize.sh script without interrupting it everything should download at the correct place..

Aurosutru commented 3 years ago

Git was already installed, but to build needed the cloned version of the repo rather than the .zip file. Here is the output after successfully installing:

Broken frame..
Broken frame..
Broken frame..
Too many broken frames..
Successfully wrote 378 frames to bvh file.. MocapNET v2.1 execution summary : __
BVH subsystem version 0.5
CPU : Intel(R) Core(TM) i7-10750H CPU @ 2.60GHz
GPU :
Average framerate for 378 samples was 128.09 fps
Code optimizations where on
You can achieve better accuracy by using the homebrewed OpenPose 2D joint estimator using --openpose tm@LAPTOP-DGSMQ87U:~/Downloads/MocapNET$

Issuing ./OpenCVTest --from shuffle.webm --novisualization was not successful, yielding the following:

tm@LAPTOP-DGSMQ87U:~/Downloads/MocapNET$ ./OpenCVTest --from shuffle.webm --novisualization Trying to open shuffle.webm
Unable to init server: Could not connect: Connection refused (feed:2551): Gtk-WARNING **: 05:15:09.266: cannot open display:
tm@LAPTOP-DGSMQ87U:~/Downloads/MocapNET$

I will open another issue to ask questions about integrating openpose with MocapNET, particularly .json files.

AmmarkoV commented 3 years ago

Hello, Great to hear that its working! The broken frames in the end are due to the input acquisition from OpenCV not knowing how many frames there are on the input streaming and trying unsucessfully to get another frame from the video file and failing. So it is normal and expected behavior..

The acquisition works like this because if you want to open a network stream ( i.e. ./MocapNET2LiveWebcamDemo --from https://youronlinecamera/stream.mp4 there will be no "end" until you get a broken connection and broken frames..

Also for the OpenPose / .Json integration feel free to open a new issue, I already added the place where I will add more documentation ( https://github.com/FORTH-ModelBasedTracker/MocapNET/blob/master/doc/OpenPose.md ) however due to having a LOT of work these days this might take a while to add!

If your issue with the windows subsytem deployment is resolved you can close the issue..!

Aurosutru commented 3 years ago

If your issue with the windows subsytem deployment is resolved you can close the issue..!

Yes, WSL is working, though the ./OpenCVTest --from shuffle.webm command didn't work. If that's not important I'll leave it to you to close this issue.

Aurosutru commented 3 years ago

Just noticed that the output listed above for the ./OpenCVTest --from shuffle.webm command is mostly struck out. It shouldn't be. Apparently that is due to another problem with the WSL terminal window and the attempt to work around its formatting.

AmmarkoV commented 3 years ago

Hello the OpenCV test wont work without a working display so it is normal behavior, closing the issue..!