search

CMU-Perceptual-Computing-Lab / openpose

OpenPose: Real-time multi-person keypoint detection library for body, face, hands, and foot estimation
https://cmu-perceptual-computing-lab.github.io/openpose
Other
30.68k stars 7.82k forks source link

Scaling Issues in 3D Reconstruction : Windows 10 #559

Closed vasavig closed 5 years ago

vasavig commented 6 years ago

Issue Summary

I am running the demo with 3d flag on. I am using two Flir cameras and I am able to extract the 2d and 3d keypoints in a JSON file. However, the 3d reconstruction seems to be converging to a very small area. Is there a reason behind scaling the x, y, and z coordinates differently in gui3d.cpp file? Could that be the reason for unclear 3d rendering?

I am trying to plot the points using another framework but it would be very helpful to see the 3d reconstruction for the live feed.

Type of Issue

Issue with 3D display

System Configuration

Operating system : Windows 10 Installation mode: VS 2015 Enterprise with Update 3 build CUDA version (cat /usr/local/cuda/version.txt in most cases):cuda 8 cuDNN version: v 5.1 CMake version (cmake --version in Ubuntu): 3.11.1 Release or Debug mode? (by defualt: release): Release 3-D Reconstruction module added? (by default: no): No GPU model (nvidia-smi in Ubuntu): Nvidia Titan X Caffe version: Default from OpenPose

If anyone knows how to fix this issue, please comment. Thanks in advance!

gineshidalgo99 commented 6 years ago

Hi @vasavig , wrong (smaller) scaling is an issue that I am trying to fix. I am not sure how where it is coming from, but I am debugging it at the moment. I'll keep you updated after it's fixed. Thanks!

(any feed-back from your side will also be welcome!)

vasavig commented 6 years ago

@gineshidalgo99 Thanks! I appreciate your prompt action. If I nail down the issue, I will post here.

gineshidalgo99 commented 6 years ago

@vasavig Just to confirm, are you getting the 3D keypoints too small when output? (i.e. --write_json), Or just in the visualization? I am still trying to isolate the error to my end. Thanks

vasavig commented 6 years ago

@gineshidalgo99 The 3D key points look all right when plotted using a different method. I plotted them using matplotlib in Python and they look all right, no scaling issues there.

gineshidalgo99 commented 6 years ago

OK, but what I meant was: did you verify the returned person 3d keypoints have the right height? (which should be ~1.60-1.90 meters?) And when introduced the camera parameters, where the translation of the camera matrix in meters or millimeters?

gineshidalgo99 commented 5 years ago

Smaller 3D values than they should be: This was fixed several months back, I am sorry I forgot to update this post.

Display: I believe this is not a bug, but rather the measurements are just either too small or too big so the 3D GUI is not able to plot in the right range. Saving the coordinates to JSON and plotting them later with any other Python/Matlab displayer would work. You can always scale the camera parameters to make them align to the OpenPose 3D GUI, which should work in the range of 1-3 meters for best results.