Open djipco opened 3 years ago
Hi,
the face_geometry
module is within the same folder. So nothing else to install, other than numpy
, opencv-python
, mediapipe
and pillow
. It's strange that the import from webcamsource import WebcamSource
seems to work when you call head_posture_rt.py
.
Are you calling head_posture_rt.py
from within the git reps root folder?
A quick hack to make things work, is just to comment the line with the face_geometry
import, and just copy the functions and classes from that file into head_posture_rt.py
.
Let me know if this problem persists, I'll write a little readme.
Best, Rasmus
So, I added a README with install instructions for all the necessary packages, including an environment.yml
, with all required version numbers. I'm using anaconda, so hope that's not a problem.
I tested the instructions and it works on my system, which is Ubuntu.
I wrongly assumed that face_geometry
came from the mediapipe
library. That's why I was confused. It's working fine now. Thanks for the clarification.
By the way, do you know what units are used by the vectors returned by cv2.solvePnP()
? I believe the rotation vector is expressed in radians but what about the translation vector? Are the values pixels or some other relative unit?
Thanks again.
Not sure what the units are, but the unit of the Canonical Face Model from mediapipe is in cm. The vectors describe a translation/rotation from the Canonical Face Model to the 3D detected face (from model coordinate system to camera coordinate system).
(see here for a png version of the Canonical Face Model with indices of the points, same indices as held by the variable, points_idx
, in the code)
The documentation of solvePnP says:
rvec
| Output rotation vector (see Rodrigues ) that, together with tvec , brings points from the model coordinate system to the camera coordinate system.
tvec
| Output translation vector.
The two vectors rotation_vector
and translation_vector
(rvec
and tvec
) are very similar to the pose_transform_mat
obtained from the get_metric_landmarks()
of my python mediapipe version. They actually should be the same, e.g. pose_transform_mat[:3,:3]
and cv2.Rodrigues(rotation_vector)[0]
should be the same, and also the last column of pose_transform_mat
should be the same as tvec
. However, somewhere a sign flip creeps in, see my comment here. This also means, one actually does not need the cv2.solvePnP()
call, because all the information is already available in the pose_transform_mat
, just in some kind of flipped coordinate system. Thus, one can just get the output of cv2.solvePnP()
by looking at pose_transform_mat
and applying some sign flips here and there. This again means that pose_transform_mat
is in the same metric space as rotation_vector
and translation_vector
.
Sorry, I went on a tangent here, but (hence the above) my best guess is that rotation_vector
is in radians and translation_vector
is in cm.
If you'd like to read up on "3D reconstruction" of camera pictures see here, scroll a bit down to Detailed Description, where the camera calibration is explained. Or here
When I try to run
head_posture_rt.py
, I get the following error:Is there anything else to install besides mediapipe?
P.S. Thanks for these examples, by the way!