TadasBaltrusaitis / OpenFace

OpenFace – a state-of-the art tool intended for facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation.
Other
6.96k stars 1.85k forks source link

Dear author, #597

Open zhao181 opened 6 years ago

zhao181 commented 6 years ago

Dear Author, I am recently reading the code about openface, Would you please tell me where the dxdR in the function CalcRigidJacobian come from? dxdR = [ 0, R(1,3), -R(1,2); -R(1,3), 0, R(1,1); R(1,2), -R(1,1), 0];

Is dxdR the jacobi matrix R(1,:) with respect to theta{x},theta{y} and $\theta$_{z}? If so, I am really puzzled about this result.

Looking forward to your reply. Thanks very much.

###########################################33 function J = CalcRigidJacobian(M, V, p, p_global)

n = size(M, 1)/3;

% Get the current 3D shape (not affected by global transform, as this
% is how the Jacobian was derived (for derivation please see
% ../derivations/orthoJacobian
shape3D = GetShape3D(M, V, p);

% Get the rotation matrix corresponding to current global orientation
R = Euler2Rot(p_global(2:4));
s = p_global(1);

% Rigid Jacobian is laid out as follows
% dx_1/ds, dx_1/dr1, dx_1/dr2, dx_1/dr3, dx_1/dtx, dx_1/dty
% dx_2/ds, dx_2/dr1, dx_2/dr2, dx_2/dr3, dx_2/dtx, dx_2/dty
% ...
% dx_n/ds, dx_n/dr1, dx_n/dr2, dx_n/dr3, dx_n/dtx, dx_n/dty
% dy_1/ds, dy_1/dr1, dy_1/dr2, dy_1/dr3, dy_1/dtx, dy_1/dty
% ...
% dy_n/ds, dy_n/dr1, dy_n/dr2, dy_n/dr3, dy_n/dtx, dy_n/dty

J = zeros(n*2, 6);

% dx/ds = X * r11  + Y * r12 + Z * r13
% dx/dr1 =  s*(r13 * Y - r12 * Z)
% dx/dr2 = -s*(r13 * X - r11 * Z)
% dx/dr3 =  s*(r12 * X - r11 * Y)
% dx/dtx = 1
% dx/dty = 0

% dy/ds = X * r21  + Y * r22 + Z * r23
% dy/dr1 =  s * (r23 * Y - r22 * Z)
% dy/dr2 = -s * (r23 * X - r21 * Z)
% dy/dr3 =  s * (r22 * X - r21 * Y)
% dy/dtx = 0
% dy/dty = 1

% set the Jacobian for x's

% with respect to scaling factor
J(1:n,1) = shape3D * R(1,:)';

% with respect to angular rotation around x, y, and z axes

% Change of x with respect to change in axis angle rotation
dxdR = [      0,  R(1,3), -R(1,2);
                     -R(1,3),       0,  R(1,1);
                    R(1,2), -R(1,1),      0];

J(1:n,2:4) = s*(dxdR * shape3D')';

% with respect to translation
J(1:n,5) = 1;
J(1:n,6) = 0;

% set the Jacobian for y's

% with respect to scaling factor
J(n+1:end,1) = shape3D * R(2,:)';

 % with respect to angular rotation around x, y, and z axes

% Change of y with respect to change in axis angle rotation
dydR = [      0,  R(2,3), -R(2,2);
        -R(2,3),       0,  R(2,1);
         R(2,2), -R(2,1),      0];

 J(n+1:end,2:4) = s*(dydR * shape3D')';

 % with respect to translation
 J(n+1:end,5) = 0;
 J(n+1:end,6) = 1;

end

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% function [Rot] = Euler2Rot(euler) rx = euler(1); ry = euler(2); rz = euler(3); Rx = [1 0 0; ... 0 cos(rx) -sin(rx); 0 sin(rx) cos(rx)]; Ry = [cos(ry) 0 sin(ry); 0 1 0; -sin(ry) 0 cos(ry)]; Rz = [cos(rz) -sin(rz) 0; sin(rz) cos(rz) 0; 0 0 1];

Rot = Rx * Ry * Rz;
TadasBaltrusaitis commented 6 years ago

Not exactly, it is the Jacobian of landmark locations w.r.t to rotation that is parametrized by axis angles.

It is easier to parametrize the affect of rotation in terms of axis angles than in terms of Euler angles. You can find more details of this parametrization here - https://www.cl.cam.ac.uk/~tb346/pub/thesis/phd_thesis.pdf specifically chapter 4.24

However, after the parameter update the axis angles are converted to Euler angles (although this is not strictly necessary)