Clarification on Eye and Lip Landmarks and Implementation of $L_{guided}$ Loss Function

[Yang011013]

Hello, which landmarks on the eyes and lips correspond to the $l_i$ landmarks here?

Are $x{s,i,:2}$ and $x{d,i,:2}$ selected from the implicit keypoints which calculated by the following equations:

Can the final $L_{guided}$ be written as in the following code?

        if self.loss_weights['landmark_guided'] != 0:
            eyes_lip_landmarks = sorted([36,40,39,90,94,89,52,62,54,72]) # selected landmarks taken from the eye and lip
            L_d_i = [x['driving_landmark'][:, i, :2] for i in eyes_lip_landmarks]
            L_s_i = [x['source_landmark'][:, i, :2] for i in eyes_lip_landmarks]
            value = (wing_loss(L_d_i, kp_driving[:, :10, :2]) + wing_loss(L_s_i, kp_source[:, :10, :2])) / 2
            loss_values['landmark_guided'] = value

[cleardusk]

Thanks for your attention.

We use 3 landmarks on the left eye, 3 landmarks on the right eye, and 4 landmarks on the mouth. But I think it’s not restricted to these selections. You have the flexibility to choose which parts to emphasize based on your specific requirements.