search

hhj1897 / face_pose_augmentation

MIT License
116 stars 19 forks source link

Correspondance Map & Visualization #13

Closed Mayur28 closed 1 year ago

Mayur28 commented 1 year ago

Hi,

I just been working through your implementation and I am having difficulty understanding what does result["correspondence_map"]represent please? I noted that it has the same spatial dimensions as the input image, however, this result has 2 channels (and displaying each channel independently does not offer any insight).

Also as a side note, I just wanted to find out how can I generate visualizations of the 3D mesh as presented in the paper please (such as the visualization shown below). NB, I have also referred to the paper High-fidelity pose and expression normalization for face recognition in the wild which uses these visualizations as well, however I was unable to produce these visualizations. image

Any assistance would be highly appreciated.

hhj1897 commented 1 year ago

The correspondence map shows the per-pixel correspondence between the input image and the warped output image (see cv2.remap or torch.nn.functional.grid_sample for more details). As the rotated face is produced by warping the input image, the correspondence map is what the algorithm really produces.

I'm afraid we don't expose the interface for producing this figure. But you can draw things like that by rendering the mesh (vertices + triangulations) represented by various variables in generate_profile_faces.

Mayur28 commented 1 year ago

Understood, thank you for your response.