search

1adrianb / face-alignment

:fire: 2D and 3D Face alignment library build using pytorch
https://www.adrianbulat.com
BSD 3-Clause "New" or "Revised" License
7.03k stars 1.34k forks source link

Where is the specific code of FAN #264

Closed yujun531 closed 3 years ago

yujun531 commented 3 years ago

Hello! In the face_alignment/api.py we can see the class FaceAlignment which is corresponding to 2 hourglass network and it can regress landmarks.

However, I could not run the specific code of FAN after download the model "2DFAN-4"(in # Initialise the face alignemnt networks and the url is https://www.adrianbulat.com/downloads/python-fan/2DFAN4-cd938726ad.zip).

And I even do not know how to read this code cause there is too many subfiles needed import in 2DFAN-4-1.6/code/torch/face_alignment/models/fan

so I want to know

  1. wheather the specific code of FAN is just "2DFAN-4"?
  2. and how to run this code
1adrianb commented 3 years ago

It really depends on what is the your end goal. The downloaded model already contains both the network architecture and the weights and as such running it directly should be possible, e.g:

 model = torch.jit.load('path_to_model_file.pth')
 out = model(image)

Note that the model in this form expects as input a facial image of 256x256 pixel and will output a set of 68 heatmaps, one for each keypoint. You can find the logic for detecting the face, cropping it, passing it to the network and finally of retrieving the keypoints from the heatmap here: https://github.com/1adrianb/face-alignment/blob/2bcfcc6ea20d0104d48942737639827fd3d5e816/face_alignment/api.py#L110

yujun531 commented 3 years ago

Thanks for your answer. I want to get the middle layer features of the second hourglass and then fuse these features into another network. If load it by using torch.jit.load self.face_alignment_net = torch.jit.load( load_file_from_url(models_urls.get(pytorch_version, default_model_urls)[network_name])) I can get the network but I could not get the middle layer features. There is only two '.pth' files

If manually decompressing '2DFAN4-cd938726ad.zip' , we get these files image

I do not know how to get the middle layers cause it even could not compile.

1adrianb commented 3 years ago

In this case, the best is to use the model definition from version 1.1: https://github.com/1adrianb/face-alignment/releases/tag/v1.1.1

There, the model is explicitly defined in python and you can directly adapt it's code to retrieve any intermediate features that you may need.

yujun531 commented 3 years ago

thank you very much！you really helped me a lot