1adrianb / face-alignment

:fire: 2D and 3D Face alignment library build using pytorch
https://www.adrianbulat.com
BSD 3-Clause "New" or "Revised" License
6.88k stars 1.33k forks source link

[Feature Request] Add support for BlazeFace back camera model #269

Closed linkksng closed 1 year ago

linkksng commented 3 years ago

The current blazeface model uses the front camera model from hollance/BlazeFace-PyTorch. According to their readme, this model is suitable only for selfie using front camera, and is not very suitable for other types of portrait photos.

For the frontal camera model, only faces that occupy more than 20% of the image area were considered due to the intended use case (the threshold for the rear-facing camera model was 5%).

A while ago they have added another model for the rear camera, which is claimed to capture smaller faces in non-selfie photos. This issue is a feature request to add support for this back camera model.

I have tried implementing it myself successfully. My approach is as follows. Perhaps someone can come up with a more elegant way.

  1. Clone face_alignment/detection/blazeface. Say the new directory name is blazefaceback. This will be the name for the new model
  2. Update face_alignment/detection/blazefaceback/net_blazeface.py with latest code of blazeface.py from upstream.
  3. Update model_urls in face_alignment/detection/blazefaceback/blazeface_detector.py
  4. In face_alignment/detection/blazefaceback/detect.py, change the input image size from 128 to 256.

After some testing, for my dataset blazefaceback can identify small faces that blazeface cannot.

1adrianb commented 3 years ago

@linkksng For compatibility reasons, in case someone prefers the old behavior probably its best to preserve the old model and logic too and simply mark it as deprecated. Depending on how much the code changed, if its possible you can perhaps add the new model too in net_blazeface.py under a different code name while minimizing/avoiding the code repetition. Then, in face_alignment/detection/blazefaceback/blazeface_detector.py add a new flag to control which model to select.

Let me know if you are interested in making a pull request.

bryant0918 commented 1 year ago

The current blazeface model uses the front camera model from hollance/BlazeFace-PyTorch. According to their readme, this model is suitable only for selfie using front camera, and is not very suitable for other types of portrait photos.

For the frontal camera model, only faces that occupy more than 20% of the image area were considered due to the intended use case (the threshold for the rear-facing camera model was 5%).

A while ago they have added another model for the rear camera, which is claimed to capture smaller faces in non-selfie photos. This issue is a feature request to add support for this back camera model.

I have tried implementing it myself successfully. My approach is as follows. Perhaps someone can come up with a more elegant way.

  1. Clone face_alignment/detection/blazeface. Say the new directory name is blazefaceback. This will be the name for the new model
  2. Update face_alignment/detection/blazefaceback/net_blazeface.py with latest code of blazeface.py from upstream.
  3. Update model_urls in face_alignment/detection/blazefaceback/blazeface_detector.py
  4. In face_alignment/detection/blazefaceback/detect.py, change the input image size from 128 to 256.

After some testing, for my dataset blazefaceback can identify small faces that blazeface cannot.

In step 3 where did you get new model_urls for the blazeface rear-facing camera models? I would love to implement this but I can't find the models anywhere.

1adrianb commented 1 year ago

Blaze back camera model added (see readme for how to use it).