search

sczhou / CodeFormer

[NeurIPS 2022] Towards Robust Blind Face Restoration with Codebook Lookup Transformer
Other
15.95k stars 3.36k forks source link

Hair cut off, some part of the image is blurred #406

Open LecterXIV opened 1 month ago

LecterXIV commented 1 month ago

Trying the Codeformer Whole Image Enhancement and getting amazing results but having some slight issue. It masks/crop the image into 512x512 everything inside will be restored. the face, including some of the hairs, however, everything outside of that 512x512x mask/cropped is blurred. Any help will be greatly appreciated. Thanks! Screenshot_1

mikecokina commented 1 week ago

@LecterXIV I faced the same issue in the face swap library. The explanation lies in the underlying processes that handle the cropped face part of the image, usually provided via the FaceRestoreHelper from facexlib. This tool detects the face, crops it out, and aligns it based on landmarks. Obviously, it also includes hair within the bounding box. After that, CodeFormer restores the entire image within the bounding box, including the hair, and everything is reconstructed in reverse order.

I made some changes in my library to replace only the facial parts, excluding the hair. Initially, I worked on a version that included hair, but it turned out to be quite tricky due to the inconsistent hair length and the way CodeFormer operates. You can find an example in the README on how to use it here: reactor-lib