Inquiry about image preprocessing.

[StelaBou]

Hello,

I am trying to run your model on a different dataset, however the results are not as expected. I understand that reenactment methods are quite sensitive to face cropping/aligning, thus I'm facing problems to run your method on VoxCeleb1 due to such issues.

Could you provide more details on how to crop images around the detected faces?

Thank you in advance.

[MoustafaMeshry]

Hi Stela,

LSR is indeed sensitive to the face alignment and cropping of the input (e.g., notice the quality drop in our out-of-domain results compared to the voxceleb2 results). VoxCeleb2 does not include the details for how the dataset is preprocessed. So, to run LSR on external images, you need to manually try different face cropping.

Here is a sample code from when we were pre-processing out-of-domain images. If I recall correctly, you have to tune the values for shiftx, shifty, and scale. I hope this helps.

sys.path.insert(1, '/path/to/repo/two_step_synthesis_meshry/facial_landmark_gen')
sys.path.insert(2, '/path/to/repo/two_step_synthesis_meshry/face_segmentation/CelebAMask-HQ/face_parsing')

from standalone_landmark_2 import run_landmark_gen
from standalone_main import run_segmentation 
from standalone_merge import merge_util 

path = '/path/to/image/folder'  # Folder that contains images to be pre-processed.

inp_type = 'image'  # `image` for image directory 
use_cropped = 1  # Run landmark detection code on cropped image instead of the original image. 

if use_cropped:
    special_crop = 1
else:
    special_cropped = 0

# Tune the following 3 parameters to control the face cropping.
shiftx = 0
shifty = 0
scale = 1.6

run_landmark_gen(
    inp_type=inp_type,
    path=path,
    special_crop=special_crop,
    use_cropped=use_cropped,
    shiftx=shiftx,
    shifty=shifty,
    scale=scale)

vid_name = path.split("/")[-1]
vid_name = vid_name.split(".")[0]
processed_path = "/".join(path.split("/")[:-1])
processed_path = os.path.join(processed_path, vid_name + "_processed_"+str(scale))

print(processed_path)
run_segmentation(path=processed_path, imsize=512)
run_segmentation(path=processed_path, imsize=256)
merge_util(path=processed_path)