Inferencer
commented
6 months ago It depends on the length of the beard but I'll give you some info on what I know,
I have a reference frame component that I will add soon which does I few things but related to your question allows all frames to use certain frames as reference, I made it to increase the quality of the teeth but did experiments with facial hair.
It can add facial hair (see Trump example below) to one frame that will then add the facial hair to the other frames so in theory it could also remove it.
If the issue is consistency because the current setup flickers or is not realistic then it should fix that I will share an example below of the appropriate length of facial hair, beyond that (if beard is too long) you would need to find a GAN (Generative adversarial network) or a filter to do the job.
You see see from the black box the limitations on the size of the face LipSick captures
https://github.com/Inferencer/LipSick/assets/121839197/bb6a4575-26d2-4eb7-a955-6a28d27336bb
https://github.com/Inferencer/LipSick/assets/121839197/97eea199-b119-480c-99e7-0758018a311f
So in summary if the beard is long and covering too much of the mouth making it impossible to lip-sync then yes it's possible to remove the beard around the mouth and lip-sync but then a filter or GAN will need to be used to bring back the beard..providing it covers the remaining existing beard under the chin that is no longer moving with the chin.
I was wondering if it would be possible to remove beard before sending to model then paste it on output video.