the single result lip sync video seems not so good

[Wild-Piggggggg]

I got some problem. I tried a single video for lip sync, but the mouth and teeth is not satisfied.

https://github.com/soumik-kanad/diff2lip/assets/72304393/c7266710-29a9-488d-b665-02c49aa6b25f

And I only modified the inference.sh and run it on the terminal by scripts/inference.sh↓

[soumik-kanad]

Have you tried using the other checkpoint?

[Wild-Piggggggg]

Yes. I tried both checkpoints e7.15_model210000_notUsedInPaper.pt and e7.24.1.3_model260000_paper.pt .
And here comes another demo.o(╥﹏╥)o

https://github.com/soumik-kanad/diff2lip/assets/72304393/3509de64-3d73-4b64-bb8b-a19bc4d26c13

[dukesun99]

Confirm the same problem here. Generated teeth are quite strange.

[soumik-kanad]

@Wild-Piggggggg I don't see the flag --is_voxceleb2. Could you please try setting --is_voxceleb2=False. Also, what is the resolution of this video?

[chandarren]

When I start reasoning about the video, it always takes up all my RAM, I've added up to 48G of RAM and the video card isn't being used, why is that?

[Wild-Piggggggg]

@Wild-Piggggggg I don't see the flag --is_voxceleb2. Could you please try setting --is_voxceleb2=False. Also, what is the resolution of this video?

Yes, I already set --is_voxceleb2=False, and the resolution is 1080×1920

[chandarren]

这是来自QQ邮箱的假期自动回复邮件。你好，我最近正在休假中，无法亲自回复你的邮件。我将在假期结束后，尽快给你回复。

[soumik-kanad]

@Wild-Piggggggg I don't see the flag --is_voxceleb2. Could you please try setting --is_voxceleb2=False. Also, what is the resolution of this video?

Yes, I already set --is_voxceleb2=False, and the resolution is 1080×1920

Can you try at a lower resolution (maybe half) and see if that works?

[soumik-kanad]

When I start reasoning about the video, it always takes up all my RAM, I've added up to 48G of RAM and the video card isn't being used, why is that?

Can you please check where in the code you have the RAM problem? I'm guessing that might be because of face detection. If that is the case, please reduce the face detection batch size.

[Wild-Piggggggg]

@Wild-Piggggggg I don't see the flag --is_voxceleb2. Could you please try setting --is_voxceleb2=False. Also, what is the resolution of this video?

Yes, I already set --is_voxceleb2=False, and the resolution is 1080×1920

Can you try at a lower resolution (maybe half) and see if that works?

I see. I'll try it at a lower resolution.

[chandarren]

When I start reasoning about the video, it always takes up all my RAM, I've added up to 48G of RAM and the video card isn't being used, why is that?

Can you please check where in the code you have the RAM problem? I'm guessing that might be because of face detection. If that is the case, please reduce the face detection batch size.

Yes,My config file code reference is from the questioner(Wild-Piggggggg ![Uploading 1123.png…]() ) above and is the same as the code he used

[chandarren]

[dongyado]

@Wild-Piggggggg is this problem has been solved by reduce resolutions?

[delonleo]

Problem is not solved by reduced resolution.

[rdev2898]

@delonleo were you able to solve the issue?

[chandarren]

这是来自QQ邮箱的假期自动回复邮件。你好，我最近正在休假中，无法亲自回复你的邮件。我将在假期结束后，尽快给你回复。