search

Rudrabha / Wav2Lip

This repository contains the codes of "A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild", published at ACM Multimedia 2020. For HD commercial model, please try out Sync Labs
https://synclabs.so
9.8k stars 2.13k forks source link

KeyError: state_dict #141

Closed bostjan39 closed 3 years ago

bostjan39 commented 3 years ago

Hi,

When running the script, right after s3fd pth file is loaded, it throws a KeyError: state_dict in load_model "s=checkpoint [state_dict]".

How to fix this?

Thanks!

prajwalkr commented 3 years ago

duplicate of https://github.com/Rudrabha/Wav2Lip/issues/142?

bostjan39 commented 3 years ago

Didn't see that issue, sorry. But how to check, if the checkpoint file is corrupted? I only downloaded it from the provided link, I know nothing about how to check it and what to check for (yup, a n00b). Thanks:)

prajwalkr commented 3 years ago

Re-download it once more and check again? The size should be 416 MB

bostjan39 commented 3 years ago

s3fd pth file has around 80MB, or am I missing something? Thanks!

prajwalkr commented 3 years ago

I think the error is coming when loading the Wav2lip checkpoint.

bodyhopper commented 3 years ago

I am also getting this error. I've tried to download the checkpoint file from both links in the readme, but it has failed with both. Both are around 85MB

prajwalkr commented 3 years ago

I think the error is coming when loading the Wav2lip checkpoint.

Please check if it is this. Please paste the complete log here

bodyhopper commented 3 years ago

Traceback (most recent call last): File "inference.py", line 279, in main() File "inference.py", line 251, in main model = load_model(args.checkpoint_path) File "inference.py", line 171, in load_model s = checkpoint["state_dict"] KeyError: 'state_dict'

This is how it looks like for me. Seems like it is the model

bostjan39 commented 3 years ago

Exactly my error too:)

prajwalkr commented 3 years ago

Yes, it is the Wav2Lip model.

What is the size of the downloaded checkpoint? And what is the command you are running? Can you paste that as well?

bostjan39 commented 3 years ago

The size is 87MB. Cmd is: python inference.py --checkpoint_path face_detection/detection/sfd/s3fd.pth --face myfiles/myvideo.mp4 --audio myfiles/myaudio.wav

Thanks!

prajwalkr commented 3 years ago

You are specifying the wrong checkpoint path. Please pass the Wav2lip checkpoint path for the argument --checkpoint_path

bodyhopper commented 3 years ago

python inference.py --checkpoint_path checkpoints/s3fd.pth --face test.mp4 --audio test.mp3

This is the command I am using that throws this 'state_dict' error. Omitting the path throws a different error. Both links to the checkpoint paths in the readme give ~85MB files

prajwalkr commented 3 years ago

Please download the Wav2Lip checkpoint (you are downloading the face detection checkpoint) and pass that for the --checkpoint_path argument.

bostjan39 commented 3 years ago

Ok, so I've tried by downloading the Wav2Lip with Gan package and put it into the checkpoints folder. Then I ran this: python inference.py --checkpoint_path checkpoints/wav2lip_gan.pth ... and although it went through, it didn't produce the result because of the missing ffmpeg module, which is another story. Did anyone had any luck finding and installing the ffmpeg module for py3?

Thanks!

janisgitendorfs commented 3 years ago

@bostjan39 You do not need to install ffmpeg on python3. You need it on your machine level so the command "ffmpeg" is available.

https://ffmpeg.org/download.html

bostjan39 commented 3 years ago

Hi, just did it and it works like a charm. Thanks!