_pickle.UnpicklingError: invalid load key, 'v'.

[mayanktiwariiiitdmj]

When I am running the code:

import sda
import scipy.io.wavfile as wav
from PIL import Image

va = sda.VideoAnimator(gpu=0, model_path="crema")
fs, audio_clip = wav.read("example/audio.wav")
still_frame = Image.open("example/image.bmp")
vid, aud = va(still_frame, audio_clip, fs=fs)

va.save_video(vid, aud, "generated.mp4")

then I am getting the following error:

Traceback (most recent call last):
  File "init_testing.py", line 5, in <module>
    va = sda.VideoAnimator(gpu=0, model_path="crema")
  File "D:\PycharmProjects\init_speech2vid\sda\sda.py", line 109, in __init__
    model_dict = torch.load(model_path, map_location=lambda storage, loc: storage.cuda(gpu))
  File "C:\ProgramData\Anaconda3\envs\init_speech2vid\lib\site-packages\torch\serialization.py", line 595, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "C:\ProgramData\Anaconda3\envs\init_speech2vid\lib\site-packages\torch\serialization.py", line 764, in _legacy_load
    magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, 'v'.

please help me in solving this error.

[Mombin]

which version of python you use? did you locate .pth file correctly?

[mayanktiwariiiitdmj]

I am using Python3.8, Please tell me how to locate the .pth file. by now I have just

1. created a virtual env
2. executed the command pip install .
3. then after installation of the required packages I am running my code.

[Mombin]

    if model_path == "grid":
        model_path = os.path.split(__file__)[0] + "/data/grid.dat"
    elif model_path == "timit":
        model_path = os.path.split(__file__)[0] + "/data/timit.dat"
    elif model_path == "crema":
        model_path = os.path.split(__file__)[0] + "/data/crema.dat"

you should have one of these ".dat" file in your direcory. do you have this?

[mayanktiwariiiitdmj]

Yes, I have these files in the ./data/ location. Each file is of size 1KB.

[Mombin]

size of the model bigger than 1KB. I think you should correct ".dat" files from original source link try redownload model and test again.

DinoMan, the original source provider, provide these three model in this link

[mayanktiwariiiitdmj]

Thank you, I have downloaded the models from the link provided in the above answer. Now the error is gone but I am facing another exception:

Traceback (most recent call last):
  File "D:/PycharmProjects/init_speech2vid/init_testing.py", line 5, in <module>
    va = sda.VideoAnimator(gpu=0, model_path="crema")
  File "D:\PycharmProjects\init_speech2vid\sda\sda.py", line 109, in __init__
    model_dict = torch.load(model_path, map_location=lambda storage, loc: storage.cuda(gpu))
  File "C:\ProgramData\Anaconda3\envs\init_speech2vid\lib\site-packages\torch\serialization.py", line 595, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "C:\ProgramData\Anaconda3\envs\init_speech2vid\lib\site-packages\torch\serialization.py", line 774, in _legacy_load
    result = unpickler.load()
  File "C:\ProgramData\Anaconda3\envs\init_speech2vid\lib\site-packages\torch\serialization.py", line 730, in persistent_load
    deserialized_objects[root_key] = restore_location(obj, location)
  File "C:\ProgramData\Anaconda3\envs\init_speech2vid\lib\site-packages\torch\serialization.py", line 817, in restore_location
    result = map_location(storage, location)
  File "D:\PycharmProjects\init_speech2vid\sda\sda.py", line 109, in <lambda>
    model_dict = torch.load(model_path, map_location=lambda storage, loc: storage.cuda(gpu))
  File "C:\ProgramData\Anaconda3\envs\init_speech2vid\lib\site-packages\torch\_utils.py", line 71, in _cuda
    with torch.cuda.device(device):
  File "C:\ProgramData\Anaconda3\envs\init_speech2vid\lib\site-packages\torch\cuda\__init__.py", line 225, in __enter__
    self.prev_idx = torch._C._cuda_getDevice()
AttributeError: module 'torch._C' has no attribute '_cuda_getDevice'

Sir, can you please help me in solving this.

[Mombin]

it's version issue.

i guess you installed CPU only torch version. isn't it?

[mayanktiwariiiitdmj]

I have just executed the following command:

pip install .

sir, Do I need to install any other package for torch? like

conda install pytorch torchvision cudatoolkit=10.2 -c pytorch

[mayanktiwariiiitdmj]

I have solved all issues. I am able to run the project now. I have one more question: When I am running the code on my own images then I am not getting desired results. Can you please tell me why?? Is there any specific parameter that I need to change?

[mayanktiwariiiitdmj]

I have successfully installed all the packages and now the project is running. The project is working fine on the sample image provided in the project. But it is not working properly on my own set of images. Can you please tell me, what I need to change to do so?

[Mombin]

It's so sad that you tried hard. Unfortunately, this code well inference in domain of trained dataset. but it is not work well in out of trained data.

if you work better than now, you should train on your dataset. this code is too old. so I suggest that you can try another state of the art source if you want to try Talking Face(audio-video generation). try this code, so called Wav2Lip, It work pretty well. link Wav2Lip

[mayanktiwariiiitdmj]

Dear Sir, I have already configured and tested Wav2Lip. While going through the literature of the work, I came across your work, and then I tried to configure it. Thank you for helping me in the configuration process of speech2vid.