search

anothermartz / Easy-Wav2Lip

Colab for making Wav2Lip high quality and easy to use
670 stars 106 forks source link

RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same #81

Open AlexHe99 opened 2 months ago

AlexHe99 commented 2 months ago

I am trying to use AMD GPU and ROCm to run it. But I got this issue when try ./run_loop.sh

(wav2lip) amd@AIG-PM:/DATA/Easy-Wav2Lip$ pip list | grep torch pytorch-triton-rocm 3.0.0 torch 2.4.0+rocm6.1 torchaudio 2.4.0+rocm6.1 torchvision 0.19.0+rocm6.1 (wav2lip) amd@AIG-PM:/DATA/Easy-Wav2Lip$ python /DATA/query_gpu.py 2.4.0+rocm6.1 Using GPU: AMD Radeon PRO W7900 GPU details: _CudaDeviceProperties(name='AMD Radeon PRO W7900', major=11, minor=0, gcnArchName='gfx1100', total_memory=46064MB, multi_processor_count=48)

(wav2lip) amd@AIG-PM:/DATA/Easy-Wav2Lip$ ./run_loop.sh opening GUI Saving config Starting Easy-Wav2Lip... Processing vTest.mp4 using TEn_sya.wav for audio imports loaded! analysing audio... 675 frames to process Using face detection data from last input Starting... Processing Wav2Lip: 0%| | 0/760 [00:01<?, ?it/s] Traceback (most recent call last): File "/DATA/Easy-Wav2Lip/inference.py", line 781, in main() File "/DATA/Easy-Wav2Lip/inference.py", line 703, in main pred = model(mel_batch, img_batch) File "/home/amd/anaconda3/envs/wav2lip/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(*args, kwargs) File "/home/amd/anaconda3/envs/wav2lip/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(*args, *kwargs) File "/DATA/Easy-Wav2Lip/models/wav2lip.py", line 96, in forward audio_embedding = self.audio_encoder(audio_sequences) # B, 512, 1, 1 File "/home/amd/anaconda3/envs/wav2lip/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/home/amd/anaconda3/envs/wav2lip/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(*args, kwargs) File "/home/amd/anaconda3/envs/wav2lip/lib/python3.10/site-packages/torch/nn/modules/container.py", line 219, in forward input = module(input) File "/home/amd/anaconda3/envs/wav2lip/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/home/amd/anaconda3/envs/wav2lip/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(args, kwargs) File "/DATA/Easy-Wav2Lip/models/conv.py", line 16, in forward out = self.conv_block(x) File "/home/amd/anaconda3/envs/wav2lip/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(*args, kwargs) File "/home/amd/anaconda3/envs/wav2lip/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(*args, *kwargs) File "/home/amd/anaconda3/envs/wav2lip/lib/python3.10/site-packages/torch/nn/modules/container.py", line 219, in forward input = module(input) File "/home/amd/anaconda3/envs/wav2lip/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/home/amd/anaconda3/envs/wav2lip/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(*args, **kwargs) File "/home/amd/anaconda3/envs/wav2lip/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 458, in forward return self._conv_forward(input, self.weight, self.bias) File "/home/amd/anaconda3/envs/wav2lip/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 454, in _conv_forward return F.conv2d(input, weight, bias, self.stride, RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same Processing failed! :( see line above 👆 Consider searching the issues tab on the github: https://github.com/anothermartz/Easy-Wav2Lip/issues

thisisvk45 commented 2 months ago

The error occurs because the model and input tensors are on different devices (CUDA vs CPU).

Fix: Ensure both are on the same device (AMD ROCm in your case).

In inference.py, update the model and inputs to use ROCm properly by moving both to the same device:

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = model.to(device)
mel_batch = mel_batch.to(device)
img_batch = img_batch.to(device)

This ensures the model and inputs are on the ROCm-supported GPU.

AlexHe99 commented 2 months ago

The error occurs because the model and input tensors are on different devices (CUDA vs CPU).

Fix: Ensure both are on the same device (AMD ROCm in your case).

In inference.py, update the model and inputs to use ROCm properly by moving both to the same device:

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = model.to(device)
mel_batch = mel_batch.to(device)
img_batch = img_batch.to(device)

This ensures the model and inputs are on the ROCm-supported GPU.

Yes. I debug and find that model is not in GPU and fixed it by https://github.com/anothermartz/Easy-Wav2Lip/commit/9a4dda75a8f7244bf10c56a7f431aac6398c43e7 image