anonymous-pits / pits

PITS: Variational Pitch Inference for End-to-end Pitch-controllable TTS without External Pitch Predictor
https://anonymous-pits.github.io/pits/
MIT License
275 stars 34 forks source link

mat1 and mat2 shapes cannot be multiplied #20

Open ppisljar opened 1 year ago

ppisljar commented 1 year ago

I am running into this error:

-- Process 0 terminated with the following error:                                                                                                                                                               │
Traceback (most recent call last):                                                                                                                                                                              │
  File "/opt/conda/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 69, in _wrap                                                                                                               │
    fn(i, *args)                                                                                                                                                                                                │
  File "/home/tts/pits/train.py", line 174, in run                                                                                                                                                              │
    train_and_evaluate(rank, epoch, hps, [net_g, net_d],                                                                                                                                                        │
  File "/home/tts/pits/train.py", line 235, in train_and_evaluate                                                                                                                                               │
    y_hat_mel = mel_spectrogram_torch(                                                                                                                                                                          │
  File "/home/tts/pits/mel_processing.py", line 119, in mel_spectrogram_torch                                                                                                                                   │
    spec = torch.matmul(mel_basis[fmax_dtype_device], spec)                                                                                                                                                     │
RuntimeError: mat1 and mat2 shapes cannot be multiplied (80x513 and 48x513)

i left config unmodified but i did change the symbol file to reflect symbols from my alphabet. i also did two updates for which i submited pull requests as i coldn;t get training to start without those.

ppisljar commented 1 year ago

i am guessing the reason is wrong torch version ? (for which i had to do https://github.com/anonymous-pits/pits/pull/18) i am starting this from your Dockerfile. what torch version was this tested with ?

anonymous-pits commented 1 year ago

Hi! I did exactly same version with dockerfile. It seems that you set True to retrun_complex and the code is based on the spectrogram with the size [B,F,T,2]. Thus, your complex spectrogram remove T instead of 2 in the line spec = torch.sqrt(spec.pow(2).sum(-1) + 1e-6). If possible, you need to downgrade torch version or change above line to spec = torch.sqrt((spec * spec.conj()).to(torch.float) + 1e-6)