Error Message: RuntimeError: Argument #4: Padding size should be less than the corresponding input dimension, but got: padding (1024, 1024) at dimension 2 of input [1, 65621, 2]

[GUUser91]

@yl4579 I added a extra line to the train_list.txt file and got this error message:

python train.py --config_path ./Configs/config.yml {'max_lr': 0.0005, 'pct_start': 0.0, 'epochs': 200, 'steps_per_epoch': 72} ctc_linear.2.linear_layer.weight does not have same shape torch.Size([178, 256]) torch.Size([80, 256]) ctc_linear.2.linear_layer.bias does not have same shape torch.Size([178]) torch.Size([80]) asr_s2s.embedding.weight does not have same shape torch.Size([178, 512]) torch.Size([80, 256]) asr_s2s.project_to_n_symbols.weight does not have same shape torch.Size([178, 128]) torch.Size([80, 128]) asr_s2s.project_to_n_symbols.bias does not have same shape torch.Size([178]) torch.Size([80]) asr_s2s.decoder_rnn.weight_ih does not have same shape torch.Size([512, 640]) torch.Size([512, 384])

Traceback (most recent call last): File "/home/bud/AuxiliaryASR/train.py", line 116, in main() File "/home/bud/AuxiliaryASR/venv/lib/python3.10/site-packages/click/core.py", line 1157, in call return self.main(args, kwargs) File "/home/bud/AuxiliaryASR/venv/lib/python3.10/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/home/bud/AuxiliaryASR/venv/lib/python3.10/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, ctx.params) File "/home/bud/AuxiliaryASR/venv/lib/python3.10/site-packages/click/core.py", line 783, in invoke return __callback(args, kwargs) File "/home/bud/AuxiliaryASR/train.py", line 98, in main train_results = trainer._train_epoch() File "/home/bud/AuxiliaryASR/trainer.py", line 186, in _train_epoch for train_steps_per_epoch, batch in enumerate(tqdm(self.train_dataloader, desc="[train]"), 1): File "/home/bud/AuxiliaryASR/venv/lib/python3.10/site-packages/tqdm/std.py", line 1182, in iter for obj in iterable: File "/home/bud/AuxiliaryASR/venv/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 630, in next data = self._next_data() File "/home/bud/AuxiliaryASR/venv/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1345, in _next_data return self._process_data(data) File "/home/bud/AuxiliaryASR/venv/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1371, in _process_data data.reraise() File "/home/bud/AuxiliaryASR/venv/lib/python3.10/site-packages/torch/_utils.py", line 694, in reraise raise exception RuntimeError: Caught RuntimeError in DataLoader worker process 2. Original Traceback (most recent call last): File "/home/bud/AuxiliaryASR/venv/lib/python3.10/site-packages/torch/utils/data/_utils/worker.py", line 308, in _worker_loop data = fetcher.fetch(index) File "/home/bud/AuxiliaryASR/venv/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 51, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/bud/AuxiliaryASR/venv/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 51, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/bud/AuxiliaryASR/meldataset.py", line 65, in getitem mel_tensor = self.to_melspec(wave_tensor) File "/home/bud/AuxiliaryASR/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/home/bud/AuxiliaryASR/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(args, kwargs) File "/home/bud/AuxiliaryASR/venv/lib/python3.10/site-packages/torchaudio/transforms/_transforms.py", line 619, in forward specgram = self.spectrogram(waveform) File "/home/bud/AuxiliaryASR/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/home/bud/AuxiliaryASR/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(args, **kwargs) File "/home/bud/AuxiliaryASR/venv/lib/python3.10/site-packages/torchaudio/transforms/_transforms.py", line 110, in forward return F.spectrogram( File "/home/bud/AuxiliaryASR/venv/lib/python3.10/site-packages/torchaudio/functional/functional.py", line 126, in spectrogram spec_f = torch.stft( File "/home/bud/AuxiliaryASR/venv/lib/python3.10/site-packages/torch/functional.py", line 648, in stft input = F.pad(input.view(extended_shape), [pad, pad], pad_mode) RuntimeError: Argument #4: Padding size should be less than the corresponding input dimension, but got: padding (1024, 1024) at dimension 2 of input [1, 65621, 2]

[yl4579]

Which line did you add? Does this error only happen when you have your extra line?

[GUUser91]

@yl4579 I just convert a wav file from this youtube video down to a 1 second 24000khz wav file. https://www.youtube.com/watch?v=GQt4SY_6-w4. Here is the converted wav file https://files.catbox.moe/2jfn9u.wav I also get the same error message if I add any more lines and wav files from this page. Note I also convert them to 24000 khz after downloading these wav files. https://huggingface.co/Bluebomber182/Judy-Hopps/tree/main/audio

[korakoe]

As I've found out... All files need to be mono! You can do this by adding:

# Convert to mono if needed
if np.ndim(wave) > 1:
    wave = np.mean(wave, axis=1)

Under sf.read() in _load_tensor()