IndexError: too many indices for tensor of dimension 2

[MiPlayer123]

Hi. I am running into the following error. I am wondering if you can help me.

[path]/asvspoof2021_air/feature_extraction.py:34: UserWarning: indexing with dtype torch.uint8 is now deprecated, please use a dtype torch.bool instead. (Triggered internally at  ../aten/src/ATen/native/IndexingUtils.h:28.)
  y[index] = (x[index] - a) / (b - a)
[path]/asvspoof2021_air/feature_extraction.py:37: UserWarning: indexing with dtype torch.uint8 is now deprecated, please use a dtype torch.bool instead. (Triggered internally at  ../aten/src/ATen/native/IndexingUtils.h:28.)
  y[index] = (c - x[index]) / (c - b)
Traceback (most recent call last):
  File "[path]/asvspoof2021_air/main_train.py", line 9, in <module>
    from dataset import *
  File "[path]/asvspoof2021_air/dataset.py", line 13, in <module>
    lfcc = LFCC(320, 160, 512, 16000, 20, with_energy=False)
  File "[path]/asvspoof2021_air/feature_extraction.py", line 87, in __init__
    self.l_dct = LinearDCT(filter_num, 'dct', norm='ortho')
  File "/[path]/asvspoof2021_air/utils_dsp.py", line 231, in __init__
    super(LinearDCT, self).__init__(in_features, in_features, bias=bias)
  File "[path]/.local/lib/python3.9/site-packages/torch/nn/modules/linear.py", line 101, in __init__
    self.reset_parameters()
  File "/[path]/asvspoof2021_air/utils_dsp.py", line 241, in reset_parameters
    self.weight.data = dct(I, norm=self.norm).data.t()
  File "[path]/asvspoof2021_air/utils_dsp.py", line 168, in dct
    V = Vc[:, :, 0] * W_r - Vc[:, :, 1] * W_i
IndexError: too many indices for tensor of dimension 2

Thanks

[yzyouzhang]

Could you please print the dimension of the tensor that cause this error? That would help me locate the issue. I guess it is a package version issue. Which version of PyTorch are you using? If you are using a very new one, you can refer to our new repo for processing audio features. https://github.com/yzyouzhang/HBAS_chapter_voice3/blob/main/audio_feature_extraction.py

[MiPlayer123]

Sorry, which tensor do you want me to print? I am using PyTorch 1.12.0 (latest) Should I replace existing feature extraction t with new audio feature extraction?

[yzyouzhang]

I mean the tensor that cause the error. I cannot locate that only from the information you sent. If I remember correctly, there is an abrupt change somewhere between PyTorch 1.1 and 1.10. I was using 1.1 in this project. If you are using 1.12, there must be some inconsistency that cause bugs. I suggest that you change to the new audio feature extraction. Thanks.

[MiPlayer123]

Thanks. I swapped out the feature extraction and somehow got ModuleNotFoundError: No module named 'ECAPA_TDNN', but I commented it out and there was no issue. My overall issue has changed from before, however:

Traceback (most recent call last):
  File "[path]/asvspoof2021_air/main_train.py", line 727, in <module>
    _, _ = train(args)
  File "[path]/asvspoof2021_air/main_train.py", line 247, in train
    feat, _, _, _, _ = training_set[23]
  File "[path]/asvspoof2021_air/dataset.py", line 58, in __getitem__
    filepath = self.all_files[idx]
IndexError: list index out of range

Here is relevant info for idx and the length: idx: 23 len: 0

[yzyouzhang]

This bug looks like the training set has no items. Did you change the path and check whether the extracted LFCCs are there?

[MiPlayer123]

Looks like extracted LFCCs are not there. I ran preprocess.py for train and dev with updated audio feature extraction with the following result: Let me know what info you need to resolve this.

[yzyouzhang]

Thanks for checking that. I just checked my code. Most of them are commented and the one left there is just for an example. Please think about what dataset you would like to augment and what augmentation method you would like to use, and I am sure you will find the code snippet from my commented code. If you just like to reproduce my paper without any change, please let me know which number in which table you are interested, and I can tell you how I augment the data specifically. Thanks.

[MiPlayer123]

Oh okay. Could you let me know the details of your model from Tables 4 and 5?

[MiPlayer123]

Also getting the following error from preprecess.py for raw dataset

Traceback (most recent call last):
  File "[dir]/asvspoof2021_air/preprocess.py", line 41, in <module>
    spec = torch.stft(320, 160, 512, 16000)
  File "[dir]/.local/lib/python3.9/site-packages/torch/functional.py", line 601, in stft
    signal_dim = input.dim()
AttributeError: 'int' object has no attribute 'dim'

[yzyouzhang]

Also getting the following error from preprecess.py for raw dataset

Traceback (most recent call last):
  File "[dir]/asvspoof2021_air/preprocess.py", line 41, in <module>
    spec = torch.stft(320, 160, 512, 16000)
  File "[dir]/.local/lib/python3.9/site-packages/torch/functional.py", line 601, in stft
    signal_dim = input.dim()
AttributeError: 'int' object has no attribute 'dim'

Which version of PyTorch are you using? Have you tried pytorch==1.1.0?

[yzyouzhang]

Oh okay. Could you let me know the details of your model from Tables 4 and 5?

Regarding the model we submitted, it is a fused model by different sub-models. Each sub-model uses a different augmentation, and you can find all of them in raw_dataset.py as different classes. And we extract features with preprocess.py.

[MiPlayer123]

Each sub-model uses a different augmentation, and you can find all of them in raw_dataset.py as different classes.

Got it.

Have you tried pytorch==1.1.0?

I did now, running into an issue I am unable to resolve. Do you have another config for raw dataset? Is the configuration for STFT right?

[yzyouzhang]

Could you please double check your PyTorch version? Since I see that you are using Python 3.9. It seems PyTorch may automatically match your Python version. I was using Python 3.6 and PyTorch 1.1.0.

[MiPlayer123]

I misunderstood a part of you code. I got preprocess working now. Will update once training starts. Thank you for your help