pytorch / audio

Data manipulation and transformation for audio signal processing, powered by PyTorch
https://pytorch.org/audio
BSD 2-Clause "Simplified" License
2.54k stars 653 forks source link

Implement SpecAugment's Time Warping and SpecAugment wrapper #3622

Open gau-nernst opened 1 year ago

gau-nernst commented 1 year ago

🚀 The feature

TimeStretch is not in SpecAugment. It should be Time Warping instead. Time Warping does not change the spectrogram shape, but "warp" the content.

I want to propose the following changes:

  1. Remove the note that TimeStretch is proposed in SpecAugment. I think it is not correct.
  2. Implement Time Warping.
  3. Implement a SpecAugment wrapper that contains Time Warping, Frequency Masking, and Time masking, and accepts the same parameters as noted in the original paper (W, F, mF, T, p, mT). Perhaps the parameter name can be changed for better clarity.

Furthermore, we can also include SpecAugment policies as given by the paper i.e. LibriSpeech basic (LB), LibriSpeech double (LD), Switchboard mild (SM) and Switchboard strong (SS), so that the users can just call SpecAugment(policy="LB") for example.

Motivation, pitch

As stated above.

Alternatives

No response

Additional context

No response

pkorobov commented 2 months ago

Agree, according to the SpecAugment paper it should do sparse image warping on a spectrogram, but it seems to do quite a different thing