search

pytorch / audio

Data manipulation and transformation for audio signal processing, powered by PyTorch
https://pytorch.org/audio
BSD 2-Clause "Simplified" License
2.49k stars 642 forks source link

Implement SpecAugment's Time Warping and SpecAugment wrapper #3622

Open gau-nernst opened 11 months ago

gau-nernst commented 11 months ago

🚀 The feature

TimeStretch is not in SpecAugment. It should be Time Warping instead. Time Warping does not change the spectrogram shape, but "warp" the content.

I want to propose the following changes:

  1. Remove the note that TimeStretch is proposed in SpecAugment. I think it is not correct.
  2. Implement Time Warping.
  3. Implement a SpecAugment wrapper that contains Time Warping, Frequency Masking, and Time masking, and accepts the same parameters as noted in the original paper (W, F, mF, T, p, mT). Perhaps the parameter name can be changed for better clarity.

Furthermore, we can also include SpecAugment policies as given by the paper i.e. LibriSpeech basic (LB), LibriSpeech double (LD), Switchboard mild (SM) and Switchboard strong (SS), so that the users can just call SpecAugment(policy="LB") for example.

Motivation, pitch

As stated above.

Alternatives

No response

Additional context

No response

pkorobov commented 3 days ago

Agree, according to the SpecAugment paper it should do sparse image warping on a spectrogram, but it seems to do quite a different thing