albertz
commented
4 years ago Thank you for the interest in our experiments.
We compared Gammatone (GT) vs MFCC at some point (for hybrid BLSTM-HMMs) and GT was consistently a bit better (on Switchboard). I also compared GT (in RASR) to MFCC (via librosa) on Switchboard for attention models, and GT seems to be much better in my initial experiments. Although the problem seems to be that the model overfits more easily with MFCC for some reason, and probably we can compensate that with more regularization. But this needs more investigation. I also compare log-mel features vs MFCC (both via librosa) on Librispeech (with attention models), and MFCC seems to be better. But maybe this is a matter of more tuning.
So, certainly, you can do MFCC, or log-mel features. You can use our simple librosa-based feature extraction pipeline for that (e.g. via the OggZipDataset), but you probably need to retune then, and will not get as good results with the same config (without tuning). This would need some work (for the tuning, and also for setting up an OggZipDataset).
Setting up the RASR pipeline is certainly more complicated, but if you invest some time, this is also doable.
Here you find RASR config files (in config and flow), which should mostly work.
I think you should find some scripts with RASR which will do the feature extraction offline and store it in RASR cache files.
Our Librispeech pipeline is certainly much simpler. You can mostly take the existing attention setup (use one which uses ogg files, for faster speed and less disk space) for the dataset (i.e. train and dev in the RETURNN config) (and then also fix extern_data or num_outputs), and otherwise use the transducer config. We are currently trying this ourselves, but cannot share experience yet.
Also, we are currently setting up a RETURNN config which simplifies the transducer training pipeline. To get to our final result from the paper, you first start with full-sum training, then extract an alignment, and then do frame-wise cross entropy training. We published all necessary scripts and configs for that, but it's currently a somewhat manual process. We have prepared a config which will do the whole pipeline in one go. This is mostly ready (we are still testing it), and we can publish it soon. Maybe we can share it already with you, if you are interested. Note that this takes long to train. It might make sense to not redo the first part of the training all the time, and use an existing alignment, so you safe the first part of the training. We are also experimenting on how to speed this up.
Maybe @Spotlight0xff wants to add sth regarding transducer, or the new simplified config?
Regarding RASR, maybe @curufinwe wants to add sth, regarding which RASR to take, and where you can find scripts and configs for GT feature extraction?
yanghongjiazheng
Dear author, I find that you have given the config files about the experiment <2020-rnn-transducer>. And this experiment is based on SWB corpus. I tried using the this corpus reproduce your results. But I'm not familiar with the features extraction process. I have not build and installed RASR toolkit successfully yet. And I do not understand why you used the gammatone features but not the mfcc features. May I use Librispeech corpus and mfcc features to reproduce your result. Or could you share some config files about transducer experiments using Librispeech corpus and mfcc features? Thanks a lot.