Tacotron2

NATURAL TTS SYNTHESIS BY CONDITIONING WAVENET ON MEL SPECTROGRAM PREDICTIONS https://arxiv.org/pdf/1712.05884.pdf

WaveNet: A Generative Model for Raw Audio https://arxiv.org/abs/1609.03499

Contents

• Simple LJ Speech DataLoader
• Mel Spectrogram Prediction network (text to Spectrogram)
• [TODO] WaveNet Vocoder (Spectrogram to raw audio)

Status

• Spectrogram network is functional but not fully trained. The model takes ~3 hours per epoch on an M6000 gpu.

Setup

1. install pytorch and torchvision:

   conda install pytorch -c pytorch

2. install other requirements:

   pip install -r requirements.txt

   Usage

   train Spectrogram Prediction Network

   python train.py

view logs in Tensorboard

tensorboard --logdir runs

Wavenet Resources

https://r9y9.github.io/wavenet_vocoder/ https://twitter.com/heiga_zen/status/832145314559750145 http://musyoku.github.io/2016/09/18/wavenet-a-generative-model-for-raw-audio/ https://www.slideshare.net/danilosoba1/generative-model-based-texttospeech