This is the official code repository for the paper "Neural HMMs are all you need (for high-quality attention-free TTS)". For audio examples, visit our demo page. pre-trained model (female) and pre-trained model (male) are also available.
data
folder such that the directory becomes data/LJSpeech-1.1
. Otherwise update the filelists in data/filelists
accordingly.git clone https://github.com/shivammehta25/Neural-HMM.git
gradient_checkpointing
it will help to fit bigger batch size during training.git clone --single-branch -b gradient_checkpointing https://github.com/shivammehta25/Neural-HMM.git
for that.git submodule init; git submodule update
pip install -r requirements.txt
bash start.sh
and it will install all the dependencies and run the container.src/hparams.py
for hyperparameters and set GPUs.
[0, 1 ..]
[]
python generate_data_properties.py
to generate data_parameters.pt
for your dataset (the default data_parameters.pt
is available for LJSpeech in the repository).python train.py
to train the model.
hparams.checkpoint_dir
.hparams.tensorboard_log_dir
.python train.py -c <CHECKPOINT_PATH>
synthesis.ipynb
.src.hparams.py
change hparams.precision
to 16
for mixed precision and 32
for full precision.
hparams.gpus
to [0, 1, 2]
for multi-GPU training and single element [0]
for single-GPU training.[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
, this is a known issue in PyTorch Dataloader.torch > 1.9.1
ImportError: cannot import name 'get_num_classes' from 'torchmetrics.utilities.data' (/opt/conda/lib/python3.8/site-packages/torchmetrics/utilities/data.py)
torch==1.11.0a0+b6df043
--extra-index-url https://download.pytorch.org/whl/cu113
torchmetrics==0.6.0
If you have any questions or comments, please open an issue on our GitHub repository.
If you use or build on our method or code for your research, please cite our paper:
@inproceedings{mehta2022neural,
title={Neural {HMM}s are all you need (for high-quality attention-free {TTS})},
author={Mehta, Shivam and Sz{\'e}kely, {\'E}va and Beskow, Jonas and Henter, Gustav Eje},
booktitle={Proc. ICASSP},
year={2022}
}
The code implementation is based on Nvidia's implementation of Tacotron 2 and uses PyTorch Lightning for boilerplate-free code.