(NEW) Paper (Submitted to ICASSP 2024)
(NEW) Paper (APSIPA ASC 2023)
Paper (INTERSPEECH 2020)
Paper (IEEE/ACM TASLP)
Original codebase on ESPNet
egs/arctic/vc_urhythmic
.egs/hificaptain_jp/vc2
for details.egs/arctic/vc2
for details.egs/l2-arctic
for details.Sequence-to-sequence (seq2seq) modeling is especially attractive to voice conversion (VC) owing to their ability to convert prosody. In particular, this repository aim to reproduce the results of the following papers/models.
A non-autoregressive seq2seq VC method that can be directly trained on a parallel dataset without any pre-trained models.
We compared three methods for ground-truth-free FAC. All methods utilize seq2seq modeling and non-parallel frame-based VC models. We provide pre-trained vocoder and non-parallel frame-based VC models.
This is the first paper that applies the Transformer model to VC. In addition to the model architecture itself, the true novelty of this paper is actually a pre-training technique based on text-to-speech (TTS). This repository provides recipes for (1) TTS pre-training and (2) fine-tuning on a VC dataset. That is to say, TTS is also available in this repository.
Originally I open-sourced the code on ESPNet, but as it grows bigger and bigger, it becomes harder to conduct scientific research on ESPNet. Therefore, this repository aims to isolate the seq2seq VC part from ESPNet to become an independently-maintained toolkit (hopefully).
git clone https://github.com/unilight/seq2seq-vc.git
cd seq2seq-vc/tools
make
Same as many speech processing based repositories (ESPNet, ParallelWaveGAN, etc.), we formulate our recipes in kaldi-style. They can be found in the egs
folder. Please check the detailed usage in each recipe.
The experiments in the AAS-VC paper are conducted using the CMU ARCTIC dataset. Please refer to the readme file in egs/arctic/vc2
.
The CMU ARCTIC and L2-ARCTIC datasets are used. Please refer to the readme file in egs/l2-arctic
.
egs/ljspeech/tts1
.egs/arctic/vc1
.@INPROCEEDINGS{fac-evaluate,
author={Wen-Chin Huang and Tomoki Toda},
booktitle={Proc. APSIPA ASC},
title={{Evaluating Methods for Ground-Truth-Free Foreign Accent Conversion}},
year={2023},
}
@inproceedings{huang20i_interspeech,
author={Wen-Chin Huang and Tomoki Hayashi and Yi-Chiao Wu and Hirokazu Kameoka and Tomoki Toda},
title={{Voice Transformer Network: Sequence-to-Sequence Voice Conversion Using Transformer with Text-to-Speech Pretraining}},
year=2020,
booktitle={Proc. Interspeech},
pages={4676--4680},
}
@ARTICLE{vtn_journal,
author={Huang, Wen-Chin and Hayashi, Tomoki and Wu, Yi-Chiao and Kameoka, Hirokazu and Toda, Tomoki},
journal={IEEE/ACM Transactions on Audio, Speech, and Language Processing},
title={Pretraining Techniques for Sequence-to-Sequence Voice Conversion},
year={2021},
volume={29},
pages={745-755},
}
This repo is greatly inspired by the following repos. Or I should say, many code snippets are directly taken from part of the following repos.
Wen-Chin Huang
Toda Labotorary, Nagoya University
E-mail: wen.chinhuang@g.sp.m.is.nagoya-u.ac.jp