search

NVIDIA / mellotron

Mellotron: a multispeaker voice synthesis model based on Tacotron 2 GST that can make a voice emote and sing without emotive or singing training data
BSD 3-Clause "New" or "Revised" License
853 stars 187 forks source link

What is the reason of filtering "_" and "~" symbols? #125

Open kremnik opened 6 months ago

kremnik commented 6 months ago

Hi @rafaelvalle

In this line: https://github.com/NVIDIA/mellotron/blob/d5362ccae23984f323e3cb024a01ec1de0493aff/text/__init__.py#L102 you are filtering "_" and "~" symbols. Also, one of the main advice to improve alignment map convergence is to add special symbols to start and end of every sentence. Usually these symbols are exactly "_" and "~" (_ex: _What is your name?~_). But you filter out exactly these symbols and do not add them anywhere in the code.

It is interesting, that you've included the "_" symbol here: https://github.com/NVIDIA/mellotron/blob/d5362ccae23984f323e3cb024a01ec1de0493aff/text/symbols.py#L11 but anyway filter it out next in sentence preprocessing.

So the questions are: 1) What is the reason of filtering out "_" and "~" symbols? 2) Why don't you use them as start and end symbols in sentences?