search

manmay-nakhashi / tortoise-tts-fastest

Faster Tortoise inference then Tortoise Fast Fork
GNU Affero General Public License v3.0
122 stars 9 forks source link

Last words of some sentences keep being repeated #5

Open libTorrentUser opened 1 year ago

libTorrentUser commented 1 year ago

First of all, congrats for the awesome job you did speeding up tortoise. Tortoise is, hands down, the best tts engine out there and you (and the tortoise-tts-fast project) have made it usable. Now, with that out of the way...

I seem to have encountered a rather peculiar bug - sometimes the end of a sentence gets repeated. Here is one example

I was now coming near the gates and it seemed that our  journey was nearly over and we had escaped, when I suddenly thought I heard the sound of many marching feet and my father looking out through the darkness cried: Run, my son, run.

Using the last commit with

voice=train_dotrice
preset=ultra_fast
enable_redaction=False

results in the Run, my son, run. part being repeated. Now, I know Aeneas father is extremely worried about his son's safety, but I think the dude got it the first time he said it :)

I tried to run the conversion again and got the same result. I'm attaching a sample audio file repeat.zip

manmay-nakhashi commented 1 year ago

@libTorrentUser repetition is a known issue in autoregrassive models, and gpt-2 you can just regerate the sentence with different sentence and see , play with temperature and top_p params of autoregrassice.

libTorrentUser commented 1 year ago

I see. I didn't know that. I thought it could be something related to the sentence structure. I made tortoise-tts-fastest read 2 whole books for me and the repetition issue almost never happens. Probably less than 0.5% of the time, if not less.

I'll give your suggestion a try later - no NVIDIA card here. I depend on the kindness of strangers to use tortoise :) Now, assuming changing temperature and top_p solve the issue, should I use the new values for everything or just when I encounter problems like this?