search

festvox / flite

A small fast portable speech synthesis system
Other
859 stars 186 forks source link

about english cmu_lts_model.c and cmu_lex_data_raw.c #35

Open zshy1205 opened 4 years ago

zshy1205 commented 4 years ago

I have a question,can you help me? why you cut the cmudict, and only 36964 english words in cmu_lex_data_raw.c I know the cmudict contains 130000 english words, and I test the cmu_lts_model, it was performed poorly in cmu_lex_data_raw.c's 36964 words, about 90% word error rate. Why does this happen?(the cmu_lts_model is trained with cmudict which is removed the 36964 words? can you help me? thanks. Forgive my poor English.

fxwderrick commented 4 years ago

@zshy1205 I found the same issue. Not sure why(maybe for smaller footprint). But you can extend/create larger cmulex dictionary following this blog https://boredomed.wordpress.com/2019/03/07/festvox-to-flite-tts-conversion/. There do exist a certain amount of pronunciation errors, even in the latest cmudict. And many words are not covered in the dictionary(that's why we need to model letter-to-sound for pronunciation guess). I suppose you could manually update the cmudict to fix them.

zshy1205 commented 4 years ago

@fxwderrick Thanks, And I have one another question. There are some words have two or more Phoneme list, when you train the G2P model with cmudict, how to do with this case?

fxwderrick commented 4 years ago

@zshy1205 If you follow the steps in the blog, you will find the raw cmudict needs to be preprocessed first, like removing polyphone. Thus the input dict used for G2P training contains only one pronunciation for each word.

zshy1205 commented 4 years ago

@fxwderrick thank u very much, I will read this blog carefully.

bringtree commented 4 years ago

@zshy1205 @fxwderrick I delete the polyphone. But the error rate is still high.

bringtree commented 4 years ago

I want to know how to use allowable and cmu_dict(0.4) to reproduce the good result. (the same as ./bin/t2p)

bringtree commented 4 years ago

@zshy1205 I check the error rate about CART. In the training data, the word error rate is about 60%. I also use sklearn package to reproduce this process in Python. The word error rate is about 60% in training data. So I think the word error rate is normal. However, when I use some unseen words(Out of vocabulary in cmudict_0.4), the python version code' result is better than the flite(word error rate: 100%, letter error rate: 55%). It is Unnormal.

bfs18 commented 1 year ago

http://cmuflite.org/packed/flite-1.4/flite-1.4-release.tar.bz2 replacing tools/make_lts.scm with the corresponding file in the tarball before converting lts model to c format solves the problem.