search

bootphon / phonemizer

Simple text to phones converter for multiple languages
https://bootphon.github.io/phonemizer/
GNU General Public License v3.0
1.18k stars 165 forks source link

language_switch and words_mismatch warning #98

Closed 123srikanth closed 2 years ago

123srikanth commented 2 years ago

Hi, I'm using phonemizer for generating hindi phonemes which I can use in building a TTS model but at some point in running the script the phonemizer stops proceeding further. I'm not sure what's happening but this is halting my work and these are the lines what I could see generating

2021-12-10 13:54:23,267 (words_mismatch:85) WARNING: words count mismatch on 100.0% of the lines (1/1) 2021-12-10 13:54:23,354 (language_switch:161) WARNING: 1 utterances containing language switches on lines 1 2021-12-10 13:54:23,354 (language_switch:163) WARNING: extra phones may appear in the "hi" phoneset 2021-12-10 13:54:23,354 (language_switch:165) WARNING: language switch flags have been removed (applying "remove-flags" policy) 2021-12-10 13:54:23,355 (words_mismatch:85) WARNING: words count mismatch on 100.0% of the lines (1/1) 2021-12-10 13:54:23,443 (language_switch:161) WARNING: 1 utterances containing language switches on lines 1 2021-12-10 13:54:23,444 (language_switch:163) WARNING: extra phones may appear in the "hi" phoneset 2021-12-10 13:54:23,444 (language_switch:165) WARNING: language switch flags have been removed (applying "remove-flags" policy) 2021-12-10 13:54:23,444 (words_mismatch:85) WARNING: words count mismatch on 100.0% of the lines (1/1) 2021-12-10 13:54:23,535 (language_switch:161) WARNING: 1 utterances containing language switches on lines 1 2021-12-10 13:54:23,535 (language_switch:163) WARNING: extra phones may appear in the "hi" phoneset 2021-12-10 13:54:23,535 (language_switch:165) WARNING: language switch flags have been removed (applying "remove-flags" policy) 2021-12-10 13:54:23,535 (words_mismatch:85) WARNING: words count mismatch on 100.0% of the lines (1/1) 2021-12-10 13:54:23,626 (language_switch:161) WARNING: 1 utterances containing language switches on lines 1 2021-12-10 13:54:23,626 (language_switch:163) WARNING: extra phones may appear in the "hi" phoneset 2021-12-10 13:54:23,626 (language_switch:165) WARNING: language switch flags have been removed (applying "remove-flags" policy) 2021-12-10 13:54:23,626 (words_mismatch:85) WARNING: words count mismatch on 100.0% of the lines (1/1) 2021-12-10 13:54:23,718 (language_switch:161) WARNING: 1 utterances containing language switches on lines 1 2021-12-10 13:54:23,719 (language_switch:163) WARNING: extra phones may appear in the "hi" phoneset 2021-12-10 13:54:23,719 (language_switch:165) WARNING: language switch flags have been removed (applying "remove-flags" policy) 2021-12-10 13:54:23,719 (words_mismatch:85) WARNING: words count mismatch on 100.0% of the lines (1/1) 2021-12-10 13:54:23,810 (language_switch:161) WARNING: 1 utterances containing language switches on lines 1 2021-12-10 13:54:23,811 (language_switch:163) WARNING: extra phones may appear in the "hi" phoneset 2021-12-10 13:54:23,811 (language_switch:165) WARNING: language switch flags have been removed (applying "remove-flags" policy) 2021-12-10 13:54:23,811 (words_mismatch:85) WARNING: words count mismatch on 100.0% of the lines (1/1)

can someone tell me what does this 4 lines mean 2021-12-10 13:54:23,810 (language_switch:161) WARNING: 1 utterances containing language switches on lines 1 2021-12-10 13:54:23,811 (language_switch:163) WARNING: extra phones may appear in the "hi" phoneset 2021-12-10 13:54:23,811 (language_switch:165) WARNING: language switch flags have been removed (applying "remove-flags" policy) 2021-12-10 13:54:23,811 (words_mismatch:85) WARNING: words count mismatch on 100.0% of the lines (1/1) and why is it not proceeding further in my script

mmmaat commented 2 years ago

Hi, this is explained in the docstring : https://github.com/bootphon/phonemizer/blob/7088b9314dcdb97adcf03eba89c4d9b411c8f701/phonemizer/phonemize.py#L118-L133

And by the way I guess your are calling phonemize line by line, this is inefficient, you better should have a single call to phonemize : https://github.com/bootphon/phonemizer#python-usage

123srikanth commented 2 years ago

can you please tell what happens when there are more number of calls to phonemize?

mmmaat commented 2 years ago

Internally a call to the top-level phonemize function is creating a backend instance. For espeak you are executing that each time you initialize an Espeak backend, this cost "a lot" of time, much more that the time spent to phonemize a little text.