AdolfVonKleist / Phonetisaurus

Phonetisaurus G2P
BSD 3-Clause "New" or "Revised" License
449 stars 122 forks source link

converting a whole sentence #34

Closed DLZML001 closed 5 years ago

DLZML001 commented 6 years ago

Hi, I was wondering if I need to convert a whole sentence, better put all the words on the same line separated by space, or every word on 1 line? since the results obtained are different in each case? for example it always converts the word 'be' to the phonemes 'B' and not to the phonemes 'B IY1'.

Thanks

AdolfVonKleist commented 6 years ago

Using the default model/behavior you need to send each word, one by one. The default model and training recipe does not learn any crossword context.

You should be able to do this pretty easily with the servlet (https://github.com/AdolfVonKleist/Phonetisaurus/blob/master/python/script/g2pserver.py)

In theory you could train a model from full sentences, assuming you have the corresponding monophone sequences (e.g. from the phoneme-aligned output of an ASR system like Kaldi). But you would need to take care to vary the length of the training segments, and make sure none are longer than a few hundred to a thousand characters (to prevent blowup when building the training lattices). In this case you would also need to train a much longer span n-gram model so as to actually learn the cross-word contexts in a useful way (probably something like max order 12-17 in English to ensure you get at least an average 2 gram span in terms of words).

I have never tried this myself for g2p, but the above works quite well for punctuation normalization in various languages.

DLZML001 commented 6 years ago

Thank you for your quick answer. What i am using is actually 'phonetisaurus-apply', giving it one text file as input with one word per line. And I am getting the output. However do you have any idea why for the word 'be' it is giving only phoneme 'B'?