search

dmort27 / epitran

A tool for transcribing orthographic text as IPA (International Phonetic Alphabet)
MIT License
630 stars 121 forks source link

Problem with 'fra-Latn' for some words #87

Open pierebean opened 3 years ago

pierebean commented 3 years ago

'C'est une belle' gives 'k'ɛst ynə bɛlə' which is incorrect for 'C'est' is pronounced 'se'.

'Russe blanc' gives 'rysə blank' which is incorrect for 'blanc' is pronounced 'blɑ̃'.

epitran.Epitran('fra-Latn').transliterate('C\'est un russe blanc')
Out[76]: "k'ɛst yn rysə blank"
digitalheir commented 2 years ago

Also, the test for 'expansion' (expected 'ɛkspɑ̃sjɔ̃', is 'ɛkspâsjɔ̃') fails.

The mapping ẫ,â is made because it's longer than â,ɑ. Is it intentional? Then test_french.py should be updated.

martino-vic commented 2 years ago

epitran is a rule based transcriber so for some languages it works better than for others. The older an orthography the more complicated the rules and the worse epitran works. That's also why the English transcription is based on a third party library that uses a grapheme2phoneme (short: g2p) AI model. See here for an example. The French orthography is also old with complicated rules, so the best way to transcribe it is to use something like e-speak-ng (supports even transcription of Belgian and Swiss French!). The disadvantage of g2p models is that you need a considerably large amount of data to train your AI model. Since, however, the vast majority of languages in the world is underdocumented and has a rather young orthography with simple rules, epitran is still extremely useful.

X-Ryl669 commented 8 months ago

From my own tests, it seems you need to:

  1. Split your sentences in words (see #61 remark where it's said that Epitran is designed to transliterate only words)
  2. Remove any punctuation mark. If you try "Cest un russe blanc", you'll get "s œ̃ rysə blɑ̃" (which is close to se œ̃ ʀys blã)
  3. Yet, small word like articles don't work either if alone ("te" gives "t" and not "tə" and "cest" gives "s" and not "se") but work well if coupled with noun (why?)