Phoneme from forced alignment is not in the gmm likes file

[rguliev]

From Yondu Tsai from kaldi-help google group.

If the phoneme from forced alignment in the alignment file is not in the gmm likes file, then there is a lookup error

'ali_prob': prob.lookup(np.arange(prob.shape[0]), ali),

[rguliev]

Hi Yondu,

How has that happened, that gmm likes does not have some phoneme? As far as I know, gmm-compute-likes provides probs for each pdf_id which are then mapped to phoneme. Could you provide more information?

[dpny518]

attached are the ali-to-phones and gmm-compute-likes for the audio and text hello one two three words.txt

ali-to-phones.txt gmm-compute-likes.txt

phones.txt

[rguliev]

Thanks. Could you also provide a mapping of pdf_id->phoneme, i.e output of pdf2phone function? It plays an important part in this case.

For convenience, a summary of the files:

• phones: 274 phones
  • ali-to-phones: 1 utterance with 305 frames
  • gmm-compute-likes: 305x2000

[dpny518]

pdf2symb.txt

[rguliev]

Thanks. So the problem is that pdf_id can't map to phoneme, thus pdf2symb is incorrect. For example:

Transition-state 1212: phone = 'aU_B hmm-state = 0 pdf = 1999
Transition-state 1234: phone = 'aU_E hmm-state = 0 pdf = 1999
Transition-state 1256: phone = 'aU_I hmm-state = 0 pdf = 1999
Transition-state 1278: phone = 'aU_S hmm-state = 0 pdf = 1999
Transition-state 1300: phone = aU_B hmm-state = 0 pdf = 1999
Transition-state 1322: phone = aU_E hmm-state = 0 pdf = 1999
Transition-state 1344: phone = aU_I hmm-state = 0 pdf = 1999
Transition-state 1366: phone = aU_S hmm-state = 0 pdf = 1999

As a workaround, you can remove _B, _E, _I, _S suffices and ' prefix. Then each pdf_id can be mapped to one phoneme. Here is an example code.

import pandas as pd
df = pd.read_csv("pdf2symb.txt", header=None, sep=" ", usecols=[4,10])
df["parsed_phone"] = (df.phone
                     .apply(lambda x: x.split("_")[0]) # Remove suffixes
                     .apply(lambda x: x[1:] if x.startswith("'") else x) # Remove `'`
                     )
df = df[["pdf", "parsed_phone"]].drop_duplicates()
pdf2symb = dict(zip(df["pdf"].astype(int), df["parsed_phone"]))

[rguliev]

@yondu22 Any updates here?)

[dpny518]

I just changed the model to be same as yours instead of using mine

[rguliev]

Which model do you mean? Can I close the issue then?:)

[dpny518]

Yes you can cLose, train model with phone position false