search

rizkiarm / LipNet

Keras implementation of 'LipNet: End-to-End Sentence-level Lipreading'
MIT License
635 stars 226 forks source link

Evaluation Protocol of The Results #16

Open chaofengc opened 7 years ago

chaofengc commented 7 years ago

Thank you for your work ! Based on your code, I got something like this in stats.csv.

Epoch,Samples,Mean CER,Mean CER (Norm),Mean WER,Mean WER (Norm),Mean BLEU,Mean BLEU (Norm)                                                                                                         
0,256,19.73047,0.79531,5.81250,0.96875,0.34846,0.34846
1,256,17.49609,0.70680,5.70312,0.95052,0.40687,0.40687

It seems that the CER is calculated based on only 256 samples. I write a simple script based on your functions and model to check the results on all unseen speakers (1, 2, 20, 22), 3971 samples in total. And I can only get about 0.12 CER.

Can you provide results evaluate on all unseen speakers examples ? Thank you.