Closed KevinWang676 closed 10 months ago
fixed in latest patch. unavailable tokens will just be skipped.
Hi, thank you so much for the fix! However, you may need to change the variable to cleaned_text
in line 40 and define sequence = []
in line 38. I fix the typos in my folk.
Also, I wonder if missing the symbols like the comma ,
would affect my training process and results. Thank you!
Hi @p0p4k, I wonder if missing some symbols like the comma ,
would affect my training process and results. Thank you!
You can add missing punctuations in the punctuations list and try training with that first.
Hi, does anyone know how to fix the
KeyError: ','
when training the Chinese dataset? I'm usingchinese_cleaners
defined here. Thank you!