Closed fteufel closed 1 year ago
Embedding code isn't perfect. GENA-LM upsampling right now does not work well with large N segments as they do weird tokenization. This skips bad embeddings and prints a warning.
Embedding code isn't perfect. GENA-LM upsampling right now does not work well with large N segments as they do weird tokenization. This skips bad embeddings and prints a warning.