Move some of the loading code and processing code (anonymization, etc.) to its own files. @justinphan3110 began this and we can complete it. @edugp i created some automatic kenlm model loading code from your HF repo. I think it would be good to have this directly in the kenlm model file.
Move some of the loading code and processing code (anonymization, etc.) to its own files. @justinphan3110 began this and we can complete it. @edugp i created some automatic kenlm model loading code from your HF repo. I think it would be good to have this directly in the kenlm model file.