Hi I'm running https://github.com/NVIDIA/NeMo/blob/main/scripts/asr_language_modeling/ngram_lm/train_kenlm.py and got (part of the log)
[NeMo I 2022-11-22 01:06:15 train_kenlm:105] Encoding the train file '/home/user/tmp.txt' ...
Read 0 lines: 1000 lines [00:00, 389334.82 lines/s]
Chunking 999 rows into 0.1219 tasks (each chunk contains 8192 elements)
Deleting previous file : /home/user/tmp.nemo.tmp.txt
[Parallel(n_jobs=-2)]: Using backend LokyBackend with 7 concurrent workers.
[NeMo W 2022-11-22 01:06:21 optimizers:55] Apex was not found. Using the lamb or fused_adam optimizer will error out.
[Parallel(n_jobs=-2)]: Done 1 tasks | elapsed: 6.0s
[Parallel(n_jobs=-2)]: Done 1 out of 1 | elapsed: 6.0s finished
Chunk : 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 478.80 chunks/s]
Finished writing 1 chunks to /home/user/tmp.nemo.tmp.txt. Current chunk index = 1
=== 1/5 Counting and sorting n-grams ===
Reading /home/user/tmp.nemo.tmp.txt
----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
-9
Traceback (most recent call last):
File "train_kenlm.py", line 165, in <module>
main()
File "train_kenlm.py", line 144, in main
raise RuntimeError("Training KenLM was not successful!")
RuntimeError: Training KenLM was not successful!
Hi I'm running
https://github.com/NVIDIA/NeMo/blob/main/scripts/asr_language_modeling/ngram_lm/train_kenlm.py
and got (part of the log)When I run
lmplz
directly I gotAny reason why this is happening and how could I solve it?