Open anvgarg opened 3 years ago
yes, the bug I found it too..., in inserting to trie command...
still no clue to solve it until now..., I will try ask in kenlm issue...
or maybe cc @alexeib have a solution for it?
i think kenlm by default only supports up to 5gram, and then you need to recompile it with diff flags to support more grams than that. dunno if that is the issue here though. stacktrace might be helpful
@wahyubram82 @anvgarg I was facing the same issue here, but I was able to get around with docker.
If you have problems related to locale settings (unicode errors in Python), you can try to execute the following commands in your container:
apt-get update -y
apt-get install --reinstall -y locales
sed -i 's/# pl_PL.UTF-8 UTF-8/pl_PL.UTF-8 UTF-8/' /etc/locale.gen
locale-gen pl_PL.UTF-8
export LANG=pl_PL.UTF-8
export LANGUAGE=pl_PL
export LC_ALL=pl_PL.UTF-8
I also experienced some issues with the python dataloader, to resolve it, you can try to infer using --num-workers 0
as infer.py
parameter.
Initially I was trying to build kenlm and the wav2letter python bindings but it didn't work. I am not sure if I made some mistake building these libraries or if following the instructions by tlikhomanenko in https://github.com/facebookresearch/wav2letter/issues/775 could cause some error. I needed to made that modification in order to viterbi work. I also use USE_CUDA=0 when building locally.
i think kenlm by default only supports up to 5gram, and then you need to recompile it with diff flags to support more grams than that. dunno if that is the issue here though. stacktrace might be helpful
Hello @alexeib, if I compile KENLM with -DKENLM_MAX_ORDER=25, I get a segmentation default (core dumped) or corrupted size vs prev_size error when running examples/speech_recognition/infer.py with a CTC Wav2Vec 2.0 model, Any word/char level LM and corresponding lexicon.
Please help
i think kenlm by default only supports up to 5gram, and then you need to recompile it with diff flags to support more grams than that. dunno if that is the issue here though. stacktrace might be helpful
Hello @alexeib, if I compile KENLM with -DKENLM_MAX_ORDER=25, I get a segmentation default (core dumped) or corrupted size vs prev_size error when running examples/speech_recognition/infer.py with a CTC Wav2Vec 2.0 model, Any word/char level LM and corresponding lexicon.
Please help
@alexeib It would be kind of you to reply
Has anyone found a solution to this problem? I get Segmentation fault (core dumped) when decoding with a 20-gram character based language model.
@maggieezzat @HJJ256 @alexeib @lucasgris @anvgarg Hi guys, I get Segmentation fault (core dumped) when run the infer.py:
from flashlight.lib.text.decoder imoprt KenLM
lm=KenLM(lm_path,word_dict)
I use the default setting about KENLM_MAX_ORDER still got this error :cry:
Hello everyone, has anyone solved this problem yet? When I use a 20-gram language model in kenLM, the following error occurs:
RuntimeError: /root/anaconda3/envs/meta/temp_dir/kenlm/lm/model.cc:49 in void lm::ngram::detail::{anonymous}::CheckCounts(const std::vector
🐛 Bug
To Reproduce
Steps to reproduce the behavior (always include the command you ran):
Code sample
Expected behavior
Environment
pip
, source): sourceAdditional context
I am trying to use wav2vec2.0 for my own dataset. While using the inference script it works fine with --w2l-decoder viterbi but gives me a segmentation fault when I try it with --w2l-decoder kenlm.I have followed all the installation steps listed in the wav2letter repository. I tried to debug and found that the trie inserts are causing a segmentation fault.