kpu
commented
4 years ago This is cutting it close especially if you have other things loaded; I would recommend converting the original ARPA to binary format again with better compression options https://neural.mt/code/kenlm/structures/.
You might be able to raise the virtual memory limit with ulimit -v followed by some number of kilobytes (which should be much greater than 2953349384/1024 because you also need space for code). Keep in mind it will likely be slow because you're getting close to swapping. Then run the code in the same shell.
Hello, when i run https://github.com/libai3/masr.git project on raspberry pi (4GB RAM), i got the memory error ,but when i run it on my pc (ubuntu 18.04 16GB RAM),it works well. i want to konw if i can modify some argument on kenlm to fit my 4GB memory or sth else wrong.Thanks!
$ python3 examples/demo-server.py Loading model... Traceback (most recent call last): File "examples/demo-server.py", line 10, in <module> import beamdecode File "/home/pi/masr/examples/../beamdecode.py", line 29, in <module> blank_index, File "/home/pi/.local/lib/python3.7/site-packages/ctcdecode/__init__.py", line 18, in __init__ self._num_labels) RuntimeError: third_party/kenlm/util/mmap.cc:122 in void* util::MapOrThrow(std::size_t, bool, int, bool, int, uint64_t) threw ErrnoException because (ret = mmap(__null, size, protect, flags, fd, offset)) == ((void *) -1). Cannot allocate memory mmap failed for size 2953349384 at offset 0