This prevents out-of-memory errors when the system's RAM is nearly full. With this commit, I can launch a worker which requires 2.4GB of RAM on a server with 2.6GB free; otherwise, I get an out of memory error. Once the LM has been loaded into memory, the footprint is huge so a fork is expensive.
This prevents out-of-memory errors when the system's RAM is nearly full. With this commit, I can launch a worker which requires 2.4GB of RAM on a server with 2.6GB free; otherwise, I get an out of memory error. Once the LM has been loaded into memory, the footprint is huge so a fork is expensive.