Shared-memory matrix

[ivonindza]

It's about optimizing memory usage when doing inference with multiple processes. Instead of each process loading the matrix, it is only loaded once in a shared memory region and used by all processes. This can also speed up the startup time if matrix is preloaded. The output matrix is not loaded at all as it is not needed during inference, which saves just under 50% of memory. There is no loss in speed, and potential savings in RAM can be huge.

[mpagli]

LGTM, nice feature, the code looks clean, will merge tomorrow if no complaint from other interested parties

[ivonindza]

Good catch, I forgot to update that