One decoder in a multi-threaded application

[alexv-ds]

Is it possible to use one decoder in a multi-threaded application, so as not to create new instances to save memory?

[dhdaines]

No, the decoder is not thread-safe, lots of internal state. I created a multi-threaded version for my PhD thesis work but from what I recall this involved one decoder per thread (but it's been a long time...)

The memory-intensive bits (basically the backpointer table) absolutely need to be thread-local and none of the rest is particularly significant. For instance the acoustic model files are typically accessed as read-only shared memory.

[dhdaines]

You can, of course, use the same decoder in multiple processes by creating it before you call fork, which is quite easy to do in Python and works well.

[alexv-ds]

You can, of course, use the same decoder in multiple processes by creating it before you call fork, which is quite easy to do in Python and works well.

When you say fork you mean linux and the fact that it uses copy-on-write optimization of memory pages and since the data of the model is read-only, then this way I will save memory. Did I understand correctly?

[dhdaines]

Yes, exactly! Only works on Unix though (and not MacOS if you are using the Python module...)

[alexv-ds]

Thanks!😊