Closed alexv-ds closed 1 year ago
No, the decoder is not thread-safe, lots of internal state. I created a multi-threaded version for my PhD thesis work but from what I recall this involved one decoder per thread (but it's been a long time...)
The memory-intensive bits (basically the backpointer table) absolutely need to be thread-local and none of the rest is particularly significant. For instance the acoustic model files are typically accessed as read-only shared memory.
You can, of course, use the same decoder in multiple processes by creating it before you call fork
, which is quite easy to do in Python and works well.
You can, of course, use the same decoder in multiple processes by creating it before you call
fork
, which is quite easy to do in Python and works well.
When you say fork you mean linux and the fact that it uses copy-on-write optimization of memory pages and since the data of the model is read-only, then this way I will save memory. Did I understand correctly?
Yes, exactly! Only works on Unix though (and not MacOS if you are using the Python module...)
Thanks!😊
Is it possible to use one decoder in a multi-threaded application, so as not to create new instances to save memory?