It turns out tflite conversion uses under the hood numexpr. Numexpr spawns a bunch of threads at module init. So if the main process converts a model, then forks and the resulting processes try converting a model, too, the state the forked processes work on appears like having been initialized. They hang waiting for results.
Using spawn and not forkserver because the latter still runs this risk. Spawning may be slower, but in the grand schema of things (training time), slightly larger init time is not that problematic.
It turns out tflite conversion uses under the hood numexpr. Numexpr spawns a bunch of threads at module init. So if the main process converts a model, then forks and the resulting processes try converting a model, too, the state the forked processes work on appears like having been initialized. They hang waiting for results.
Using
spawn
and notforkserver
because the latter still runs this risk. Spawning may be slower, but in the grand schema of things (training time), slightly larger init time is not that problematic.