CYGNUS-RD / reconstruction

Camera and scope analysis tools
0 stars 25 forks source link

Move from multi-thread to multi-process for ROOT happiness #187

Closed emanueledimarco closed 2 years ago

emanueledimarco commented 2 years ago

This seems something trivial, but the former shares the memory among the different threads, the second keeps the memory of each subprocess separate and this fixes completely the issue of the chunk TFiles being filled randomly (and provoking crash if one of the TFile is not opened when another asyncronous jobs want to write there).

So this is a crucial fix of PR #186 for running with python 3.8 with many many subprocesses at the LNGS queues.

This of course crashes on my laptop (only in multithread mode) with python3.9, but this is something for another nightmares night.