redmod-team / profit

Probabilistic Response mOdel Fitting with Interactive Tools
https://profit.readthedocs.io
MIT License
14 stars 9 forks source link

Local Runner: `OSError: Too many open files` #150

Closed Rykath closed 2 years ago

Rykath commented 2 years ago

After @mkendler previously reported this bug, I reproduced this problem on my local machine with proFit 0.5.dev23+g038f0f5 for python 3.8.2.

Using the mockup example with ntrain: 2000 and ulimit -n set to 1024, run 1008 has the following error:

Traceback (most recent call last):
  File "/data/Data/Uni/BA/profit/profit/main.py", line 92, in main
  File "/data/Data/Uni/BA/profit/profit/run/default.py", line 63, in spawn_array
  File "/data/Data/Uni/BA/profit/profit/run/default.py", line 51, in spawn_run
  File "/usr/lib/python3.8/multiprocessing/process.py", line 121, in start
  File "/usr/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
  File "/usr/lib/python3.8/multiprocessing/context.py", line 276, in _Popen
  File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
  File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 69, in _launch
OSError: [Errno 24] Too many open files

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/Data/Uni/BA/profit/venv/bin/profit", line 33, in <module>
  File "/data/Data/Uni/BA/profit/profit/main.py", line 94, in main
  File "/data/Data/Uni/BA/profit/profit/run/default.py", line 96, in cancel_all
  File "/usr/lib/python3.8/multiprocessing/process.py", line 133, in terminate
AttributeError: 'NoneType' object has no attribute 'terminate'
Rykath commented 2 years ago

Workaround: After fixing #149, using the ZeroMQ Interface also prevents this bug.

→ The next suspect is the Memmap Interface which opens the shared file.

Rykath commented 2 years ago

Apparently PR #152 also fixes this as the memap interface is now initialized in another process.