smeerten / ssnake

A program for the analysis of NMR data.
Other
20 stars 15 forks source link

ssnake freezes on windows when computing Czjzek simulation or ffm #156

Open jtrebosc opened 10 months ago

jtrebosc commented 10 months ago

ssnake freeze on large CPU workstation when computing Czjzek spectra or running ffm NUS calculation. Reason: multithreading is limited to 63 threads on windows.

Error message:

Exception in thread Thread-1 (_handle_workers): Traceback (most recent call last): File "C:\Users\choco0822\AppData\Local\anaconda3\Lib\threading.py", line 1038, in _bootstrap_inner self.run() File "C:\Users\choco0822\AppData\Local\anaconda3\Lib\threading.py", line 975, in run self._target(*self._args, **self._kwargs) File "C:\Users\choco0822\AppData\Local\anaconda3\Lib\multiprocessing\pool.py", line 522, in _handle_workers cls._wait_for_updates(current_sentinels, change_notifier) File "C:\Users\choco0822\AppData\Local\anaconda3\Lib\multiprocessing\pool.py", line 502, in _wait_for_updates wait(sentinels, timeout=timeout) File "C:\Users\choco0822\AppData\Local\anaconda3\Lib\multiprocessing\connection.py", line 878, in wait ready_handles = _exhaustive_wait(waithandle_to_obj.keys(), timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\choco0822\AppData\Local\anaconda3\Lib\multiprocessing\connection.py", line 810, in _exhaustive_wait res = _winapi.WaitForMultipleObjects(L, False, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ValueError: need at most 63 handles, got a sequence of length 130

There is a need for a maximum thread count variable that can be set by user.

jtrebosc commented 9 months ago

Tests on Threadripper (64 cores) show that bug is triggered when using 62 cores. So 61 core max can be used. Maybe some handles are used by main thread. So how to determine automatically the max core to be used ?