search

LernerLab / GuPPy

Guided Photometry Analysis in Python, a free and open-source FP analysis tool.
GNU General Public License v3.0
63 stars 11 forks source link

Guppy "Read raw data" error - "handles" #22

Closed MannyEsguerra closed 2 years ago

MannyEsguerra commented 2 years ago

Hello:

I'm using latest Guppy as of 7/14/22. With one of my three current datasets, when I try to "Read Raw Data" I get this error and the GUI gets stuck (green progress bar keeps moving). Windows 10, running on AMD Ryzen Threadripper 3970x 32-core processor, 64 logical processors. Thank you for any help.

Manny E.

image

venus-sherathiya commented 2 years ago

does it work with other data which you have analyzed without errors before ?

MannyEsguerra commented 2 years ago

I just checked that, and now it does not work with data I've been able to analyze before. Now I get the same "handles" error.

On other forums I've seen people having problems with 64-core processors in packages like scikit and loky, and their solution has been to change a parameter "n_jobs" to 30 or less. Is there an equivalent in Guppy?

Example: https://github.com/pycaret/pycaret/issues/38

venus-sherathiya commented 2 years ago

thats what I was gonna suggest search for "mp.cpu_count()" in readTevTsq.py file and change it to "mp.cpu_count()-5" (as you have 32 cores) let me know if it works or not

MannyEsguerra commented 2 years ago

I decremented both instances of mp.cpu_count() in readTevTsq.py and Read Raw Data succeeded.

But now I get the same handles error for PSTH Computation:

Signal data and event timestamps are extracted.
Computing PSTH, Peak and Area for each event...
Average for group :  False
Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Users\Windows\anaconda3\envs\guppy\lib\threading.py", line 932, in _bootstrap_inner
    self.run()
  File "C:\Users\Windows\anaconda3\envs\guppy\lib\threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\Windows\anaconda3\envs\guppy\lib\multiprocessing\pool.py", line 519, in _handle_workers
    cls._wait_for_updates(current_sentinels, change_notifier)
  File "C:\Users\Windows\anaconda3\envs\guppy\lib\multiprocessing\pool.py", line 499, in _wait_for_updates
    wait(sentinels, timeout=timeout)
  File "C:\Users\Windows\anaconda3\envs\guppy\lib\multiprocessing\connection.py", line 879, in wait
    ready_handles = _exhaustive_wait(waithandle_to_obj.keys(), timeout)
  File "C:\Users\Windows\anaconda3\envs\guppy\lib\multiprocessing\connection.py", line 811, in _exhaustive_wait
    res = _winapi.WaitForMultipleObjects(L, False, timeout)
ValueError: need at most 63 handles, got a sequence of length 66
venus-sherathiya commented 2 years ago

do the same thing in the files computePsth.py and findTransientsFreqAndAmp.py file I will make sure to change the code so that number of cores can become a user-defined parameter on the GUI

MannyEsguerra commented 2 years ago

All steps are working now after adjusting the cpu count. One question: Why is this version producing three outputs? They seem to be identical.

We really appreciate getting your help in almost real-time, especially with a looming grant resubmission deadline. Thank you so much.