raphaelvallat / yasa

YASA (Yet Another Spindle Algorithm): a Python package to analyze polysomnographic sleep recordings.
https://raphaelvallat.com/yasa/
BSD 3-Clause "New" or "Revised" License
417 stars 113 forks source link

Slow wave detection reproducibility question #146

Closed acorbellini closed 8 months ago

acorbellini commented 1 year ago

Hi!

This is more a question than a bug, so feel free to close it if not relevant at all! I'm not an expert so some parts of the explanation might not be accurate.

I had some issues reproducing sw_detect on different machines because of, presumably, difference in floating point arithmetic. I had this scenario:

  1. There is a slow wave at the very beginning of the input data (my data has discontinuous sections of eeg readings)
  2. The first part of the slow wave was going down

In my example, after filtering, the data_filt vector (inside the sw_detect code) contains either:

a) a zero (or maybe a very small negative number) at the beginning of the array, then there was no crossing at the beginning, so that slow wave was not detected. b) some small positive number (something around 1e-14). This depends on the library/machine where I ran the code, then the slow wave was detected due a crossing at the beginning of the data.

Just to make the tests reproducible in different computers, I added a temporary fix after filtering in sw_detect (in line 1647 for example in detection.py):

data_filt[data_filt < 1e-10] = 0

I'm not sure if this makes sense, maybe I am missing some normalization or something else is wrong in the parameters to sw_detect. Please, let me know if you need more details, and thanks for this awesome library.

raphaelvallat commented 1 year ago

Hi @acorbellini,

Thanks for describing this interesting corner case. How long was the recording that you used? I think that your proposal is valid, so please feel free to submit a PR. I would use an even smaller threshold, e.g. 1e-12 just to make sure that we're not accidentally changing the morphology of the slow-waves.

raphaelvallat commented 1 year ago

Question: in your recordings, how many samples return True for data_filt < 1e-10, and are they all at the beginning of the recording?