Closed pmyers16 closed 3 years ago
Calling fit
would have the following flow:
X, y = _check_input_raw(X, None)
Check n_jobs
sig = X[idx, :]
hfo_statistic_arr = self._compute_hfo_statistic(sig)
hfo_detection_arr = self._threshold_statistic(hfo_statistic_arr)
hfos = self._post_process_ch_hfos(hfo_detection_arr)
Then merge channels into chs_hfos
This LGTM. ping me when you want me to look at PR.
Describe the problem
The current implementation has an internal state that stores a (n_channels x n_times x n_bands) matrix, which ends up eating up 30+GB of RAM if using linear band method and 20+ GB of RAM if using the log band method for a 30 minute clip sampled at 2000 Hz.
Describe your solution
Default to processing each channels data sequentially, parallelizing over bands, and merging in post-processing when array sizes are more manageable.
Describe possible alternatives
The alternative is to keep it as it is since it runs, just inefficiently