SpikeInterface / spikeinterface

A Python-based module for creating flexible and robust spike sorting pipelines.
https://spikeinterface.readthedocs.io
MIT License
517 stars 184 forks source link

Avoid random and computation in preprocessing init class #1118

Open samuelgarcia opened 1 year ago

samuelgarcia commented 1 year ago

Some classes for preprocessing like WhitenRecording,NormalizeByQuantileRecording,ZScoreRecordingare using internallyget_random_data_chunks()`.

This make the end user experience easier but this is rather bad for:

For parralel processing particularly it is really really bad because:

We have a way to store _kwargs but we should have another way (other dict) that would enable to restore the class very quickly in the same state without any random in between.

zm711 commented 3 weeks ago

How close are we to this? The issue is two years old and I doubt we really look at it. It is an ongoing effort, but I think we've made huge progress no?