Save the preprocessing pipelines for simple reuse in curation GUI

[yger]

Since we are now benchmarking drift registration, and that results starts to work, I was wondering if we could start thinking about a way to properly save a "preprocessing" pipeline, such that it could be reloaded and applied on the fly to the data. This issue came up after thinking about the manual curation step in a world where data would have been distorted via motion correction. Ideally, one would like to be able to view, dynamically, the data before and after motion correction. So if I have a pre-processing chain, let's say filtering -> destriping -> CAR -> z_score -> motion correction, could it be possible to save it (alongside with all the input argument) such that, in the curation step, one could load the recording, create again this lazy pre-processing chains and apply it (or not) on the fly? We want to avoid data duplication, but also to appreciate the effect of the motion correction, and spot possible problems, it would be good to be able to turn it on or off in a GUI or in ipywidgets. Of course, when you know the chain, you can create several intermediate recordings and visualize each of them individually, but for complex chains this can start to be tedious.

[samuelgarcia]

Yep. rec.dump_to_json() make a dict saved in a JSON that describe the chain with all kwargs and can be loaded back with load_extractor()

[alejoe91]

With motion signals???

[yger]

Ok, good to know, if all the arguments/inputs are also saved, then perfect. As said, I want the dump to be self-contained, so as @alejoe91 is saying with motion signals. Ideally, one would like to be able to check the traces interactively at all steps of the pre-processing pipeline. I'll have a look