Retrospective visualization of all plots for debugging purposes (“tapas_physio_review”)

[mrikasper]

Purpose

In PhysIO, several levels of visual debugging are offered to check different preprocessing steps of the peripheral recordings, as well as model outputs. In an interactive run, these plots are generated successively as Matlab Figures where you can zoom in, inspect results etc.

However, a use case that becomes more and more relevant is running PhysIO remotely (e.g., on a high-performance cluster) for a multitude of subjects. To perform effective quality control, one has to inspect the output (debugging) figures retrospectively.

The goal of this issue is to provide a comprehensive tapas_physio_review function that takes the created output file (physio.mat) and re-generates the same figures that are now created in an interactive run of PhysIO, without having to recompute any of the data processing steps.

Methods

The main goal here is to separate data manipulation from visualization steps.

This will require identification of all local functions that plot figures in PhysIO, and also the required auxilary data to be saved for a retrospective re-creation of the plots

A rudimentary version of tapas_physio_review already exists, but it does not cover most of the visualizations of an interactive run.

[likeajumprope]

This issue is being dealt with in this branch

I have updated tapas_physio_review() with a function to retrospectively create Figure 2 (Preproc Peak detection)

[likeajumprope]

@mrikasper Assuming that I save, as discussed, the he variables that I need for plotting in review. That means that I need to carry them from the location where they are used for plotting while the toolbox is running, up to main_create_regressors, where the physio structure is assembled. How do I do that? Can I just create a subfield in verbose, for example verbose.review? I tried that but a lot of things are done to the verbose structure throughout the execution of the toolbox, and I fear that it might get overwritten.

[mrikasper]

It's not clear a priori what's the best place. Yes, verbose should be input and output to any function that includes a plot, so it seems a natural choice. And if it's handled according to specifications, only subfields verbose like verbose.fig_handles are changed within functions, so that shouldn't be a problem. So you could include a subfield verbose.review and then subfields within that, named after the function within which the plot is typically called, which contain the necessary variables for the plot, e.g. verbose.review.create_retroicor_regressors.R

This is clear and sustainable in terms of development, because the function definition is unaltered (input/output variables only contains verboseas before). However, it will lead to a lot of duplication of variables and therefore blow up the size of the physio.mat.

Alternatively, you have a .review.<function_name> subfield for each field, i.e., physio.sync, physio.preproc, physio.model and only populate the review parts with variables if they didn't already get saved in the parent field (e.g., physio.ons_secs.c exists, so don't duplicate it in physio.ons_secs.review.detect_cardiac_pulses.c). This will be much more memory-efficient, but requires some more interface changes of the function headers where the plotting during processing currently happens.

You can also mix the two variants by only putting variables in verbose.review.detect_cardiac_pulses.ons_secs that are not already stored in physio.ons_secs. This would avoid duplication and is still possible without changes to the function definition.

I have to sleep over it, right now I would prefer the last variant, but let me know your thoughts and questions.

[likeajumprope]

Hi Lars,

I have actually now added a physio.verbose.review matrix and it works for the peak detection function. I have also thought about it and I think this is the cleaner and safer option, nobody will ever temper with that field and it allows to store a snapshot of the variables as they are when printing.

[mrikasper]

Could you give some more details or link the code here what you mean by matrix? A structure variable?