LBHB / NEMS0

THIS VERSION OF NEMS IS NO LONGER SUPPORTED. PLEASE USE THE NEW NEMS REPOSITORY OR INSTALL NEMS_DB TO GET NEMS0 SUPPORT.
GNU General Public License v3.0
8 stars 4 forks source link

Migrate to masking approach for selecting subsets of the signal #113

Open bburan opened 6 years ago

bburan commented 6 years ago

We have explored multiple approaches for selecting subsets of a signal including RasterizedSignalSubset. After a conversation with @svdavid where he filled me in on the masking approach that @svdavid and @crheller set up, I think we should migrate towards the masking approach for everything. To summarize what we discussed in pseudocode.

recording = load_recording(batch, cell)
est, val = function_to_select_my_est_val_data(recording)
e1, e2, e3, e4, e5 = jackknife(est, n=5)

Here, the _data attribute on the signals in e1, e2, e3, e4, e5, est and val are basically references to the _data attribute on the signals in recording (i.e., they all share the same data array and it will be set to immutable to prevent accidental manipulation). However, each recording will have an independent mask that indicates the portion of the recording that can be used for analysis. Recordings/Signals will support the following primitive methods. Methods that accept inplace as a parameter will return a copy when the argument is False or modify the existing object if True:

as_continuous(mask=True)

By default, return a copy of data containing only the masked portions. Set mask=False to get the full dataset.

set_mask(mask, inplace=True)

Set the mask on the signal.

clear_mask(inplace=True)

Remove the mask from the signal

or_mask(mask, inplace=True)
and_mask(mask, inplace=True)
xor_mask(mask, inplace=True)

Perform the appropriate boolean operation between the current mask and the new mask, and set that as the mask for the object.

extract_epoch(mask=True)

As before, but returns only epochs that are not masked. Need to clarify behavior for partially masked epochs, but this should be user-configurable (e.g., add a parameter partial to indicate whether partial epochs are OK or not).

All other methods will then use these methods to manipulate the mask. For example, select_epochs(mask_mode='and') should create a mask and then use and_mask to modify the mask, select_epochs(mask_mode='replace') would replace the mask entirely (e.g., ignoring any mask created by an est/val split).

svdavid commented 6 years ago

Some recent conversations with @crheller about using masks for epoch selection without actually applying the mask. I think this takes care of a lot of this issues raised here. Basic sequence of operations is to define a mask and then pass that mask to a select_epochs call.

What could still use a bit of work is operations on the mask (like clearing).