Make it easier to clean the data.

[gmierz]

This is now resolved. We can do the following:

from pupillib.core.workers.processors.trial_processor import (
    filter_rm_trials_with_vals_gt,
    filter_rm_trials_with_vals_lt
)
from pupillib.pupil_lib import script_run

# Get data from script_run
plib_runner = script_run(yaml_path='path/to/yaml', cache='path/to/cache')

datastore = plib_runner.data_store
datastore.process_trials(filter_rm_trials_with_vals_lt, ltval=0.5)

# Trials with values less than 0.5 are now rejected

All processor functions accept the trial data as input (first argument), and all args passed to process trials will be passed to it. See the following for an example processor:

def filter_trials_with_mean_lt(trial_data, meanval=None):
    if not meanval:
        return False
   elif mean(trial_data) < meanval:
        return True
   return False

Or, to modify data:

import numpy as np

def subtract_x_from_trials(trial_data, x=None):
    if not x:
        return trial_data
    else
        return list(np.asarray(trial_data) - x)

datastore.process_trials(subtract_x_from_trials, x=5)
datastore.datasets['dataset1].process_trials(subtract_x_from_trials, x=4)