Apply stimulus shift and latency correction inside gridsearch, not in the preprocessing steps

[caiw]

The problem

On running gridsearch on visual functions on the Russian dataset, I found that it was working but the latency was obviously wrong:

On searching through the code, @neukym and I realised the mistake: the data preprocessing was applying latency correction for audio-delivered stimuli, but this wasn't actually being used, and the gridsearch was applying latency-stretch correction for audio-delivered stimuli; and neither were implemented for visual-delivered stimuli.

After discussion with @neukym, we decided that because the shift/stretch correction is (potentially) different for each stimulus-delivery modality, it makes sense to do this online, tailored to the specific function being run, during the gridsearch, instead of separately preprocessing the trialwise data for each modality.

This PR

• Removes all shift/stretch correction from the preprocessing and adds it into the gridsearch.
  • Fixes #286
• Adds a new CLI argument --input-stream [auditory|visual|tactile] to the gridsearch invoker, which applies the appropriate corrections for that modality
  • Genericises the correction away from audio_shift_correction alone
  • Adds other-modality fields to all the config files
  • Fixes #285
• Preprocessing doesn't any longer edit events and know about slices, instead it has a simple set of validation checks
  • Fixes #284
  • Second validation check not yet implemented, moved to new issue: #289
• Hopefully solves the original problem of gridsearch not working on visual functions
  • Fixes #287

For the reviewer

@ollieparish I'm asking you for review as you wrote most of this code! A few questions and notes we had while reading/writing the code.

• [x] Can you verify that we understand what the ave_mode argument is doing? We've written our understanding into comments which hopefully are accurate (please correct as necessary).
  • [x] In particular, please verify that things behave correctly when using ave_mode == "concatenate": we understand this to be concatenating repetitions into one long stimulus, and that it is to be only used in single_participant_override mode (to prevent concatenating multiple participants data into one). But in particular in this case would we also need to concatenate the function onto itself?
• [x] There are lots of * 2 and // 2 in do_gridsearch. Please could you verify that this is doing what it should and add a brief comment as to what they're doing?

@neukym - request for testing to confirm syntax breaks and results matches the previous results.

• [x] TVL Russian
• [x] TVL English
• [x] CIECAM02 Russian

Also covered in this PR (via @neukym)

• Added visual_delivery_latency, tactile_delivery_latency, visual_delivery_shift_correction and tactile_delivery_shift_correction fields to all the config files (covered by a different issue, #291)

[neukym]

last commit addressed most of #291

[neukym]

I've found a possible major problem with trying to fix things this way for the visual stream - I have found at least one example in dataset 4 where the visual frames pause for a whole two seconds, and then carry on. (participant 1, second presentation).

This means most of that repetition is out by a whole 2000ms - i.e. unusable if we use this method. The other three reps I looked at were very stable (exactly the same, continuous drift) - but I don't know how common this 'visual frames pause' issue is. Perhaps we should be checking for this and then rejecting any reps that have this issue?

(the whole reason that I put in triggers every second was to guard against this possibility – but I didn't check it really was necessary)

Depending how many 'pauses' are present, we may have to use the events as an override of some sort. New issue: #295

@ollieparish can you still review this PR, but don't merge.

[neukym]

@neukym confirmed it works for English TVL (I had to redo the trialwise data for it to work, found some minor bugs, now fixed.):

[neukym]

@neukym confirmed it works for Russian TVL:

[neukym]

@neukym confirmed it works for Russian CIECAM02, and fixes issue.