Fix Issue 484 --- `pspm_sf`

[teddychao]

Fixes #484.

Introduction

Missing epochs often appear in the imported data. It is expected to let users know the imported data contain missing epochs since they impact the results of processing. Specifically for this issue, the source data contains missing epochs at the beginning. Since some processing requires the initial values for determine its parameters, such missing epochs finally lead to errors. At the same time, the missing epochs cannot be ignored in VBA processing, and therefore there are further issues inside VBA functions. This pull request aims at resolving such initial missing epochs that disallow following analysis.

Methods

• When there are missing epochs, let users know missing epochs exist and the results will be affected.
  • Users can adjust the field options.missingthreshold to skip trials that contain missing epochs more than the threshold time.
• Initial missing epochs should be filtered out when selecting initial values for determining some parameters.
• Interpolation will be performed before processing VBA analysis.

Results

• Overall

  • This pull request has been updated in line with the recent pull request #502 .

• src/pspm_dcm_inv.m

  • DCM analysis requires the first three values of the imported data for determining priors. Now it uses non-*NaN- values for determine such parameters (Line 556 --- Line 559).
  • .missing_data as a field of model has been described in the help text.
  • .missing is not used thus removed from help text.
  • .missing_data does not require a default value because there may be no missing values.

• src/pspm_options.m

  • Line 415: update the options for pspm_sf to allow users adjust the threshold.
    • The field will be transferred to sf_dcm when sf calls sf_dcm.

• src/pspm_sf.m

  • Line 28--33: help text describes model.missing is used for specifying the missing epochs.
  • Line 183--192: model.missing is used to load missing epochs.
  • Line 219--238: Loading missing epochs.
  • Line 288--289: use pspm_time2index to load missing data, from time ('seconds') into timepoints.

• src/pspm_sf_dcm.m

  • Line 115 --- Line 116
    • Now it uses only non-*NaN- values for determining X0 and nresp.
  • Line 154 --- Line 163
  • Now reminds users when there are missing epochs if it is the situation since the results may be dominated with NaN.
  • Now allows users to change the threshold for skipping trials that contain missing epochs if they wish.
  • Line 169
  • Now it uses pspm_interpolate to fill the NaNs before using VBA functions.

[teddychao]

Hi @dominikbach @uzaygokay many thanks for approving this. I will send this updated package to Carlos for him to test. I will merge it when he feels comfortable with the updated code. Thank you

[dominikbach]

I realise now there is a more fundamental problem.

pspm_sf takes one or multiple data files as data inputs, and an epoch definition file as event input. It will then pass the data to pspm_sf_dcm for each identified epoch.

This means that the missing epoch definition needs to be retrieved in pspm_sf (if method is "dcm"), and a missing index needs to be passed on to pspm_sf_dcm.

[teddychao]

I realise now there is a more fundamental problem.

pspm_sf takes one or multiple data files as data inputs, and an epoch definition file as event input. It will then pass the data to pspm_sf_dcm for each identified epoch.

This means that the missing epoch definition needs to be retrieved in pspm_sf (if method is "dcm"), and a missing index needs to be passed on to pspm_sf_dcm.

This is indeed different from the design in pspm_dcm and pspm_dcm_inv... Can we open another pull request for this problem after agreeing the changes in this pull request that introduces SF's usage of missing epochs?

[teddychao]

I realise now there is a more fundamental problem.

pspm_sf takes one or multiple data files as data inputs, and an epoch definition file as event input. It will then pass the data to pspm_sf_dcm for each identified epoch.

This means that the missing epoch definition needs to be retrieved in pspm_sf (if method is "dcm"), and a missing index needs to be passed on to pspm_sf_dcm.

I think this issue has been resolved through the recent pull request #502 . I have merged #502 into this pull request.