Derivatives in Convolve and Lag Transform with error

[sjshim]

Traceback (most recent call last):
  File "/home/users/sjshim/miniconda3/envs/fitlins/lib/python3.9/site-packages/nipype/interfaces/base/core.py", line 398, in run
    runtime = self._run_interface(runtime)
  File "/oak/stanford/groups/russpold/data/network_grant/fitlins/fitlins/interfaces/bids.py", line 250, in _run_interface
    self._results['all_specs'] = self._load_graph(runtime, graph)
  File "/oak/stanford/groups/russpold/data/network_grant/fitlins/fitlins/interfaces/bids.py", line 258, in _load_graph
    specs = node.run(inputs, group_by=node.group_by, **filters)
  File "/home/users/sjshim/miniconda3/envs/fitlins/lib/python3.9/site-packages/bids/modeling/statsmodels.py", line 461, in run
    node_output = BIDSStatsModelsNodeOutput(
  File "/home/users/sjshim/miniconda3/envs/fitlins/lib/python3.9/site-packages/bids/modeling/statsmodels.py", line 569, in __init__
    dfs = self._collections_to_dfs(collections)
  File "/home/users/sjshim/miniconda3/envs/fitlins/lib/python3.9/site-packages/bids/modeling/statsmodels.py", line 663, in _collections_to_dfs
    coll = coll.to_df(sampling_rate=self.sampling_rate)
  File "/home/users/sjshim/miniconda3/envs/fitlins/lib/python3.9/site-packages/bids/variables/collections.py", line 617, in to_df
    collection = self.resample(
  File "/home/users/sjshim/miniconda3/envs/fitlins/lib/python3.9/site-packages/bids/variables/collections.py", line 523, in resample
    return self._densify_and_resample(
  File "/home/users/sjshim/miniconda3/envs/fitlins/lib/python3.9/site-packages/bids/variables/collections.py", line 435, in _densify_and_resample
    _variables[v.name] = v.resample(sr_arg, kind=kind)
  File "/home/users/sjshim/miniconda3/envs/fitlins/lib/python3.9/site-packages/bids/variables/variables.py", line 537, in resample
    var.resample(sampling_rate, True, kind)
  File "/home/users/sjshim/miniconda3/envs/fitlins/lib/python3.9/site-packages/bids/variables/variables.py", line 566, in resample
    f = interp1d(x, y, kind=kind)
  File "/home/users/sjshim/miniconda3/envs/fitlins/lib/python3.9/site-packages/scipy/interpolate/interpolate.py", line 436, in __init__
    _Interpolator1D.__init__(self, x, y, axis=axis)
  File "/home/users/sjshim/miniconda3/envs/fitlins/lib/python3.9/site-packages/scipy/interpolate/polyint.py", line 54, in __init__
    self._set_yi(yi, xi=xi, axis=axis)
  File "/home/users/sjshim/miniconda3/envs/fitlins/lib/python3.9/site-packages/scipy/interpolate/polyint.py", line 124, in _set_yi
    raise ValueError("x and y arrays must be equal in length along "
ValueError: x and y arrays must be equal in length along interpolation axis.

I get this error when trying to call derivatives for the Convolve transform as well as for Lag.

[adelavega]

Can you paste your model please?

[sjshim]

{ "Name": "cuedTS", "Description": "network discovery cuedTS analysis model", "BIDSModelVersion": "1.0.0", "Input": { "task": "cuedTS", "echo": null }, "Nodes": [ { "Level": "run", "Name": "run_cuedTS", "GroupBy": ["run", "session", "subject"], "Transformations": { "Transformer": "pybids-transforms-v1", "Instructions": [ { "Name": "Factor", "Input": ["trial_type_junk"] }, { "Name": "Scale", "Input": ["response_time"], "Demean": true, "Rescale": false, "Output": ["demeaned_response_time"] }, { "Name": "Rename", "Input": ["trial_type_junk.cstay", "trial_type_junk.cswitch_tstay", "trial_type_junk.cswitch_tswitch"], "Output": ["cstay", "cswitch_tstay", "cswitch_tswitch"] }, { "Name": "Convolve", "Model": "spm", "Derivative": true, "Input": ["cstay", "cswitch_tstay", "cswitch_tswitch", "demeaned_response_time"] } ] }, "Model": { "Type": "glm", "X": ["cstay", "cswitch_tstay", "cswitch_tswitch", "demeaned_response_time", "framewise_displacement", "rot_x", "rot_x_derivative1", "rot_y", "rot_y_derivative1", "rot_z", "rot_z_derivative1", "trans_x", "trans_x_derivative1", "trans_y", "trans_y_derivative1", "trans_z", "trans_z_derivative1", "non_steady_state_outlier00", "non_steady_state_outlier01", "non_steady_state_outlier02", "non_steady_state_outlier03", "non_steady_state_outlier04", "non_steady_state_outlier05", "non_steady_state_outlier06", "non_steady_state_outlier07", "non_steady_state_outlier08", "non_steady_state_outlier09", "a_comp_cor_00", "a_comp_cor_01", "a_comp_cor_02", "a_comp_cor_03", "a_comp_cor_04", 1 ] }, "Contrasts":[ { "Name": "task", "ConditionList": ["cstay", "cswitch_tstay", "cswitch_tswitch"], "Weights": [0.333,0.333,0.333], "Test": "t" }, { "Name": "cue_switch_cost", "ConditionList": ["cswitch_tstay", "cstay"], "Weights": [1, -1], "Test": "t" }, { "Name": "task_switch_cost", "ConditionList": ["cswitch_tswitch", "cswitch_tstay"], "Weights": [1, -1], "Test": "t" } ], "DummyContrasts": { "Conditions": [ "cstay", "cswitch_tstay", "cswitch_tswitch", "demeaned_response_time" ], "Test": "t" } }, { "Level": "subject", "Name": "subject_cuedTS", "GroupBy": ["subject", "contrast"], "Model": { "Type": "meta", "X": [1] }, "DummyContrasts": { "Test": "t" } } ], "Edges": [ { "Source": "run_cuedTS", "Destination": "subject_cuedTS" } ] }

[adelavega]

On the surface, your model looks good. Unfortunately, it's hard to debug this without your event files. If you wouldn't mind sharing an event file that would help. You could also try removing Scale or changing some of the Convolve parameters to see if the interaction of those two is the problem.

I've seen this problem before, and usually its due to the implicit densification of the variable that happens prior to Convolve, and some numeric imprecision.

Related issue: https://github.com/bids-standard/pybids/issues/361

[sjshim]

Thank you so much! Will take a look at the issues you suggested, Attaching the event file we have as an example. sub-s03_ses-02_task-cuedTS_events.tsv.zip

[sjshim]

It does seem like this might be an issue relating to the densification issue mentioned above, but I don't feel confident that I have a grasp on how to debug through this.

I have removed the Scale transform and when I use the following specification using Lag, I am able to get the results. However, I get the same error when using the Derivative: true option in Convolve instead.

I'm attaching both of my models.json files! Archive.zip