🕵️ Determine cause of fewer timepoints than regressors

[shnizzedy]

2. For diff-proj-SB151-lessmotion, I'm not sure why so much is getting censored. The problem here is that so much gets censored that there are fewer timepoints left than regressors. I'm running this one by the rest of the team to see if they have any ideas.

[shnizzedy]

Here's what Steve had to say after looking at FD.1D:

we used to write out the frames excluded/included and unfortunately that code was removed at some point

would like to see which of these FD numbers were flagged for censoring

obviously more than 25 are lower than 0.5

I would check working dir; see if the workdir node for nuisance/censoring has a param for which frames are flagged for censoring

[shnizzedy]

Can you give me the whole nuisance_regressor subdirectory of the working directory for one or both of these subjects?

diff-proj-SB151-lessmotion

[shnizzedy]

From nuisance_regression_before-filt_2_0/_scan_rest/find_offending_time_points/_report/report.rst:

Original Inputs
---------------

* dvars_file_path : <undefined>
* dvars_threshold : <undefined>
* fd_j_file_path : <undefined>
* fd_j_threshold : <undefined>
* fd_p_file_path : /out/working/resting_preproc_sub-151_sub-151/gen_motion_stats_afni_mean_3dvolreg_0/_scan_rest/calculate_FD/FD.1D
* fd_p_threshold : 0.3
* function_str : CPAC.nuisance.utils.find_offending_time_points
* number_of_previous_trs_to_censor : 1
* number_of_subsequent_trs_to_censor : 2

Execution Inputs
----------------

* dvars_file_path : <undefined>
* dvars_threshold : <undefined>
* fd_j_file_path : <undefined>
* fd_j_threshold : <undefined>
* fd_p_file_path : /out/working/resting_preproc_sub-151_sub-151/gen_motion_stats_afni_mean_3dvolreg_0/_scan_rest/calculate_FD/FD.1D
* fd_p_threshold : 0.3
* function_str : CPAC.nuisance.utils.find_offending_time_points
* number_of_previous_trs_to_censor : 1
* number_of_subsequent_trs_to_censor : 2

---

That's exactly what censored. Here's what passed through ('censor' == 1) and what remains when filtering by that threshold combined with the numbers of previous and subsequent TRs to censor:

df.loc[df[
    (df['FD'] < 0.3) 
    & (df['FD'].shift(1) < 0.3) 
    & (df['FD'].shift(-1) < 0.3) 
    & (df['FD'].shift(-2) < 0.3)
].index + 1][['censor', 'FD']]

	censor	FD
6	1	0.233122
20	1	0.080875
21	1	0.123705
22	1	0.164094
23	1	0.272617
24	1	0.225421
25	1	0.206429
35	1	0.083271
36	1	0.144700
37	1	0.132204
38	1	0.083319
39	1	0.055851
40	1	0.152085
41	1	0.224554
42	1	0.284892
43	1	0.200419
59	1	0.206083
111	1	0.175182
112	1	0.240602
113	1	0.222656
114	1	0.213301
115	1	0.271763
116	1	0.176624
117	1	0.071998
118	1	0.143761