New user with many errors when running fMRI-prepped BOLD images through standard fc pipeline

[bwinsto2]

Thank you for making this pipeline! We are excited to try it out in our lab. We have already run fMRIprep on multi-session resting state data, and we are attempting to run (or at least test) one of the RSFC pipelines. We tried following the instructions to the letter, but the pipeline is failing very early and often; there must be a problem with how we're specifying the location of our files or the design file(?) Per the instructions, we are trying to run a functional pipeline without the structural one first.

Cohort file

id0,id1,img
sub-abc123,ses-Baseline,sub-abc123/ses-Baseline/func/sub-abc123_ses-Baseline_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz
sub-abc123,ses-Session1,sub-abc123/ses-Session1/func/sub-abc123_ses-Session1_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz
sub-abc123,ses-Session4,sub-abc123/ses-Session4/func/sub-abc123_ses-Session4_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz
sub-abc123,ses-Session5,sub-abc123/ses-Session5/func/sub-abc123_ses-Session5_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz

Design File To test things out, we're simply using an unaltered version of the following (from cloned Github repo):

fc-36p_despike.dsn

Error message I'm not sure exactly what to post here--there are hundreds of errors. Here are a few snippets (clearly the pipeline isn't really running)

Constructing a pipeline based on user specifications
····································································
· [D][/design/design.dsn]
INFO: /root/matlab/startup.m does not exist ... creating
sed: cannot rename /design/sedHvoTT8: Device or resource busy
· [I][/scratch]
· [C][/cohort/cohort.csv]
Error in file(file, ifelse(append, "a", "w")) :
  cannot open the connection
Calls: write.table -> file
In addition: Warning message:
In file(file, ifelse(append, "a", "w")) :
  cannot open file '/cohort/cohort.csv': Read-only file system
Execution halted
· [O][/xcpOutput]
· [M][Executing serially]

Cohort file issue is the same as #268 Later, it's not finding the design file?

###################################################################
#  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  #
#                                                               #
#  ⊗                XCP ENGINE LOCALISER                    ⊗  #
#                                                               #
#  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  #
###################################################################
/xcpEngine/core/parseArgsLocaliser: line 113: /data/jux/BBL/projects/testACT/script/designfiles/fc-36p_despike.dsn: No such file or directory

Current processing step:
Localising subject metadata...
····································································
· [Parsing subject identifiers]
· [Creating output directory]
· [/xcpOutput/sub-abc123/ses-Baseline]
· [Initialising space mapping]
· [Initialising derivative maps]
· [Initialising atlas system]
· Ingress
· [Native space: sub-abc123_ses-Baseline_native]
· [Anatomical space: sub-abc123_ses-Baseline_anatomical]
fixed '' is not an image
· [Priming analytic audit]
· [Priming quality assessment]
····································································
Processing step complete:
Localising subject metadata...

Current processing step:
Localising image
····································································
· [Verifying that images are present]
· [Conforming and localising images]
····································································
Processing step complete:
Localising image

Current processing step:
Localising design
····································································
· [Writing localised design]
/xcpEngine/core/functions/printx: line 2: /data/jux/BBL/projects/testACT/script/designfiles/fc-36p_despike.dsn: No such file or directory
· [Evaluating subject variables]
cp: missing destination file operand after '/xcpOutput/sub-abc123/ses-Baseline/template.nii.gz'
Try 'cp --help' for more information.

later,

###################################################################
#  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  #
#                                                               #
#  ⊗                FMRI PREPROCESSING MODULE               ⊗  #
#                                                               #
#  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  ⊗  #
###################################################################

[I][/xcpOutput/sub-abc123/ses-Baseline/sub-abc123_ses-Baseline.nii.gz]
[O][/xcpOutput/sub-abc123/ses-Baseline/prestats]
/xcpEngine/core/functions/rerun: line 3: ((: == 1 : syntax error: operand expected (error token is "== 1 ")

Current processing step:
Ensure matching orientation
····································································
· Input: /scratch/sub-abc123_ses-Baseline-prestats-858165278~TEMP~.nii.gz
· Template:
· Output root:
· checking the orientation of img and template
NATIVE:LPI TEMPLATE:
· img and template orientation are not the same
· make it: LPI ->
/xcpEngine/core/functions/cleanup: line 3: ((: == 1 : syntax error: operand expected (error token is "== 1 ")

Module complete

later,

###################################################################
#  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  #
#                                                               #
#  ·                CONFOUND MODEL MODULE                   ·  #
#                                                               #
#  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  ·  #
###################################################################

[I][/xcpOutput/sub-abc123/ses-Baseline/sub-abc123_ses-Baseline.nii.gz]
[O][/xcpOutput/sub-abc123/ses-Baseline/confound2]
/xcpEngine/core/functions/rerun: line 3: ((: == 1 : syntax error: operand expected (error token is "== 1 ")

Current processing step:
Generating confound matrix
····································································
· Generating temporal mask
Error in getopt_options(object, args) :
  Error in getopt(spec = spec, opt = args) : flag "i" requires an argument
Calls: parse_args -> parse_options -> getopt_options
Execution halted
Error in getopt_options(object, args) :
  Error in getopt(spec = spec, opt = args) : flag "i" requires an argument
Calls: parse_args -> parse_options -> getopt_options
Execution halted
Error in getopt_options(object, args) :
  Error in getopt(spec = spec, opt = args) : flag "i" requires an argument
Calls: parse_args -> parse_options -> getopt_options
Execution halted
· [Computing framewise displacement]
Error in file(file, "rt") : cannot open the connection
Calls: unname -> as.matrix -> read.table -> file
In addition: Warning message:
In file(file, "rt") :
  cannot open file '/xcpOutput/sub-abc123/ses-Baseline/confound2/mc/sub-abc123_ses-Baseline_realignment.1D': No such file or directory
Execution halted
· [Computing DVARS]
· [Selecting quality criteria]
· [Applying framewise threshold to time series]
Error in getopt_options(object, args) :
  Error in getopt(spec = spec, opt = args) : flag "m" requires an argument
Calls: parse_args -> parse_options -> getopt_options
Execution halted
· [Evaluating data quality]
/xcpEngine/core/functions/temporal_mask: line 197: /xcpOutput/sub-abc123/ses-Baseline/confound2/mc/sub-abc123_ses-Baseline_tmask.1D: No such file or directory
· relative maximum motion
Error in file(file, "rt") : cannot open the connection
Calls: read.table -> file
In addition: Warning message:
In file(file, "rt") :
  cannot open file '/xcpOutput/sub-abc123/ses-Baseline/confound2/mc/sub-abc123_ses-Baseline_relRMS.1D': No such file or directory
Execution halted
· relative mean motion
Error in file(file, "rt") : cannot open the connection
Calls: read.table -> file
In addition: Warning message:
In file(file, "rt") :
  cannot open file '/xcpOutput/sub-abc123/ses-Baseline/confound2/mc/sub-abc123_ses-Baseline_relRMS.1D': No such file or directory
Execution halted
The nuisance matrix is being selected base on design files
/xcpEngine/modules/confound2/confound2.mod: line 155: ((: == 1 : syntax error: operand expected (error token is "== 1 ")
/xcpEngine/modules/confound2/confound2.mod: line 172: ((: == 1 : syntax error: operand expected (error token is "== 1 ")
/xcpEngine/modules/confound2/confound2.mod: line 190: ((: == 1 : syntax error: operand expected (error token is "== 1 ")
/xcpEngine/modules/confound2/confound2.mod: line 215: ((: == 1 : syntax error: operand expected (error token is "== 1 ")
/xcpEngine/modules/confound2/confound2.mod: line 235: ((: == 1 : syntax error: operand expected (error token is "== 1 ")
/xcpEngine/modules/confound2/confound2.mod: line 271: ((: != 0 : syntax error: operand expected (error token is "!= 0 ")
/xcpEngine/modules/confound2/confound2.mod: line 300: ((: > 0 : syntax error: operand expected (error token is "> 0 ")
/xcpEngine/modules/confound2/confound2.mod: line 326: ((: > 1 : syntax error: operand expected (error token is "> 1 ")
/xcpEngine/modules/confound2/confound2.mod: line 346: ((: == 1 : syntax error: operand expected (error token is "== 1 ")
/xcpEngine/modules/confound2/confound2.mod: line 381: ((: == 1 : syntax error: operand expected (error token is "== 1 ")
/xcpEngine/modules/confound2/confound2.mod: line 403: ((: == 1 : syntax error: operand expected (error token is "== 1 ")
····································································

Runtime Information

xcpengine-docker \
  -c /data2/1305/ignore/xcp/1305_cohort.csv \
  -d /data2/Github/xcpEngine/designs/fc-36p_despike.dsn \
  -i /data2/1305/derivatives/xcp/tmp_abc \
  -o /data2/1305/derivatives/xcp \
  -r /data2/1305/derivatives/fmriprep

Additional context Is it the case that it is not locating the image files correctly? Is it b/c the cohort file is read-only? Might I have to edit the design file? Apologies for being a bit lost with this.

Many thanks, Brian

[a3sha2]

there is problem with acess to image in the directory and all your command is good

[bwinsto2]

Thanks for your answer. Do you think it is a permissions issue? I just tried running again and double-checked. Everything is open for read, write, and execute access. Or by access do you mean it's not finding the right images? All that's needed is the BOLD file, right?

[daphneneuro17]

I am experiencing virtually the same issue also running in Docker. I fixed the "read only" error for the cohort file by running Docker directly; however, this has not fixed the fact that the pipeline is not receiving my imaging files.

I tried running the cbf_all pipeline as well with similar results.

[a3sha2]

@bwinsto2 pls can you try singularity image or docker image pls. Sorry for late reply

@daphneneuro17 that should be a permission issue at your end

[bwinsto2]

@a3sha2 thanks for the reply, but I'm not sure what else to try. I downloaded the newest docker image and ran with the same command. I've tried with other datasets, but I'm receiving the same errors. I'm not sure how this could be a permissions issue because I've made sure the design file, cohort file, and all data files are open for read, write, and execute access for all users.

It sounds like @daphneneuro17 is experiencing the same problem.

[bwinsto2]

It looks like this is the same (unresolved?) problem in #199 and #204

I thought this might be associated with #268, but here is my command for running docker directly, and the error I get:

Here is the analysis variables section of the design file:

I have tried keeping the design analysis variable in the design file as the default from the repository. I have also tried changing the variable to the place on my disk where I keep the file (referring to itself). I get the same error.

@a3sha2 or @mattcieslak any ideas? @a3sha2 you have noted previously that the issue is with "docker binding": is there anything more specific you could suggest to resolve this?

Thank you.

[a3sha2]

@bwinsto2 the problem is directory of the design file

[a3sha2]

@bwinsto2 the main issue is design file, you need to give design file permission after you save it in another directory. For xcpengine to work, the design and cohort file must be read and write. try chmod +rwx on both design and cohort file. the issues #199 and #204 have resolved

[mattcieslak]

It looks like you have the order of your arguments reversed for the -v flags. If your cohort file located on your machine at /data2/1305/ignore/xcp/1305_cohort.csv then your command should have -v /data2:/data2

You can check by doing something like:

$ docker run --rm -it --entrypoint bash -v /data2:/data2 -v /tmp:/tmp
$ find /data2/1305

and this should show your files. Let us know if this works!

[bwinsto2]

@mattcieslak thanks for looking into this! I suppose I misunderstood docker binding in this scenario. I did what you suggested, and the pipeline is running! Funny that adding a single character ("2") made all the difference.

Appreciate your help @a3sha2 and @mattcieslak

[bwinsto2]

Suppose it's worth mentioning that my issue using the wrapper is perhaps unrelated(?).

I don't know if @daphneneuro17 has gotten it working by interacting with Docker directly, but it sounds like she had the same issue running w/ the wrapper.