Flags reverting to default settings

[Larmarbar]

I'm running qsiprep with --recon-only using docker. I noticed that when the docker command gets executed, both the -w as well as the --bids-filter-file sometimes revert to the default value despite other (valid) directories/files being specified in the input. I didn't realize for quite some time as it appears to do this swap completely silently as far as I can tell. While the first line of console output still correctly reflects the input, the actual docker command does not.

Command I'm running:

qsiprep-docker /data/ICE_analysis/raw /data/ICE_analysis/derivatives participant -w /home/lbartels/scratch/ICE_Work_Dir --participant_label M07 --bids-filter-file /home/lbartels/Documents/ICE_Analysis/ice_analysis/bids_filter/filter_ses-2122pre.json --recon-only --recon-spec amico_noddi --recon-input /data/ICE_analysis/derivatives/qsiprep --freesurfer-input /data/ICE_analysis/derivatives/fmriprep/sourcedata/freesurfer --nthreads 1 --skip-odf-reports --fs-license-file /home/lbartels/Documents/ICE_Analysis/ice_analysis/license.txt -v

Console output:

['/home/lbartels/.local/bin/qsiprep-docker', '/data/ICE_analysis/raw', '/data/ICE_analysis/derivatives', 'participant', '-w', '/home/lbartels/scratch/ICE_Work_Dir', '--participant_label', 'M07', '--bids-filter-file', '/home/lbartels/Documents/ICE_Analysis/ice_analysis/bids_filter/filter_ses-2122pre.json', '--recon-only', '--recon-spec', 'amico_noddi', '--recon-input', '/data/ICE_analysis/derivatives/qsiprep', '--freesurfer-input', '/data/ICE_analysis/derivatives/fmriprep/sourcedata/freesurfer', '--nthreads', '1', '--skip-odf-reports', '--fs-license-file', '/home/lbartels/Documents/ICE_Analysis/ice_analysis/license.txt', '-v']

RUNNING: docker run --rm -it -v /home/lbartels/Documents/ICE_Analysis/ice_analysis/license.txt:/opt/freesurfer/license.txt:ro -v /data/ICE_analysis/raw:/data:ro -v /data/ICE_analysis/derivatives/qsiprep:/qsiprep-output:ro -v /data/ICE_analysis/derivatives/fmriprep/sourcedata/freesurfer:/sngl/freesurfer-input:ro -v /home/lbartels/Documents/ICE_Analysis/ice_analysis/bids_filter/filter_ses-2122pre.json:/sngl/filter/filter.json:ro -v /data/ICE_analysis/derivatives:/out -v /home/lbartels/scratch/ICE_Work_Dir:/scratch pennbbl/qsiprep:0.16.1 /data /out participant --recon-input /qsiprep-output --freesurfer-input /sngl/freesurfer-input --recon-spec amico_noddi --bids-filter-file /sngl/filter/filter.json --participant_label M07 --recon-only --nthreads 1 --skip-odf-reports -v -w /scratch

The -w issue can be resolved by using the --work_dir flag instead

Input command: qsiprep-docker /data/ICE_analysis/raw /data/ICE_analysis/derivatives participant **--work_dir /home/lbartels/scratch/ICE_Work_Dir** --participant_label M07 --bids-filter-file /home/lbartels/Documents/ICE_Analysis/ice_analysis/bids_filter/filter_ses-2122pre.json --recon-only --recon-spec amico_noddi --recon-input /data/ICE_analysis/derivatives/qsiprep --freesurfer-input /data/ICE_analysis/derivatives/fmriprep/sourcedata/freesurfer --nthreads 1 --skip-odf-reports --fs-license-file /home/lbartels/Documents/ICE_Analysis/ice_analysis/license.txt -v

['/home/lbartels/.local/bin/qsiprep-docker', '/data/ICE_analysis/raw', '/data/ICE_analysis/derivatives', 'participant', '--work_dir', '/home/lbartels/scratch/ICE_Work_Dir', '--participant_label', 'M07', '--bids-filter-file', '/home/lbartels/Documents/ICE_Analysis/ice_analysis/bids_filter/filter_ses-2122pre.json', '--recon-only', '--recon-spec', 'amico_noddi', '--recon-input', '/data/ICE_analysis/derivatives/qsiprep', '--freesurfer-input', '/data/ICE_analysis/derivatives/fmriprep/sourcedata/freesurfer', '--nthreads', '1', '--skip-odf-reports', '--fs-license-file', '/home/lbartels/Documents/ICE_Analysis/ice_analysis/license.txt', '-v']

RUNNING: docker run --rm -it -v /home/lbartels/Documents/ICE_Analysis/ice_analysis/license.txt:/opt/freesurfer/license.txt:ro -v /data/ICE_analysis/raw:/data:ro -v /data/ICE_analysis/derivatives/qsiprep:/qsiprep-output:ro -v /data/ICE_analysis/derivatives/fmriprep/sourcedata/freesurfer:/sngl/freesurfer-input:ro -v /home/lbartels/Documents/ICE_Analysis/ice_analysis/bids_filter/filter_ses-2122pre.json:/sngl/filter/filter.json:ro -v /data/ICE_analysis/derivatives:/out pennbbl/qsiprep:0.16.1 /data /out participant --recon-input /qsiprep-output --freesurfer-input /sngl/freesurfer-input --recon-spec amico_noddi --bids-filter-file /sngl/filter/filter.json --work_dir /home/lbartels/scratch/ICE_Work_Dir --participant_label M07 --recon-only --nthreads 1 --skip-odf-reports -v

However, I have no idea how to resolve the issue of the --bids-filter-file issue, any help or pointers would be greatly appreciated!

Edit: This issue appears to not be present when I am not using the --recon-only flag on at least one subject, can the --bids-filter-file flag simply not be used with recon only?

[smeisler]

Hi @Larmarbar,

qsiprep-docker is a wrapper around docker run. So, in your qsiprep-docker command, if you specify a work directory with -w, it is renamed to /scratch in the actual qsiprep command via -v /home/lbartels/scratch/ICE_Work_Dir:/scratch. In terms of operations, the two methods (renaming the drive while mounting it vs. just using the full path) should be identical and both are valid.

In short, the renaming of your scratch and bids-filter-file are expected, as that is what the qsiprep-docker does when preparing the actual qsiprep command. Does this answer your question? Besides that, are you getting any errors?

Best, Steven

[Larmarbar]

Thanks for pointing that out, that makes a lot of sense! It appears my issue is that when running with --recon-only, the --bids-filter-file seems to be ignored? Is this by design? This is why I thought it might just get reverted back to the default setting.

[smeisler]

Hi @Larmarbar, what makes you think it is being ignored? Both of the docker run commands in your original post are using your filter file, mapped in the container to /sngl/filter/filter.json. Are your outputs not looking as expected?

[Larmarbar]

It appeared to still run reconstruction on both sessions for a given subject rather than just the one specified in the filter file when using --recon-only. When I am running qsiprep without the --recon-only flag on the same data, it works as expected and only uses the specified session. I'm not sure, but think this might be an issue specific to trying to disentangle multiple sessions for a given subject during recon. I am running into issues with this even when not using filter files, but this is a separate problem. I have now found a work-around where I just specify a distinct output directory specific to each session and then run qsiprep incl. recon using a bids filter and that appears to give me the results I'm looking for and avoids the issue I was experiencing with the bids filter file.

[mattcieslak]

You're right @Larmarbar, the BIDS filter file doesn't work for recon-only. The reason was that there is no official BIDS spec for dMRI derivatives. However, that's no reason in practice to skip the filter file for reconstruction. I'd consider this a bug

[Larmarbar]

That's good to know! I think noting this in the documentation for the bids-filter flag would already be very helpful!