search

nipy / nipype

Workflows and interfaces for neuroimaging packages
https://nipype.readthedocs.org/en/latest/
Other
749 stars 530 forks source link

MapNode for datasink mixed outputs with two subjects #2152

Closed anbai106 closed 7 years ago

anbai106 commented 7 years ago

Hi, nipype experts:

I have a question about datasink, I use MapNode to parallelize my pipeline, and use datasink to grab all the outputs for each subject like this:

# get the information for datasinker.
get_identifiers = pe.MapNode(niu.Function(
    input_names=['subject_id', 'caps_directory'], output_names=['base_directory', 'subst_tuple_list', 'regexp_substitutions'],
    function=get_subid_sesid_t1_to_b0), name='get_subid_sesid_t1_2_b0', iterfield=['subject_id'])
get_identifiers.inputs.caps_directory = caps_directory

datasink

datasink = pe.MapNode(nio.DataSink(), name='datasinker', iterfield=['base_directory', 'substitutions', 'regexp_substitutions'])
datasink.inputs.remove_dest_dir = True

I use substituions to change the name of the images and regexp_substitutions to delete the folders that I do not want to have, and the get_subid_sesid_t1_to_b0 function is defined by:

def get_subid_sesid_t1_to_b0(subject_id, caps_directory): """ This is to extract the base_directory for the DataSink including participant_id and sesion_id in CAPS directory, also the tuple_list for substitution :param subject_id: :return: base_directory for DataSink """ import os

participant_id = subject_id.split('_')[0]
session_id = subject_id.split('_')[1]
base_directory = os.path.join(caps_directory, 'subjects', participant_id, session_id, 'noddi', 'postprocessing')
subst_tuple_list = [
    (subject_id + '_brainExtractedT1w_flirt.mat', subject_id + '_t1-to-b0_withResampling.mat'),
    (subject_id + '_tissue-whitematter_binaryMask_flirt.nii.gz', subject_id + '_binarymask-whitematter_reslicedOnDiffusionSpace.nii.gz'),
    (subject_id + 'mrtrix_matrix.mat', subject_id + '_t1-to-b0_withoutResampling.mat'),
    (subject_id + 'desikan_deformed_image.nii.gz', subject_id + '_parcellation-desikan_onDiffusionSpace.nii.gz'),
    (subject_id + 'destrieux_deformed_image.nii.gz', subject_id + '_parcellation-destrieux_onDiffusionSpace.nii.gz'),
]
# to delete the folders that u do not want
regexp_substitutions = [
    # I don't know why it's adding this empty folder, so I remove it:
    # NOTE, . means any characters and * means any number of repetition in python regex
    (r'/_destrieux_in_diffusion_space./', r'/'),
    (r'/_desikan_in_diffusion_space./', r'/'),
    (r'/_apply_flirt_registration./', r'/'),
    (r'/_registration_t1_to_b0./', r'/'),
    (r'/_convert_flirt_to_mrtrix./', r'/'),
]

return base_directory, subst_tuple_list, regexp_substitutions

And then I connect the nodes, but the results in my resulting folder is mixed like this:

sub-PREVDEMALS0010001BE_ses-M0_brainExtractedT1w_flirt.mat sub-PREVDEMALS0010001BE_ses-M0_brainExtractedT1w_flirt.nii.gz sub-PREVDEMALS0010001BE_ses-M0_desikan_deformed_image.nii.gz sub-PREVDEMALS0010001BE_ses-M0_destrieux_deformed_image.nii.gz sub-PREVDEMALS0010001BE_ses-M0_mrtrix_matrix.mat sub-PREVDEMALS0010001BE_ses-M0_tissue-whitematter_binaryMask_flirt.nii.gz sub-PREVDEMALS0010003PB_ses-M0_binarymask-whitematter_reslicedOnDiffusionSpace.nii.gz sub-PREVDEMALS0010003PB_ses-M0_brainExtractedT1w_flirt.nii.gz sub-PREVDEMALS0010003PB_ses-M0_desikan_deformed_image.nii.gz sub-PREVDEMALS0010003PB_ses-M0_destrieux_deformed_image.nii.gz sub-PREVDEMALS0010003PB_ses-M0_mrtrix_matrix.mat sub-PREVDEMALS0010003PB_ses-M0_t1-to-b0_withResampling.mat

which **sub-PREVDEMALS0010001BE*** should not be in this folder....

any suggestion will be appreciated

Hao

anbai106 commented 7 years ago

This is the the report.rst file:

Node: io

Hierarchy : _datasinker0 Exec ID : _datasinker0

Original Inputs

  • _outputs : {'t1-register-b0.@out_desikan_in_diffusion_space': ['/Users/junhao.wen/test/NODDI/working_dir/noddi_postprocessing_pipeline/t1_b0_registration_pipeline/desikan_in_diffusion_space/mapflow/_desikan_in_diffusion_space0/sub-PREVDEMALS0010003PB_ses-M0_desikan_deformed_image.nii.gz', '/Users/junhao.wen/test/NODDI/working_dir/noddi_postprocessing_pipeline/t1_b0_registration_pipeline/desikan_in_diffusion_space/mapflow/_desikan_in_diffusion_space1/sub-PREVDEMALS0010001BE_ses-M0_desikan_deformed_image.nii.gz'], 't1-register-b0.@out_destrieux_in_diffusion_space': ['/Users/junhao.wen/test/NODDI/working_dir/noddi_postprocessing_pipeline/t1_b0_registration_pipeline/destrieux_in_diffusion_space/mapflow/_destrieux_in_diffusion_space0/sub-PREVDEMALS0010003PB_ses-M0_destrieux_deformed_image.nii.gz', '/Users/junhao.wen/test/NODDI/working_dir/noddi_postprocessing_pipeline/t1_b0_registration_pipeline/destrieux_in_diffusion_space/mapflow/_destrieux_in_diffusion_space1/sub-PREVDEMALS0010001BE_ses-M0_destrieux_deformed_image.nii.gz'], 't1-register-b0.@out_mrtrix_matrix': ['/Users/junhao.wen/test/NODDI/working_dir/noddi_postprocessing_pipeline/t1_b0_registration_pipeline/convert_flirt_to_mrtrix/mapflow/_convert_flirt_to_mrtrix0/sub-PREVDEMALS0010003PB_ses-M0_mrtrix_matrix.mat', '/Users/junhao.wen/test/NODDI/working_dir/noddi_postprocessing_pipeline/t1_b0_registration_pipeline/convert_flirt_to_mrtrix/mapflow/_convert_flirt_to_mrtrix1/sub-PREVDEMALS0010001BE_ses-M0_mrtrix_matrix.mat'], 't1-register-b0.@out_flirt_matrix': ['/Users/junhao.wen/test/NODDI/working_dir/noddi_postprocessing_pipeline/t1_b0_registration_pipeline/registration_t1_to_b0/mapflow/_registration_t1_to_b00/sub-PREVDEMALS0010003PB_ses-M0_brainExtractedT1w_flirt.mat', '/Users/junhao.wen/test/NODDI/working_dir/noddi_postprocessing_pipeline/t1_b0_registration_pipeline/registration_t1_to_b0/mapflow/_registration_t1_to_b01/sub-PREVDEMALS0010001BE_ses-M0_brainExtractedT1w_flirt.mat'], 't1-register-b0.@out_wm_mask_in_diffusion_mask': ['/Users/junhao.wen/test/NODDI/working_dir/noddi_postprocessing_pipeline/t1_b0_registration_pipeline/apply_flirt_registration/mapflow/_apply_flirt_registration0/sub-PREVDEMALS0010003PB_ses-M0_tissue-whitematter_binaryMask_flirt.nii.gz', '/Users/junhao.wen/test/NODDI/working_dir/noddi_postprocessing_pipeline/t1_b0_registration_pipeline/apply_flirt_registration/mapflow/_apply_flirt_registration1/sub-PREVDEMALS0010001BE_ses-M0_tissue-whitematter_binaryMask_flirt.nii.gz'], 't1-register-b0.@out_registered_t1': ['/Users/junhao.wen/test/NODDI/working_dir/noddi_postprocessing_pipeline/t1_b0_registration_pipeline/registration_t1_to_b0/mapflow/_registration_t1_to_b00/sub-PREVDEMALS0010003PB_ses-M0_brainExtractedT1w_flirt.nii.gz', '/Users/junhao.wen/test/NODDI/working_dir/noddi_postprocessing_pipeline/t1_b0_registration_pipeline/registration_t1_to_b0/mapflow/_registration_t1_to_b01/sub-PREVDEMALS0010001BE_ses-M0_brainExtractedT1w_flirt.nii.gz']}
  • base_directory : /Users/junhao.wen/test/NODDI/CAPS
  • bucket :
  • container : subjects/sub-PREVDEMALS0010003PB/ses-M0/noddi/postprocessing
  • creds_path :
  • encrypt_bucket_keys :
  • ignore_exception : False
  • local_copy :
  • parameterization : True
  • regexp_substitutions : [('/_destrieux_in_diffusion_space./', '/'), ('/_desikan_in_diffusion_space./', '/'), ('/_apply_flirt_registration./', '/'), ('/_registration_t1_to_b0./', '/'), ('/_convert_flirt_to_mrtrix./', '/')]
  • remove_dest_dir : True
  • strip_dir :
  • substitutions : [('sub-PREVDEMALS0010003PB_ses-M0_brainExtractedT1w_flirt.mat', 'sub-PREVDEMALS0010003PB_ses-M0_t1-to-b0_withResampling.mat'), ('sub-PREVDEMALS0010003PB_ses-M0_tissue-whitematter_binaryMask_flirt.nii.gz', 'sub-PREVDEMALS0010003PB_ses-M0_binarymask-whitematter_reslicedOnDiffusionSpace.nii.gz'), ('sub-PREVDEMALS0010003PB_ses-M0mrtrix_matrix.mat', 'sub-PREVDEMALS0010003PB_ses-M0_t1-to-b0_withoutResampling.mat'), ('sub-PREVDEMALS0010003PB_ses-M0desikan_deformed_image.nii.gz', 'sub-PREVDEMALS0010003PB_ses-M0_parcellation-desikan_onDiffusionSpace.nii.gz'), ('sub-PREVDEMALS0010003PB_ses-M0destrieux_deformed_image.nii.gz', 'sub-PREVDEMALS0010003PB_ses-M0_parcellation-destrieux_onDiffusionSpace.nii.gz')]

Execution Inputs

  • _outputs : {'t1-register-b0.@out_desikan_in_diffusion_space': ['/Users/junhao.wen/test/NODDI/working_dir/noddi_postprocessing_pipeline/t1_b0_registration_pipeline/desikan_in_diffusion_space/mapflow/_desikan_in_diffusion_space0/sub-PREVDEMALS0010003PB_ses-M0_desikan_deformed_image.nii.gz', '/Users/junhao.wen/test/NODDI/working_dir/noddi_postprocessing_pipeline/t1_b0_registration_pipeline/desikan_in_diffusion_space/mapflow/_desikan_in_diffusion_space1/sub-PREVDEMALS0010001BE_ses-M0_desikan_deformed_image.nii.gz'], 't1-register-b0.@out_destrieux_in_diffusion_space': ['/Users/junhao.wen/test/NODDI/working_dir/noddi_postprocessing_pipeline/t1_b0_registration_pipeline/destrieux_in_diffusion_space/mapflow/_destrieux_in_diffusion_space0/sub-PREVDEMALS0010003PB_ses-M0_destrieux_deformed_image.nii.gz', '/Users/junhao.wen/test/NODDI/working_dir/noddi_postprocessing_pipeline/t1_b0_registration_pipeline/destrieux_in_diffusion_space/mapflow/_destrieux_in_diffusion_space1/sub-PREVDEMALS0010001BE_ses-M0_destrieux_deformed_image.nii.gz'], 't1-register-b0.@out_mrtrix_matrix': ['/Users/junhao.wen/test/NODDI/working_dir/noddi_postprocessing_pipeline/t1_b0_registration_pipeline/convert_flirt_to_mrtrix/mapflow/_convert_flirt_to_mrtrix0/sub-PREVDEMALS0010003PB_ses-M0_mrtrix_matrix.mat', '/Users/junhao.wen/test/NODDI/working_dir/noddi_postprocessing_pipeline/t1_b0_registration_pipeline/convert_flirt_to_mrtrix/mapflow/_convert_flirt_to_mrtrix1/sub-PREVDEMALS0010001BE_ses-M0_mrtrix_matrix.mat'], 't1-register-b0.@out_flirt_matrix': ['/Users/junhao.wen/test/NODDI/working_dir/noddi_postprocessing_pipeline/t1_b0_registration_pipeline/registration_t1_to_b0/mapflow/_registration_t1_to_b00/sub-PREVDEMALS0010003PB_ses-M0_brainExtractedT1w_flirt.mat', '/Users/junhao.wen/test/NODDI/working_dir/noddi_postprocessing_pipeline/t1_b0_registration_pipeline/registration_t1_to_b0/mapflow/_registration_t1_to_b01/sub-PREVDEMALS0010001BE_ses-M0_brainExtractedT1w_flirt.mat'], 't1-register-b0.@out_wm_mask_in_diffusion_mask': ['/Users/junhao.wen/test/NODDI/working_dir/noddi_postprocessing_pipeline/t1_b0_registration_pipeline/apply_flirt_registration/mapflow/_apply_flirt_registration0/sub-PREVDEMALS0010003PB_ses-M0_tissue-whitematter_binaryMask_flirt.nii.gz', '/Users/junhao.wen/test/NODDI/working_dir/noddi_postprocessing_pipeline/t1_b0_registration_pipeline/apply_flirt_registration/mapflow/_apply_flirt_registration1/sub-PREVDEMALS0010001BE_ses-M0_tissue-whitematter_binaryMask_flirt.nii.gz'], 't1-register-b0.@out_registered_t1': ['/Users/junhao.wen/test/NODDI/working_dir/noddi_postprocessing_pipeline/t1_b0_registration_pipeline/registration_t1_to_b0/mapflow/_registration_t1_to_b00/sub-PREVDEMALS0010003PB_ses-M0_brainExtractedT1w_flirt.nii.gz', '/Users/junhao.wen/test/NODDI/working_dir/noddi_postprocessing_pipeline/t1_b0_registration_pipeline/registration_t1_to_b0/mapflow/_registration_t1_to_b01/sub-PREVDEMALS0010001BE_ses-M0_brainExtractedT1w_flirt.nii.gz']}
  • base_directory : /Users/junhao.wen/test/NODDI/CAPS
  • bucket :
  • container : subjects/sub-PREVDEMALS0010003PB/ses-M0/noddi/postprocessing
  • creds_path :
  • encrypt_bucket_keys :
  • ignore_exception : False
  • local_copy :
  • parameterization : True
  • regexp_substitutions : [('/_destrieux_in_diffusion_space./', '/'), ('/_desikan_in_diffusion_space./', '/'), ('/_apply_flirt_registration./', '/'), ('/_registration_t1_to_b0./', '/'), ('/_convert_flirt_to_mrtrix./', '/')]
  • remove_dest_dir : True
  • strip_dir :
  • substitutions : [('sub-PREVDEMALS0010003PB_ses-M0_brainExtractedT1w_flirt.mat', 'sub-PREVDEMALS0010003PB_ses-M0_t1-to-b0_withResampling.mat'), ('sub-PREVDEMALS0010003PB_ses-M0_tissue-whitematter_binaryMask_flirt.nii.gz', 'sub-PREVDEMALS0010003PB_ses-M0_binarymask-whitematter_reslicedOnDiffusionSpace.nii.gz'), ('sub-PREVDEMALS0010003PB_ses-M0mrtrix_matrix.mat', 'sub-PREVDEMALS0010003PB_ses-M0_t1-to-b0_withoutResampling.mat'), ('sub-PREVDEMALS0010003PB_ses-M0desikan_deformed_image.nii.gz', 'sub-PREVDEMALS0010003PB_ses-M0_parcellation-desikan_onDiffusionSpace.nii.gz'), ('sub-PREVDEMALS0010003PB_ses-M0destrieux_deformed_image.nii.gz', 'sub-PREVDEMALS0010003PB_ses-M0_parcellation-destrieux_onDiffusionSpace.nii.gz')]

Execution Outputs

  • out_file : ['/Users/junhao.wen/test/NODDI/CAPS/subjects/sub-PREVDEMALS0010003PB/ses-M0/noddi/postprocessing/t1-register-b0/sub-PREVDEMALS0010003PB_ses-M0_desikan_deformed_image.nii.gz', '/Users/junhao.wen/test/NODDI/CAPS/subjects/sub-PREVDEMALS0010003PB/ses-M0/noddi/postprocessing/t1-register-b0/sub-PREVDEMALS0010001BE_ses-M0_desikan_deformed_image.nii.gz', '/Users/junhao.wen/test/NODDI/CAPS/subjects/sub-PREVDEMALS0010003PB/ses-M0/noddi/postprocessing/t1-register-b0/sub-PREVDEMALS0010003PB_ses-M0_t1-to-b0_withResampling.mat', '/Users/junhao.wen/test/NODDI/CAPS/subjects/sub-PREVDEMALS0010003PB/ses-M0/noddi/postprocessing/t1-register-b0/sub-PREVDEMALS0010001BE_ses-M0_brainExtractedT1w_flirt.mat', '/Users/junhao.wen/test/NODDI/CAPS/subjects/sub-PREVDEMALS0010003PB/ses-M0/noddi/postprocessing/t1-register-b0/sub-PREVDEMALS0010003PB_ses-M0_binarymask-whitematter_reslicedOnDiffusionSpace.nii.gz', '/Users/junhao.wen/test/NODDI/CAPS/subjects/sub-PREVDEMALS0010003PB/ses-M0/noddi/postprocessing/t1-register-b0/sub-PREVDEMALS0010001BE_ses-M0_tissue-whitematter_binaryMask_flirt.nii.gz', '/Users/junhao.wen/test/NODDI/CAPS/subjects/sub-PREVDEMALS0010003PB/ses-M0/noddi/postprocessing/t1-register-b0/sub-PREVDEMALS0010003PB_ses-M0_destrieux_deformed_image.nii.gz', '/Users/junhao.wen/test/NODDI/CAPS/subjects/sub-PREVDEMALS0010003PB/ses-M0/noddi/postprocessing/t1-register-b0/sub-PREVDEMALS0010001BE_ses-M0_destrieux_deformed_image.nii.gz', '/Users/junhao.wen/test/NODDI/CAPS/subjects/sub-PREVDEMALS0010003PB/ses-M0/noddi/postprocessing/t1-register-b0/sub-PREVDEMALS0010003PB_ses-M0_brainExtractedT1w_flirt.nii.gz', '/Users/junhao.wen/test/NODDI/CAPS/subjects/sub-PREVDEMALS0010003PB/ses-M0/noddi/postprocessing/t1-register-b0/sub-PREVDEMALS0010001BE_ses-M0_brainExtractedT1w_flirt.nii.gz', '/Users/junhao.wen/test/NODDI/CAPS/subjects/sub-PREVDEMALS0010003PB/ses-M0/noddi/postprocessing/t1-register-b0/sub-PREVDEMALS0010003PB_ses-M0_mrtrix_matrix.mat', '/Users/junhao.wen/test/NODDI/CAPS/subjects/sub-PREVDEMALS0010003PB/ses-M0/noddi/postprocessing/t1-register-b0/sub-PREVDEMALS0010001BE_ses-M0_mrtrix_matrix.mat']

Runtime info

  • duration : 0.12841
  • hostname : UMR-COGIM-MP008

Environment



* ANTSPATH : /usr/local/bin
* Apple_PubSub_Socket_Render : /private/tmp/com.apple.launchd.y9tXOMjej2/Render
* CAMINO_TRACKVIS : /Users/junhao.wen/Software/camino-trackvis/camino-trackvis-0.2.8.1
* CC : /usr/bin/clang
* CLICOLOR : 1
* CLINICAPATH : /Users/junhao.wen/Hao/Project/CLINICAPATH
* CLINICA_HOME : /Users/junhao.wen/Hao/Project/clinica
* CXX : /usr/bin/clang++
* DEFAULT_USER : junhao.wen
* DISPLAY : /private/tmp/com.apple.launchd.pN9QLbr6LO/org.macosforge.xquartz:0
* DSI_PATH : /Applications/Diffusion\ Toolkit.app/Contents/MacOS/matrices/
* EIGEN_CFLAGS : -isystem /usr/local/Cellar/eigen/3.2.9/include/eigen3/
* ETS_TOOLKIT : qt4
* FIX_VERTEX_AREA :
* FMRI_ANALYSIS_DIR : /Applications/freesurfer/fsfast
* FREESURFER_HOME : /Applications/freesurfer
* FSFAST_HOME : /Applications/freesurfer/fsfast
* FSF_OUTPUT_FORMAT : nii.gz
* FSLDIR : /usr/local/fsl
* FSLGECUDAQ : cuda.q
* FSLLOCKDIR :
* FSLMACHINELIST :
* FSLMULTIFILEQUIT : TRUE
* FSLOUTPUTTYPE : NIFTI_GZ
* FSLREMOTECALL :
* FSLTCLSH : /usr/local/fsl/bin/fsltclsh
* FSLWISH : /usr/local/fsl/bin/fslwish
* FSL_BIN : /usr/local/fsl/bin
* FSL_DIR : /usr/local/fsl
* FS_OVERRIDE : 0
* FUNCTIONALS_DIR : /Applications/freesurfer/sessions
* GREP_OPTIONS : --color=auto
* HOME : /Users/junhao.wen
* ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS : 4
* JAVA_HOME : /Library/Java/Home
* LANG : en_US.UTF-8
* LC_CTYPE :
* LESS : -R
* LOCAL_DIR : /Applications/freesurfer/local
* LOGNAME : junhao.wen
* LSCOLORS : gxBxhxDxfxhxhxhxhxcxcx
* MANPATH : /Users/junhao.wen/Software/camino/camino-code-be29107b79b741fea838a50cc419700dd79536bf/man:
* MATLABCMD : /Applications/MATLAB_R2016a.app/bin/
* MINC_BIN_DIR : /Applications/freesurfer/mni/bin
* MINC_LIB_DIR : /Applications/freesurfer/mni/lib
* MNI_DATAPATH : /Applications/freesurfer/mni/data
* MNI_DIR : /Applications/freesurfer/mni
* MNI_PERL5LIB : /Applications/freesurfer/mni/lib/../Library/Perl/Updates/5.12.3
* MRTRIX_HOME : /Users/junhao.wen/Software/mrtrix3_24042017
* OLDPWD : /Applications/PyCharmCE.app/Contents/bin
* OS : Darwin
* PAGER : less
* PATH : /Users/junhao.wen/Hao/Project/clinica_realted/GBSS/NODDI:/Applications/MRIcron:/Applications/Convert3DGUI.app/Contents/bin:/Users/junhao.wen/Software/camino-trackvis/camino-trackvis-0.2.8.1/bin:/Applications/TrackVis.app/Contents/MacOS:/Users/junhao.wen/Software/camino/camino-code-be29107b79b741fea838a50cc419700dd79536bf/bin:/usr/bin/java:/Users/junhao.wen/Software/afni/abin/macosx_10.7_Intel_64:/usr/local/bin:/usr/local/fsl/bin:/Users/junhao.wen/Hao/MRI-modalities/Logiciels/FreeSurfer/QAtools_v1.1:/Applications/freesurfer/bin:/Applications/freesurfer/fsfast/bin:/Applications/freesurfer/tktools:/usr/local/fsl/bin:/Applications/freesurfer/mni/bin:/Users/junhao.wen/.local/bin:/Users/junhao.wen/Software/mrtrix3_24042017/bin:/usr/local/opt/qt5/bin:/Library/TeX/texbin:/Users/junhao.wen/anaconda2/bin:/Users/junhao.wen/bin:/usr/local/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/aria2/bin:/Library/TeX/texbin:/Applications/MATLAB_R2016a.app/bin/:/opt/local/bin
* PERL5LIB : /Applications/freesurfer/mni/lib/../Library/Perl/Updates/5.12.3
* PIPEDREAMPATH : /Users/junhao.wen/Software/pipedream/pipedream
* PYCHARM_HOSTED : 1
* PYTHONIOENCODING : UTF-8
* PYTHONPATH : /Users/junhao.wen/Hao/Project/NODDI
* PYTHONUNBUFFERED : 1
* QA_TOOLS : /Users/junhao.wen/Hao/MRI-modalities/Logiciels/FreeSurfer/QAtools_v1.1
* QT_API : pyqt
* SHELL : /bin/zsh
* SPM_HOME : /Users/junhao.wen/Software/spm12
* SSH_AUTH_SOCK : /private/tmp/com.apple.launchd.JgBWc7hpUO/Listeners
* SSH_KEY_PATH : ~/.ssh/dsa_id
* SUBJECTS_DIR : /Applications/freesurfer/subjects
* TMPDIR : /var/folders/5r/10gy0dnn1x1dykzyb4117pcc000hzl/T/
* USER : junhao.wen
* XPC_FLAGS : 0x0
* XPC_SERVICE_NAME : com.jetbrains.pycharm.184992
* ZSH : /Users/junhao.wen/.oh-my-zsh
* __CF_USER_TEXT_ENCODING : 0x43F3:0x0:0x1

 ~/test/NODDI/working_dir/noddi_postprocessing_pipeline/t1_b0_registration_pipeline/datasinker/mapflow/_datasinker0  ls
_0x310393888238810bc2b2f3e7a8f368e0.json
_inputs.pklz
_node.pklz
_report
result__datasinker0.pklz

I do not know why this subject's result sub-PREVDEMALS0010001BE_ses-M0 has been mixed inside this subject...

anbai106 commented 7 years ago

Was this because that we have to give datasink's parameters explicitly? like this:

    write_node = npe.MapNode(name='WritingCAPS',
                             iterfield=['container'] + self.get_output_fields(),
                             interface=nio.DataSink(infields=self.get_output_fields()))
    write_node.inputs.base_directory = self.caps_directory
    write_node.inputs.parameterization = False
    write_node.inputs.container = ['subjects/' + self.subjects[i] + '/' + self.sessions[i] +
                                   '/fmri/preprocessing' for i in range(len(self.subjects))]
    write_node.inputs.remove_dest_dir = True
    write_node.inputs.regexp_substitutions = [
        (r't1_brain_mask/c3(.+)_maths_dil_ero_thresh_fillh\.nii\.gz$', r'\1_brainmask.nii.gz'),
        (r'mc_params/rp_a(.+)\.txt$', r'\1_confounds.tsv'),
        (r'native_fmri/ua(.+)\.nii.gz$', r'\1_space-meanBOLD.nii.gz'),
        (r't1_fmri/rua(.+)\.nii.gz$', r'\1_space-T1w.nii.gz'),
        (r'mni_fmri/wrua(.+)\.nii.gz$', r'\1_space-Ixi549Space.nii.gz'),
        (r'mni_smoothed_fmri/swrua(.+)\.nii.gz$',
         r'\1_space-Ixi549Space_fwhm-'+'-'.join(map(str, self.parameters['full_width_at_half_maximum']))+'.nii.gz'),
        # I don't know why it's adding this empty folder, so I remove it:
        (r'trait_added', r''),
    ]

Because in my case, I create another function to get the parameters like container, substitutions, and connect this function with MapNode datasink

anbai106 commented 7 years ago

Solved, I will close this