nipy / nipype

Workflows and interfaces for neuroimaging packages
https://nipype.readthedocs.org/en/latest/
Other
746 stars 530 forks source link

[BUG] FSL BET fails when bet actually did run successfully #1818

Open oesteban opened 7 years ago

oesteban commented 7 years ago

Summary

FSL BET interface fails if anything is written to the standard error.

Actual behavior

warnings can be written to the stderr, so any warning will make nipype understand that bet failed, although it didn't. The expected outputs are generated and in place.

170214-22:12:01,269 workflow ERROR:
         [u'Node fsl_bet.a0 failed to run on host c549-404.stampede.tacc.utexas.edu.']
170214-22:12:01,270 workflow INFO:
         Saving crash info to crash-20170214-221201-oesteban-fsl_bet.a0-a7591f3c-02ed-4605-81a6-5dd66896e05a.pklz
170214-22:12:01,270 workflow INFO:
         Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python2.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 52, in run_node
    result['result'] = node.run(updatehash=updatehash)
  File "/usr/local/miniconda/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.py", line 366, in run
    self._run_interface()
  File "/usr/local/miniconda/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.py", line 476, in _run_interface
    self._result = self._run_command(execute)
  File "/usr/local/miniconda/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.py", line 607, in _run_command
    result = self._interface.run()
  File "/usr/local/miniconda/lib/python2.7/site-packages/nipype/interfaces/base.py", line 1085, in run
    runtime = self._run_wrapper(runtime)
  File "/usr/local/miniconda/lib/python2.7/site-packages/nipype/interfaces/base.py", line 1728, in _run_wrapper
    runtime = self._run_interface(runtime)
  File "/usr/local/miniconda/lib/python2.7/site-packages/nipype/interfaces/fsl/preprocess.py", line 144, in _run_interface
    self.raise_exception(runtime)
  File "/usr/local/miniconda/lib/python2.7/site-packages/nipype/interfaces/base.py", line 1686, in raise_exception
    **runtime.dictcopy()))
RuntimeError: Command:
bet sub-S0576VBM_run-1_T1w_conformed_corrected.nii.gz sub-S0576VBM_run-1_T1w_conformed_corrected_brain.nii.gz -A
Standard output:

Standard error:
/bin/sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
/bin/sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)

Return code: 0
Interface BET failed to run. 

Expected behavior

Successful exit if bet actually run

How to replicate the behavior

Hard to say, I'm working with the poldracklab/mriqc docker image, converted to singularity container. I haven't

Platform details:

singularity exec $CRNENV_SINGULARITY_IMAGES/poldracklab_mriqc_0.9.0-0-2017-02-06-610bfaf4e52d.img python -c "import nipype; print(nipype.get_info())"
/bin/sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
{'nibabel_version': '2.1.0', 'networkx_version': '1.11', 'numpy_version': '1.12.0', 'sys_platform': 'linux2', 'sys_version': '2.7.12 |Continuum Analytics, Inc.| (default, Jul  2 2016, 17:42:40) \n[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]', 'commit_source': u'(none found)', 'commit_hash': u'<not found>', 'pkg_path': '/usr/local/miniconda/lib/python2.7/site-packages/nipype', 'sys_executable': '/usr/local/miniconda/bin/python', 'traits_version': '4.6.0', 'scipy_version': '0.18.1'}
chrisgorgo commented 7 years ago

I believe this is because FSL BET does not use correct exit codes and the only way we know it has failed is that it prints something on the standard error. In this case problems with LC_ALL led to stderr output.

On Tue, Feb 14, 2017 at 4:01 PM, Oscar Esteban notifications@github.com wrote:

Summary

FSL BET interface fails if anything is written to the standard error. Actual behavior

warnings can be written to the stderr, so any warning will make nipype understand that bet failed, although it didn't. The expected outputs are generated and in place.

170214-22:12:01,269 workflow ERROR: [u'Node fsl_bet.a0 failed to run on host c549-404.stampede.tacc.utexas.edu.'] 170214-22:12:01,270 workflow INFO: Saving crash info to crash-20170214-221201-oesteban-fsl_bet.a0-a7591f3c-02ed-4605-81a6-5dd66896e05a.pklz 170214-22:12:01,270 workflow INFO: Traceback (most recent call last): File "/usr/local/miniconda/lib/python2.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 52, in run_node result['result'] = node.run(updatehash=updatehash) File "/usr/local/miniconda/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.py", line 366, in run self._run_interface() File "/usr/local/miniconda/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.py", line 476, in _run_interface self._result = self._run_command(execute) File "/usr/local/miniconda/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.py", line 607, in _run_command result = self._interface.run() File "/usr/local/miniconda/lib/python2.7/site-packages/nipype/interfaces/base.py", line 1085, in run runtime = self._run_wrapper(runtime) File "/usr/local/miniconda/lib/python2.7/site-packages/nipype/interfaces/base.py", line 1728, in _run_wrapper runtime = self._run_interface(runtime) File "/usr/local/miniconda/lib/python2.7/site-packages/nipype/interfaces/fsl/preprocess.py", line 144, in _run_interface self.raise_exception(runtime) File "/usr/local/miniconda/lib/python2.7/site-packages/nipype/interfaces/base.py", line 1686, in raise_exception **runtime.dictcopy())) RuntimeError: Command: bet sub-S0576VBM_run-1_T1w_conformed_corrected.nii.gz sub-S0576VBM_run-1_T1w_conformed_corrected_brain.nii.gz -A Standard output:

Standard error: /bin/sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8) /bin/sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)

Return code: 0 Interface BET failed to run.

Expected behavior

Successful exit if bet actually run How to replicate the behavior

Hard to say, I'm working with the poldracklab/fmriprep docker image, converted to singularity container. I haven't Platform details:

singularity exec $CRNENV_SINGULARITY_IMAGES/poldracklab_mriqc_0.9.0-0-2017-02-06-610bfaf4e52d.img python -c "import nipype; print(nipype.get_info())" /bin/sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8) sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8) sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8) sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8) {'nibabel_version': '2.1.0', 'networkx_version': '1.11', 'numpy_version': '1.12.0', 'sys_platform': 'linux2', 'sys_version': '2.7.12 |Continuum Analytics, Inc.| (default, Jul 2 2016, 17:42:40) \n[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]', 'commit_source': u'(none found)', 'commit_hash': u'', 'pkg_path': '/usr/local/miniconda/lib/python2.7/site-packages/nipype', 'sys_executable': '/usr/local/miniconda/bin/python', 'traits_version': '4.6.0', 'scipy_version': '0.18.1'}

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/nipy/nipype/issues/1818, or mute the thread https://github.com/notifications/unsubscribe-auth/AAOkpyX516mwvKWxlzGjLWEJrRFdtvtQks5rckBjgaJpZM4MBJn8 .

satra commented 7 years ago

https://github.com/nipy/nipype/blob/f5ed4d68fc922fa3eea6a43036b09f79bfa94acb/nipype/interfaces/fsl/preprocess.py#L143

oesteban commented 7 years ago

Maybe check whether word "warning" is not in every line of stderr?