search

nipy / nipype

Workflows and interfaces for neuroimaging packages
https://nipype.readthedocs.org/en/latest/
Other
751 stars 530 forks source link

FIRST segmentation throws traiterror #2872

Open RiaanZoetmulder opened 5 years ago

RiaanZoetmulder commented 5 years ago

Summary

I am writing code to create several segmentations, one of which is segmentations using FIRST. This seems to create some issues however. It throws this error and no crash log: 

                Currently running:
                   * _segmentation1
                   * _segmentation0

190203-13:17:07,238 nipype.workflow WARNING: [Node] Error on "_segmentation1" (/mnt/e/Projects/medical-taskonomy/testdata/example_preprocess/segmentation/mapflow/_segmentation1) 190203-13:17:08,681 nipype.workflow ERROR: Node _segmentation1 failed to run on host 18-012665. 190203-13:17:08,684 nipype.workflow ERROR: Saving crash info to /home/rzoetmulder/crash-20190203-131708-rzoetmulder-_segmentation1-57d356fa-459c-44c2-aa7c-a08fc2626ff8.pklz Traceback (most recent call last): File "/home/rzoetmulder/anaconda3/envs/nipype/lib/python2.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 69, in run_node result['result'] = node.run(updatehash=updatehash) File "/home/rzoetmulder/anaconda3/envs/nipype/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.py", line 473, in run result = self._run_interface(execute=True) File "/home/rzoetmulder/anaconda3/envs/nipype/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.py", line 557, in _run_interface return self._run_command(execute) File "/home/rzoetmulder/anaconda3/envs/nipype/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.py", line 637, in _run_command result = self._interface.run(cwd=outdir) File "/home/rzoetmulder/anaconda3/envs/nipype/lib/python2.7/site-packages/nipype/interfaces/base/core.py", line 371, in run outputs = self.aggregate_outputs(runtime) File "/home/rzoetmulder/anaconda3/envs/nipype/lib/python2.7/site-packages/nipype/interfaces/base/core.py", line 472, in aggregate_outputs raise error TraitError: The trait 'vtk_surfaces' of a FIRSTOutputSpec instance is an existing file name, but the path '/mnt/e/Projects/medical-taskonomy/testdata/example_preprocess/segmentation/mapflow/_segmentation1/segmented-R_HippL_Accu_first.vtk' does not exist.

190203-13:17:08,705 nipype.workflow INFO: [MultiProc] Running 1 tasks, and 0 jobs ready. Free memory (GB): 57.12/57.32, Free processors: 3/4.
Currently running:

190203-13:17:20,704 nipype.workflow INFO: [MultiProc] Running 0 tasks, and 0 jobs ready. Free memory (GB): 57.32/57.32, Free processors: 4/4. 190203-13:17:22,664 nipype.workflow INFO:

190203-13:17:22,667 nipype.workflow ERROR: could not run node: example_preprocess.segmentation 190203-13:17:22,670 nipype.workflow INFO: crashfile: /home/rzoetmulder/crash-20190203-131708-rzoetmulder-_segmentation1-57d356fa-459c-44c2-aa7c-a08fc2626ff8.pklz 190203-13:17:22,674 nipype.workflow ERROR: could not run node: example_preprocess.segmentation 190203-13:17:22,676 nipype.workflow INFO: crashfile: /home/rzoetmulder/crash-20190203-131720-rzoetmulder-_segmentation0-6b6a4de0-b131-4dfe-9e1f-916d4c8c79b2.pklz 190203-13:17:22,679 nipype.workflow INFO:

RuntimeError Traceback (most recent call last)

in () 105 process.write_graph(graph2use='colored', format='pdf', simple_form=True) 106 --> 107 process.run('MultiProc', plugin_args={'n_procs': 4}) /home/rzoetmulder/anaconda3/envs/nipype/lib/python2.7/site-packages/nipype/pipeline/engine/workflows.pyc in run(self, plugin, plugin_args, updatehash) 597 if str2bool(self.config['execution']['create_report']): 598 self._write_report_info(self.base_dir, self.name, execgraph) --> 599 runner.run(execgraph, updatehash=updatehash, config=self.config) 600 datestr = datetime.utcnow().strftime('%Y%m%dT%H%M%S') 601 if str2bool(self.config['execution']['write_provenance']): /home/rzoetmulder/anaconda3/envs/nipype/lib/python2.7/site-packages/nipype/pipeline/plugins/base.pyc in run(self, graph, config, updatehash) 189 190 self._remove_node_dirs() --> 191 report_nodes_not_run(notrun) 192 193 # close any open resources /home/rzoetmulder/anaconda3/envs/nipype/lib/python2.7/site-packages/nipype/pipeline/plugins/tools.pyc in report_nodes_not_run(notrun) 80 logger.debug(subnode._id) 81 logger.info("***********************************") ---> 82 raise RuntimeError(('Workflow did not execute cleanly. ' 83 'Check log for details')) 84 RuntimeError: Workflow did not execute cleanly. Check log for details ### Actual behavior All the other methods (canny edge detector, BET etc.) work. So my installation of FSL/Nipype should be correct. I have set FIRST to only segment one structure namely: 'R_HippL_Accu'. It throws the error shown above, I think it just doesn't save the surface mesh in the correct folder. It doesn't work with different structures either. I have already gone to https://miykael.github.io/nipype_tutorial/notebooks/basic_error_and_crashes.html to figure out the crashes, though I couldn't fix it. ### Script/Workflow details [Link to code](https://pastebin.com/z5cWMh1U) ### Platform details: Using windows with linux subsystems. Version of Linux: Ubuntu version of windows: windows 10 ### Execution environment Anaconda with jupyter lab.
satra commented 5 years ago

this is strange as we have a very similar workflow that we used here with FSL 5.0.9 and nipype 0.12: https://github.com/ReproNim/simple_workflow/blob/master/run_demo_workflow.py#L103

could you please let us know which version of FSL you are using?

RiaanZoetmulder commented 5 years ago

'which fsl' give:

/usr/lib/fsl/5.0/fsl

So version 5.0

satra commented 5 years ago

we will need the subversion to check. could you please paste the output of this command?

python -c "from nipype.interfaces.fsl import Info; print(Info.version())"

RiaanZoetmulder commented 5 years ago

This yields: 5.0.9

satra commented 5 years ago

thanks @RiaanZoetmulder - the only things i can think of are that your FIRST data files are missing:

you can check this with:

ls $FSLDIR/data/first/models*

and when you install FSL in your windows subsystem, you can do:

apt-get install fsl-core fsl-first-data

other than these two, i'm at a loss as to why those files would not be produced by the same version of FSL.

mgxd commented 5 years ago

@RiaanZoetmulder this may be related to #2813 - could you check the type of the transformation matrix generated? it should be in the /mnt/e/Projects/medical-taskonomy/testdata/example_preprocess/segmentation/mapflow/_segmentation1 folder

RiaanZoetmulder commented 5 years ago

thanks @RiaanZoetmulder - the only things i can think of are that your FIRST data files are missing:

you can check this with:

ls $FSLDIR/data/first/models*

and when you install FSL in your windows subsystem, you can do:

apt-get install fsl-core fsl-first-data

other than these two, i'm at a loss as to why those files would not be produced by the same version of FSL.

Hey, Thanks for your response. I have run the command apt-get install fsl-first-data . fsl-core should already be installed otherwise BET would already not be working right?

I have checked the directory, I can find the files here:

/usr/share/fsl/data/first/models_336_bin

There is a bunch of .bmv files there, with the names of anatomical parts of the brein before the dot.

Should I link these files somewhere?

RiaanZoetmulder commented 5 years ago

@RiaanZoetmulder this may be related to #2813 - could you check the type of the transformation matrix generated? it should be in the /mnt/e/Projects/medical-taskonomy/testdata/example_preprocess/segmentation/mapflow/_segmentation1 folder

Do you mean the .mat file?

It is named: age0002_ABI50032_brain_to_std_sub.mat

RiaanZoetmulder commented 5 years ago

Just to make sure you have the overview of what I am doing. Before first I rotate, resample and interpolate and skull strip. Might this influence it?

satra commented 5 years ago

@RiaanZoetmulder - the files do not have to be linked. as long as FSL_DIR is defined it should work (in principle)

a couple of other ways to debug:

  1. use the cli

nipypecli crash crashfile.pklz -r -i

this will run the node in an isolated directory and drop you into the ipython debugger (if ipython is available in your python environment). then you can check if first generated the files anywhere at all.

  1. go into the node directory. you will find a command.txt and just execute the command inside it and check if the surface files are created.

  2. run the same thing inside a docker container to check if it's specific to your OS. i should say we don't have good testing on windows because of it's various options and interactions with the windows subsystem for unix.

RiaanZoetmulder commented 5 years ago

I have run first in a different way. Firstly, the command line tool seems to work. When I run the python code like this however (different from my current implementation):

from nipype.interfaces import fsl first = fsl.FIRST() first.inputs.in_file = '/mnt/e/Projects/medical-taskonomy/testdata/data/original/age0001_ABI50030.nii' first.inputs.out_file = 'segmented.nii' res = first.run()

What happens when out_file is set to something in my windows environment in the mnt folder is that it saves the temporary outputs there, but when it looks for the vtk files it doesn't look in the output folder.

I can run it however in my home folder in the linux subsystem. Very strange behavior.

satra commented 5 years ago

@RiaanZoetmulder - if you run it directly it will not do several things to isolate the computation. you can instead run it inside a Node object.

from nipype import Node
from nipype.interfaces import fsl
first = Node(fsl.FIRST(), name='FIRST')
first.inputs.in_file = '/mnt/e/Projects/medical-taskonomy/testdata/data/original/age0001_ABI50030.nii'
first.inputs.out_file = 'segmented.nii'
res = first.run()
RiaanZoetmulder commented 5 years ago

@RiaanZoetmulder - if you run it directly it will not do several things to isolate the computation. you can instead run it inside a Node object.

from nipype import Node
from nipype.interfaces import fsl
first = Node(fsl.FIRST(), name='FIRST')
first.inputs.in_file = '/mnt/e/Projects/medical-taskonomy/testdata/data/original/age0001_ABI50030.nii'
first.inputs.out_file = 'segmented.nii'
res = first.run()

I have done that to check what happens:

from nipype import Node from nipype.interfaces import fsl first = Node(fsl.FIRST(), name='FIRST') first.inputs.in_file = '/mnt/e/Projects/medical-taskonomy/testdata/data/original/age0001_ABI50030.nii' first.inputs.out_file = 'segmented.nii' res = first.run()

This puts the output in the tmp/tmpdOjL15/FIRST and works

However if I want to change my output folder to:

first.inputs.out_file = '/mnt/e/Projects/medical-taskonomy/testdata/data/segmented.nii'

It puts all of the .vtk and .bvar files in that folder, but when it collects all of these to make the final segmentation file it looks in the /tmp/tmpdOjL15/FIRST/ folder.

This is the error it throws:

190208-16:04:19,11 nipype.workflow INFO: [Node] Setting-up "FIRST" in "/tmp/tmpdOjL15/FIRST". 190208-16:04:19,44 nipype.workflow INFO: [Node] Running "FIRST" ("nipype.interfaces.fsl.preprocess.FIRST"), a CommandLine Interface with command: run_first_all -m auto -i /tmp/tmpdOjL15/FIRST/age0001_ABI50030.nii -o /mnt/e/Projects/medical-taskonomy/testdata/data/segmented.nii 190208-16:14:32,157 nipype.interface INFO: stdout 2019-02-08T16:14:32.157138:23403 190208-16:14:32,319 nipype.workflow WARNING: [Node] Error on "FIRST" (/tmp/tmpdOjL15/FIRST)

TraitError Traceback (most recent call last)

in () 4 first.inputs.in_file = '/mnt/e/Projects/medical-taskonomy/testdata/data/original/age0001_ABI50030.nii' 5 first.inputs.out_file = '/mnt/e/Projects/medical-taskonomy/testdata/data/segmented.nii' ----> 6 res = first.run() /home/rzoetmulder/anaconda3/envs/nipype/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.pyc in run(self, updatehash) 471 472 try: --> 473 result = self._run_interface(execute=True) 474 except Exception: 475 logger.warning('[Node] Error on "%s" (%s)', self.fullname, outdir) /home/rzoetmulder/anaconda3/envs/nipype/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.pyc in _run_interface(self, execute, updatehash) 555 self._update_hash() 556 return self._load_results() --> 557 return self._run_command(execute) 558 559 def _load_results(self): /home/rzoetmulder/anaconda3/envs/nipype/lib/python2.7/site-packages/nipype/pipeline/engine/nodes.pyc in _run_command(self, execute, copyfiles) 635 logger.info(message) 636 try: --> 637 result = self._interface.run(cwd=outdir) 638 except Exception as msg: 639 result.runtime.stderr = '%s\n\n%s'.format( /home/rzoetmulder/anaconda3/envs/nipype/lib/python2.7/site-packages/nipype/interfaces/base/core.pyc in run(self, cwd, ignore_exception, **inputs) 369 runtime = self._run_interface(runtime) 370 runtime = self._post_run_hook(runtime) --> 371 outputs = self.aggregate_outputs(runtime) 372 except Exception as e: 373 import traceback /home/rzoetmulder/anaconda3/envs/nipype/lib/python2.7/site-packages/nipype/interfaces/base/core.pyc in aggregate_outputs(self, runtime, needed_outputs) 470 "'%s'." % (val, self.__class__.__name__, key)) 471 raise FileNotFoundError(msg) --> 472 raise error 473 474 return outputs TraitError: The trait 'vtk_surfaces' of a FIRSTOutputSpec instance is an existing file name, but the path '/tmp/tmpdOjL15/FIRST/segmented-L_Hipp_first.vtk' does not exist.
satra commented 5 years ago

@RiaanZoetmulder - ah then we know the issue here. i would file this as a fixable FIRST bug (would love a PR if you have the bandwidth to do so - i'm sure we can guide you).

in general when Node is used, we don't expect the workflow developer to specify an absolute output path. in general Nipype suggests that this should happen with a DataSink. however, we can consider the scenario in FIRST, where such a file is specified and adjust the outputs appropriately.

RiaanZoetmulder commented 5 years ago

@RiaanZoetmulder - ah then we know the issue here. i would file this as a fixable FIRST bug (would love a PR if you have the bandwidth to do so - i'm sure we can guide you).

in general when Node is used, we don't expect the workflow developer to specify an absolute output path. in general Nipype suggests that this should happen with a DataSink. however, we can consider the scenario in FIRST, where such a file is specified and adjust the outputs appropriately.

Sent you an email regarding that! Let's continue the conversation there!

mgxd commented 5 years ago

Hi Riann - I went ahead and made a PR that addresses this bug. You can try it out by installing that branch: pip install -U https://github.com/mgxd/nipype/archive/fix/first.zip