search

reckbo / dtipipe

0 stars 0 forks source link

DwiEddy task fails to run #2

Open tashrifbillah opened 4 years ago

tashrifbillah commented 4 years ago

Include the following in DwiEddy.py:

if __name__=='__main__':
    build([DwiEddy(dicom_dir='/home/tb571/Documents/dwi_validation_data/dtitest_GE_Nottingham/A',
    output_session_dir='/tmp',
    output_basename='myOutput',
    dcm2niix_bin='/home/tb571/Downloads/dcm2niix/console/dcm2niix',
    fsldir='/home/tb571/fsl', num_proc_eddy=16)], local_scheduler=True)

Execute the above script:

python /path/to/DwEddy.py

Error

   DwiEddy(dicom_dir=/home/tb571/Documents/dwi_validation_data/dtitest_GE_Nottingham/A, output_session_dir=/tmp, output_basename=myOutput, dcm2niix_bin=/home/tb571/Downloads/dcm2niix/console/dcm2niix, fsldir=/home/tb571/fsl)
ERROR: [pid 117307] Worker Worker(salt=941824505, workers=1, host=pnl-z840-2, username=tb571, pid=117307) failed    DwiEddy(dicom_dir=/home/tb571/Documents/dwi_validation_data/dtitest_GE_Nottingham/A, output_session_dir=/tmp, output_basename=myOutput, dcm2niix_bin=/home/tb571/Downloads/dcm2niix/console/dcm2niix, fsldir=/home/tb571/fsl)
Traceback (most recent call last):
  File "/home/tb571/miniconda3/lib/python3.6/site-packages/luigi/worker.py", line 184, in run
    raise RuntimeError('Unfulfilled %s at run time: %s' % (deps, ', '.join(missing)))
RuntimeError: Unfulfilled dependency at run time: DwiNifti__home_tb571_Down__home_tb571_Docu_myOutput_0cdd573d67
DEBUG: 1 running tasks, waiting for next task to finish
INFO: Informed scheduler that task   DwiEddy__home_tb571_Down__home_tb571_Docu__home_tb571_fsl_3f210b3202   has status   FAILED
DEBUG: Asking scheduler for work...
DEBUG: Done
DEBUG: There are no more tasks to run at this time
DEBUG: There are 1 pending tasks possibly being run by other workers
DEBUG: There are 1 pending tasks unique to this worker
DEBUG: There are 1 pending tasks last scheduled by this worker
INFO: Worker Worker(salt=941824505, workers=1, host=pnl-z840-2, username=tb571, pid=117307) was stopped. Shutting down Keep-Alive thread
INFO:
===== Luigi Execution Summary =====

Scheduled 3 tasks of which:
* 1 complete ones were encountered:
    - 1 DicomDir(dicom_dir=/home/tb571/Documents/dwi_validation_data/dtitest_GE_Nottingham/A)
* 1 ran successfully:
    - 1 DwiNifti(dicom_dir=/home/tb571/Documents/dwi_validation_data/dtitest_GE_Nottingham/A, output_session_dir=/tmp, output_basename=myOutput, dcm2niix_bin=/home/tb571/Downloads/dcm2niix/console/dcm2niix)
* 1 failed:
    - 1 DwiEddy(...)

May be similar to what I have reported here https://github.com/spotify/luigi/issues/2867

tashrifbillah commented 4 years ago

By the way, I have been able to solve https://github.com/spotify/luigi/issues/2867. Given that knowledge, I'll take a look this issue again.

tashrifbillah commented 4 years ago

Please see my DWI-->Tracts pipeline here. The only thing I do more than you is define intermediate outputs in one place. On the other hand, I use basic luigi.Task all through my pipeline other than your custom BaseTask.