search

populse / populse_mia

Multiparametric Image Analysis
Other
10 stars 9 forks source link

[initialisation step] Export unconnected mandatory plugs does not work while Export all unconnected plugs works #286

Closed servoz closed 1 year ago

servoz commented 2 years ago

Anat_mriqc_pipeline (on the mriqcPlusReport branch of mia_processes) does not seem to work if we do Export unconnected mandatory plugs while everything seems to work fine if we do Export all unconnected plugs.

It could be interesting to reproduce the issue with a smaller pipeline but I haven't had the time to look into it yet and I'm opening this ticket to don't forget ... and try to fix it ASAP. So I have no idea for the moment if the issue comes from mia_processes, populse_mia or capsul ...

To reproduce:

  1. Open a project with an anatomic file
  2. Go to Pipeline Manager
  3. From the Package part (left of the window), drag and drop mia_processes-> pipelines-> processes -> Anat_mriqc_pipeline pipeline into the pipeline editor (New Pipeline tab)
  4. Right click on the anat_mriqc_pipeline_1 node then select "export unconnected mandatory plugs" Screenshot from 2022-09-14 17-33-09
  5. Left click on the inputs node, then in the node controller part (right of the window) hit the Filter button for the anat_file parameter
  6. Select the file to process and click the Ok button
  7. Run pipeline

The Run falls during initialisation with:

- Pipeline initialising ...

Completion / workflow...
pipeline: <capsul.pipeline.pipeline.Pipeline object at 0x7f34aee29db0>

. anat_mriqc_pipeline_1.conformimage (mia_processes.bricks.preprocess.others.processing.ConformImage) MIA node ...

. anat_mriqc_pipeline_1.anat_skullstrip_pipeline.n4_bias_field_correction (mia_processes.bricks.preprocess.ants.processes.N4BiasFieldCorrection) MIA node ...

. anat_mriqc_pipeline_1.anat_skullstrip_pipeline.skull_stripping (mia_processes.bricks.preprocess.afni.processes.SkullStripping) MIA node ...

. anat_mriqc_pipeline_1.anat_skullstrip_pipeline.calc (mia_processes.bricks.preprocess.afni.processes.Calc) MIA node ...

. anat_mriqc_pipeline_1.anat_skullstrip_pipeline.binarize (mia_processes.bricks.preprocess.others.processing.Binarize) MIA node ...

. anat_mriqc_pipeline_1.segment (mia_processes.bricks.preprocess.fsl.processes.Segment) MIA node ...

. anat_mriqc_pipeline_1.fwhmx (mia_processes.bricks.reports.processes.FWHMx) MIA node ...

. anat_mriqc_pipeline_1.anat_spatial_norm.mask_moving_image (mia_processes.bricks.preprocess.others.processing.Mask) MIA node ...

. anat_mriqc_pipeline_1.anat_spatial_norm.template_mask (mia_processes.bricks.preprocess.others.processing.Template) MIA node ...

During the <mia_processes.bricks.preprocess.others.processing.Template object at 0x7f3453b72720> process initialisation, some possible problems were detected:
- inheritance_dict attribute was not found ...

. anat_mriqc_pipeline_1.anat_spatial_norm.template (mia_processes.bricks.preprocess.others.processing.Template) MIA node ...

During the <mia_processes.bricks.preprocess.others.processing.Template object at 0x7f3453aaa2c0> process initialisation, some possible problems were detected:
- inheritance_dict attribute was not found ...

. anat_mriqc_pipeline_1.anat_spatial_norm.mask_fixed_image (mia_processes.bricks.preprocess.others.processing.Mask) MIA node ...

. anat_mriqc_pipeline_1.anat_spatial_norm.affine_initializer (mia_processes.bricks.preprocess.ants.processes.AffineInitializer) MIA node ...

. anat_mriqc_pipeline_1.anat_spatial_norm.registration (mia_processes.bricks.preprocess.ants.processes.Registration) MIA node ...

. anat_mriqc_pipeline_1.anat_headmask_pipeline.enhance (mia_processes.bricks.preprocess.others.processing.Enhance) MIA node ...

. anat_mriqc_pipeline_1.anat_headmask_pipeline.denoise (mia_processes.bricks.preprocess.dipy.processes.Denoise) MIA node ...

. anat_mriqc_pipeline_1.anat_headmask_pipeline.gradient_threshold (mia_processes.bricks.preprocess.others.processing.GradientThreshold) MIA node ...

. anat_mriqc_pipeline_1.list_to_file (mia_processes.bricks.tools.tools.List_To_File) MIA node ...

During the <mia_processes.bricks.tools.tools.List_To_File object at 0x7f34538e5180> process initialisation, some possible problems were detected:
- inheritance_dict attribute was not found ...

. anat_mriqc_pipeline_1.anat_mni_tpms_pipeline.template_WM (mia_processes.bricks.preprocess.others.processing.Template) MIA node ...

During the <mia_processes.bricks.preprocess.others.processing.Template object at 0x7f34539bb090> process initialisation, some possible problems were detected:
- inheritance_dict attribute was not found ...

. anat_mriqc_pipeline_1.anat_mni_tpms_pipeline.template_CSF (mia_processes.bricks.preprocess.others.processing.Template) MIA node ...

During the <mia_processes.bricks.preprocess.others.processing.Template object at 0x7f3486e750e0> process initialisation, some possible problems were detected:
- inheritance_dict attribute was not found ...

. anat_mriqc_pipeline_1.anat_mni_tpms_pipeline.template_GM (mia_processes.bricks.preprocess.others.processing.Template) MIA node ...

During the <mia_processes.bricks.preprocess.others.processing.Template object at 0x7f3455aa8f90> process initialisation, some possible problems were detected:
- inheritance_dict attribute was not found ...

. anat_mriqc_pipeline_1.anat_mni_tpms_pipeline.applytransforms_WM (mia_processes.bricks.preprocess.ants.processes.ApplyTransforms) MIA node ...

. anat_mriqc_pipeline_1.anat_mni_tpms_pipeline.applytransforms_CSF (mia_processes.bricks.preprocess.ants.processes.ApplyTransforms) MIA node ...

. anat_mriqc_pipeline_1.anat_mni_tpms_pipeline.applytransforms_GM (mia_processes.bricks.preprocess.ants.processes.ApplyTransforms) MIA node ...

. anat_mriqc_pipeline_1.anat_mni_tpms_pipeline.files_to_list (mia_processes.bricks.tools.tools.Files_To_List) MIA node ...

During the <mia_processes.bricks.tools.tools.Files_To_List object at 0x7f3453a83450> process initialisation, some possible problems were detected:
- inheritance_dict attribute was not found ...

. anat_mriqc_pipeline_1.anat_airmask_pipeline.rotationmask (mia_processes.bricks.preprocess.others.processing.RotationMask) MIA node ...

. anat_mriqc_pipeline_1.anat_airmask_pipeline.template (mia_processes.bricks.preprocess.others.processing.Template) MIA node ...

During the <mia_processes.bricks.preprocess.others.processing.Template object at 0x7f34539d9ef0> process initialisation, some possible problems were detected:
- inheritance_dict attribute was not found ...

. anat_mriqc_pipeline_1.anat_airmask_pipeline.applytransforms (mia_processes.bricks.preprocess.ants.processes.ApplyTransforms) MIA node ...

. anat_mriqc_pipeline_1.anat_airmask_pipeline.artifactmask (mia_processes.bricks.preprocess.others.processing.ArtifactMask) MIA node ...

. anat_mriqc_pipeline_1.harmonize (mia_processes.bricks.preprocess.others.processing.Harmonize) MIA node ...

. anat_mriqc_pipeline_1.anatiqms (mia_processes.bricks.reports.processes.AnatIQMs) MIA node ...

Error during initialisation of the "NoName" pipeline ...!
Traceback:
  File "/data/Git_Projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/pipeline_manager_tab.py", line 1464, in initialize
    self.test_init = self.init_pipeline()
  File "/data/Git_Projects/populse_mia/python/populse_mia/user_interface/pipeline_manager/pipeline_manager_tab.py", line 1544, in init_pipeline
    self.workflow = workflow_from_pipeline(
  File "/data/Git_Projects/capsul/capsul/pipeline/pipeline_workflow.py", line 1589, in workflow_from_pipeline
    = workflow_from_pipeline_structure(
  File "/data/Git_Projects/capsul/capsul/pipeline/pipeline_workflow.py", line 1449, in workflow_from_pipeline_structure
    sjob = jobs[process]
KeyError: <mia_processes.pipelines.preprocess.anat_mriqc_pipeline.Anat_mriqc_pipeline object at 0x7f3453c8da40>

On the other hand if we use in step 3:

  1. Right click on the anat_mriqc_pipeline_1 node then select "export all unconnected plugs" The run works fine !

Careful, this pipeline doesn't work with version 2 of the GUI controller, we have to check "version 1 controller" in the Mia preferences.

denisri commented 1 year ago

I think I have fixed it: 502d8bc9df1032ec96b2298e3c48b38fb5ed2921 Can you try it on your side ? I can't actually run the pipeline on my laptop right now because it seems to need Afni, which I have not installed. By the way, this pipeline requires several extra dependencies that I had to install:

pip3 install templateflow
pip3 install statsmodels
pip3 install -U nipype
pip3 install nitime
pip3 install nipy
pip3 install nilearn
pip3 install reportlab

Note that even nipype needs to be upgraded compared to the version in current casa-distro-5.3 containers (otherwise I get an error TypeError: 'NoneType' object is not iterable), and even then, I can't see the pipeline at the expected location "mia_processes-> pipelines-> processes -> Anat_mriqc_pipeline pipeline", so I had to add it manually (using "add process in pypeline" then type the module name). It it a process cache issue anywhere ?

Then templateflow seems to download lots of data during pipeline init, and cache them because it doesn't do it the next times, but I don't know where it stores things.

servoz commented 1 year ago

You remind me of the requirements I have to add in the info (templateflow, etc.). I'll do that right away.

For the Anat_mriqc_pipeline pipeline, I worked on it and it was a bit broken... Now it should work ... but now this pipeline has been greatly simplified and there is no more optional plug!!!!!.

I will test with the Bold_mriqc_pipeline which has not been simplified yet and still contains optional plugs! Thanks

servoz commented 1 year ago

Just tested with Bold_mriqc_pipeline and your last commit in capsul do the job ! Thanks. I think this should also fix few of the issues I've been seeing with creating various pipelines lately... to be seen. Also, I think this also fixes a ticket I opened recently. I'm going to test this right away! I think this ticket can be closed now.