CVR pipeline issue with iteration

[manuegrx]

I would like to iterate on several subjects the CVR pipeline and I have some issues.

My first attempt was to test the via input filter brick process way

But an issue arises:

raise TraitError(
traits.trait_errors.TraitError: The 'func_file' trait of a Make_CVR_reg_physio instance must be a pathlike object or string representing a file or a _Undefined or None, but a value of [<undefined>] <class 'traits.trait_list_object.TraitListObject'> was specified.

The func_files parameter is link to the func_file parameter of the Make_CVR_reg_phyiso brick and to the func_files parameter of the Bold_spatial_preprocessing1 brick.
In the first case, the type is the following:

  self.add_trait(
            "func_file",
            Either(
                ImageFileSPM(),
                Undefined,
                copyfile=False,
                output=False,
                optional=False,
                desc=func_file_desc,
            ),
        )

and for the Bold_spatial_preprocessing1 the type is :

self.add_trait(
            "in_files",
            InputMultiPath(
                Either(ImageFileSPM(), List(ImageFileSPM()), Undefined),
                copyfile=True,
                output=False,
                optional=False,
                desc=in_files_desc,
            ),
        )

I suppose that the problem is that in one case it could be a list and in the other not.
How could I manage this issue ? I am not sure if the best way is to look how to modify the pipeline or to look how to change the way the iteration is done

[manuegrx]

I also test an other way to do the iteration, using a regular iterative way using the Files_To_List brick and it is not working either.

We can create the iterative pipeline and launch it but it fails with same issue:

error in setting parameters of process Make_CVR_reg_physio, with dict: {'func_file': ['/home/gourieuxe/Documents/MIA_projects/projetVaso_CVR_assemblynet_2/data/raw_data/sub-002-ses-01--0-func-sub-002_ses-01_task-cvr240dyn_run-12_bold-.nii'], 'cvr_reg': '/home/gourieuxe/Documents/MIA_projects/projetVaso_CVR_assemblynet_2/data/derived_data/sub-002_data/CVR_physio_reg.mat', 'output_directory': '/home/gourieuxe/Documents/MIA_projects/projetVaso_CVR_assemblynet_2/data/derived_data'}

[....]
    raise TraitError(

traits.trait_errors.TraitError: The 'func_file' trait of a Make_CVR_reg_physio instance must be a pathlike object or string representing a file or a _Undefined or None, but a value of ['/home/gourieuxe/Documents/MIA_projects/projetVaso_CVR_assemblynet_2/data/raw_data/sub-002-ses-01--0-func-sub-002_ses-01_task-cvr240dyn_run-12_bold-.nii'] <class 'list'> was specified.

I also try to not use Files_to_List for func_files but in this case we cannot select the functional data with "Filter"

[servoz]

very interesting!

[manuegrx]

I believe I've figured out a way to work around the issue but it is not a "deep correction" (it is not at the level of populse or capsul but in each pipelinel) so I do not know if it is the ideal solution.

My idea is to add in the main pipeline (here CVR pipeline) a brick when an input is used for several bricks and that the brick inputs are not the same type.

For example, for the CVR pipeline I create a brick (InputMultiPath_to_File) to "convert" the type InputMultiPath(Either(ImageFileSPM(), List(ImageFileSPM()), Undefined) to a file ImageFileSPM(output=True, desc=file_desc).

This brick can be added to the main pipeline before the brick Make_Cvr_Reg_Physio and then the type of the main input "func_files" is InputMultiPath(Either(ImageFileSPM(), List(ImageFileSPM()), Undefined)  for both 1_spatial_preprocessing and InputMultiPath_to_File bricks :

Then we can iterate the CVR pipeline as explain is the doc using a regular iterative way using the Files_To_List brick and it is working this time (it is also working with the brick ReduceNode from capsul):

However, the pipeline can not be correctly initialized using the input filter brick process way to iterate. There is no more error (as before) but the pipeline is not initialized correctly and for now I do not understand why...

(The proposed solution is in the branch https://github.com/populse/mia_processes/tree/85-cvr-pipeline-issue-with-iteration )

[servoz]

great!

However, I think this patch could also work with the existing List_To_File brick. While we wait for a real fix, I think it might do the job (the user just needs to modify the existing pipeline with the List_To_File brick, then save the new pipeline to their personal library).

I'm very busy at the moment and doubt I'll have time to look at this in detail now.

[manuegrx]

Indeed it is also working with the existing List_To_File brick if you used the pipeline only for iteration but not if you want to use it for only one subject.

With the InputMultiPath_to_File brick it works for iteration and for one subject, so it could be added to the main CVR pipeline in the main branch while we wait for a potential real fix. If it is okay for you I can do it (I can maybe change the name of the brick ...)

[servoz]

I think that if the user wants to do the calculation on a single subject, it's enough to use the pipeline without changing anything.

[manuegrx]

Okay ! I'm leaving this issue open to not forgot to fix the real issue.

To summarize, for now, if you want to use iteration with the CVR pipeline, follow theses steps:

• Open the C02_inalation pipeline and click to Edit sub-pipeline

• Add the brick List_To_File and save the new pipeline in your personal library

• Iterate this new CVR pipeline as explain is the doc using a regular iterative way using the Files_To_List brick.

[servoz]

Great!

I think it's best to avoid putting patches in the master. We (I?) need to take a serious look at the issue you've pointed out and fix it properly and permanently (for all 3 iteration modes).

This pipeline has not been tested in iteration mode. There was already a lot of work to be done to make it work properly on a single subject with reproducible results compared with what was obtained with the Amigo software.

I think we're going to discover other problems in iteration mode (the opposite would be really surprising!)... The first that springs to mind is certainly the patient reference in the report, which could be the one used for the first subject (to be checked, but it's quite possible!).

So I think this ticket could be used to list all the issues in iteration mode for the CVR pipeline, and why not fix them!

[manuegrx]

I spotted at least one other issue with the subject information.

In my case, I don't want to fill the patient_info parameter for each subject, I want to used the information already in the database (as planned in the code).

It is correctly working for the first subject of the iteration but not for the next ones. It is because the parameter self.patient_info in the class ReportCO2inhalCvr is not reset after the first subject. So for example at line 1395 the value of self.patient_info["PatientRef"] is not None or Undefined but it is already the value of the first subject and so it is not updated for the second subject.

For now, I don't have any leads

[servoz]

So the first 2 issues we see with iterating the CVR pipeline are : -1TraitError with the func_file of the Make_CVR_reg_physio instance (must be a pathlike object and we observe an object of type list). -2 The patient reference in the reports remains the same as that used for the first subject in all reports.