[BUG]: Extra dimension added to parcellation after warping to another template space via ANTs

[LeSasse]

Is there an existing issue for this?

• [X] I have searched the existing issues

Current Behavior

When I apply a mask in FC extraction using a YAML pipeline it fails.

Expected Behavior

I would like it to apply the mask correctly and not fail.

Steps To Reproduce

1. See environment below.
2. See the Yaml here:

   
   workdir: /tmp

datagrabber: kind: DataladAOMICID1000

preprocess: kind: Smoothing using: "nilearn" "on": "BOLD" smoothing_params: fwhm: 10

storage: kind: HDF5FeatureStorage uri: data/aomic_id1000.hdf5

markers:

• name: parccortical-Schaefer100x17FSLMNI_marker-empiricalFC kind: FunctionalConnectivityParcels parcellation: Schaefer100x17 cor_method: correlation cor_method_params: empirical: true masks:
  • inherit

queue: jobname: smoothing_test kind: HTCondor collect: on_success_only env: kind: venv name: env mem: 20G verbose: 8 pre_run: | source /data/group/appliedml/tools/ants_2.5.0/ants.sh

### Environment

```markdown
❱ junifer wtf
junifer:
  version: 0.0.4.dev822
python:
  version: 3.10.10
  implementation: CPython
dependencies:
  click: 8.1.7
  numpy: 1.26.4
  scipy: 1.11.4
  datalad: 0.19.6
  pandas: 2.1.4
  nibabel: 5.2.1
  nilearn: 0.10.2
  sqlalchemy: 2.0.28
  ruamel.yaml: 0.17.21
  httpx: 0.26.0
  tqdm: 4.66.1
  templateflow: 24.2.0
  looseversion: None
system:
  platform: Linux-6.6.13+bpo-amd64-x86_64-with-glibc2.36
environment:
  LC_CTYPE: en_US.UTF-8
  PATH: /home/lsasse/miniconda3/bin:/home/lsasse/miniconda3/condabin:/home/lsasse/bin:/home/lsasse/.dotfiles/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin:/usr/local/games:/usr/games

Relevant log output

It results in the following error:

[DEBUG] Done dropping for Dataset(/tmp/tmplwwl0ock/datadir) 
[DEBUG] Finished dropping datasets at Dataset(/tmp/tmplwwl0ock/datadir) 
Traceback (most recent call last):
  File "/data/project/identification_prediction/projects/test_junifer_smoothing/env/bin/junifer", line 8, in <module>
    sys.exit(cli())
  File "/data/project/identification_prediction/projects/test_junifer_smoothing/env/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/data/project/identification_prediction/projects/test_junifer_smoothing/env/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/data/project/identification_prediction/projects/test_junifer_smoothing/env/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/data/project/identification_prediction/projects/test_junifer_smoothing/env/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/data/project/identification_prediction/projects/test_junifer_smoothing/env/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/data/project/identification_prediction/projects/test_junifer_smoothing/env/lib/python3.10/site-packages/junifer/api/cli.py", line 224, in run
    api_run(
  File "/data/project/identification_prediction/projects/test_junifer_smoothing/env/lib/python3.10/site-packages/junifer/api/functions.py", line 174, in run
    mc.fit(datagrabber_object[t_element])
  File "/data/project/identification_prediction/projects/test_junifer_smoothing/env/lib/python3.10/site-packages/junifer/markers/collection.py", line 101, in fit
    m_value = marker.fit_transform(data, storage=self._storage)
  File "/data/project/identification_prediction/projects/test_junifer_smoothing/env/lib/python3.10/site-packages/junifer/pipeline/pipeline_step_mixin.py", line 229, in fit_transform
    return self._fit_transform(input=input, **kwargs)
  File "/data/project/identification_prediction/projects/test_junifer_smoothing/env/lib/python3.10/site-packages/junifer/markers/base.py", line 201, in _fit_transform
    t_out = self.compute(input=t_input, extra_input=extra_input)
  File "/data/project/identification_prediction/projects/test_junifer_smoothing/env/lib/python3.10/site-packages/junifer/markers/functional_connectivity/functional_connectivity_base.py", line 138, in compute
    aggregation = self.aggregate(input, extra_input=extra_input)
  File "/data/project/identification_prediction/projects/test_junifer_smoothing/env/lib/python3.10/site-packages/junifer/markers/functional_connectivity/functional_connectivity_parcels.py", line 102, in aggregate
    return parcel_aggregation.compute(input, extra_input=extra_input)
  File "/data/project/identification_prediction/projects/test_junifer_smoothing/env/lib/python3.10/site-packages/junifer/markers/parcel_aggregation.py", line 191, in compute
    parcellation_bin = math_img(
  File "/data/project/identification_prediction/projects/test_junifer_smoothing/env/lib/python3.10/site-packages/nilearn/image/image.py", line 1059, in math_img
    result = eval(formula, data_dict)
  File "<string>", line 1, in <module>
ValueError: ("Input formula couldn't be processed, you provided 'np.logical_and(img, mask)',", 'operands could not be broadcast together with shapes (60,77,65,1) (60,77,65) ')

### Anything else?

I guess it is missing np.squeeze() somewhere.

[LeSasse]

When I remove

    masks:
      - inherit

from the marker it works.

[LeSasse]

I did not actually define a mask in the preprocessor.

[LeSasse]

I have re-opened it, because the AOMIC ID1000 datagrabber does have BOLD_masks that I should be able to inherit in the marker extraction. But I think the preprocessor for Smoothing from #161 is messing with it.

[synchon]

@LeSasse I believe I've found the bug and it's that when we warp parcellation via ANTs, it somehow adds an extra axis when loading via nibabel. If you try with a parcellation without the need to warp, it'll be ok. So, I've added an explicit np.squeeze in nilearn's math_img. Check this branch out for your run and let me know if it solves it and you get correct results.

[github-actions[bot]]

PR Preview Action v1.4.7 :---: Preview removed because the pull request was closed. 2024-04-10 17:09 UTC

[LeSasse]

Works like a charm!

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (0fde637) to head (c0c893b).

Additional details and impacted files

[](https://app.codecov.io/gh/juaml/junifer/pull/324?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml) ```diff @@ Coverage Diff @@ ## main #324 +/- ## ========================================= Coverage 100.00% 100.00% ========================================= Files 1 1 Lines 1 1 ========================================= Hits 1 1 ``` | [Flag](https://app.codecov.io/gh/juaml/junifer/pull/324/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml) | Coverage Δ | | |---|---|---| | [docs](https://app.codecov.io/gh/juaml/junifer/pull/324/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml) | `100.00% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=juaml#carryforward-flags-in-the-pull-request-comment) to find out more.