Limitations of CIFTI support?

[mgxd]

Hi Anderson,

Our lab has been processing our data with the HCPPipeline, and are now looking to run PALM on the CIFTI output. We are following the documentation for Case 1, but have run into the follow error:

octave: X11 DISPLAY environment variable not set
octave: disabling GUI features
=======================================================================
             ___         ___                         ___
            /  /\       /  /\                       /__/\
           /  /::\     /  /::\                     |  |::\
          /  /:/\:\   /  /:/\:\    ___     ___     |  |:|:\
         /  /:/~/:/  /  /:/~/::\  /__/\   /  /\  __|__|:|\:\
        /__/:/ /:/  /__/:/ /:/\:\ \  \:\ /  /:/ /__/::::| \:\
        \  \:\/:/   \  \:\/:/__\/  \  \:\  /:/  \  \:\~~\__\/
         \  \::/     \  \::/        \  \:\/:/    \  \:\
          \  \:\      \  \:\         \  \::/      \  \:\
           \  \:\      \  \:\         \__\/        \  \:\
            \__\/       \__\/                       \__\/

=======================================================================
                 Permutation Analysis of Linear Models
=======================================================================
Running PALM alpha111 using Octave 4.0.0 with the following options:
-i /input/combined133_gambling_reward_gtr_loss.dscalar.nii
-d /input/design.mat
-t /input/design.con
-n 5
Found HCP Workbench executable in /opt/workbench/bin_linux64/wb_command
Reading input 1/1: /input/combined133_gambling_reward_gtr_loss.dscalar.nii

While running:
/opt/workbench/bin_linux64/../exe_linux64/wb_command -cifti-convert -to-gifti-ext /input/combined133_gambling_reward_gtr_loss.dscalar.nii palm_0UAqFz.gii

ERROR: NAME OF FILE: palm_0UAqFz.gii

Output stream for external file reports its status as bad.

warning: [XML] Tag <![CDATA[ opened but not closed.
warning: [XML] Tag <![CDATA[ opened but not closed.
warning: [XML] Tag <![CDATA[ opened but not closed.
warning: [XML] Tag <![CDATA[ opened but not closed.
warning: [XML] Tag <![CDATA[ opened but not closed.

Our input image is a CIFTI dscalar composed of multiple subjects particular COPE - this file includes both surface and volumetric data. I was hoping this would play nice with PALM, but based off of https://github.com/andersonwinkler/PALM/blob/8218b71c4019a75610272ea9800685cd486bdef1/palm_ciftiread.m#L2-L3 it looks like that is not the case?

(In the meantime, I am going to follow the recommendations in Case 2 of the FSL documentation)

[andersonwinkler]

Hi Mathias,

Correct, support for CIFTI is limited. In fact, it will not read a surface that may be embedded in the file, only the scalars associated with that surface. To pass the surface geometry to PALM (for example, if you are using spatial statistics, such as cluster extent or TFCE), you'd use the option "-s", with a separate file containing that surface then.

The "surface only" mentioned in the code refers to the fact that only the surface part of a CIFTI is supported, as opposed to volume + surface data that this file format can hold. The "surface only" doesn't refer to the surface geometry, but to surface-based scalar data.

The error message above is coming from wb_command, which is the backend for CIFTI I/O. It may be that the contents of the file are not adequate for the extension that PALM is giving to it. This is corroborated by the various unclosed CDATA fields (which is also unusual on its own right if the file is valid). Please, consider stripping the surface off of the file, leaving only the scalars, and try again. If it still fails, please feel free to send me the file and I'll have a look.

All the best,

Anderson

[mgxd]

Hi Anderson,

Thanks for the detailed reply! Re: the wb_command error - I failed to mentioned that I had previously ran it standalone, and it produced 2 files: palm_0UAqFz.gii and palm_0UAqFz.gii.data (I believe these correspond to the surface / everything else of the file). The gii.data file produced was invalid, which is probably the reason for CDATA fields warning.

I've since implemented the processing outlined in Case 2 of the documentation, and it seems to process the subcortical nii just fine. But I'm encountering a new issue when passing in the surface file (-s)

Running PALM alpha111 using Octave 4.0.0 with the following options:
-i /input/combined_133_gambling_reward_gtr_loss_R.func.gii
-d /design/design.mat
-t /design/design.con
-T
-logp
-s /surf/R_midthickness.surf.gii /input/combined_133_gambling_reward_gtr_loss_R_area.func.gii
-o /input/palm/palm
Found HCP Workbench executable in /opt/workbench/bin_linux64/wb_command
Loading surface 1/1: /surf/R_midthickness.surf.gii
Error using read_gifti_file_standalone>gifti_Data (/PALM/fileio/@gifti/private/read_gifti_file_standalone.m:167)
/PALM/fileio/@gifti/private/zstream.mex: failed to load: /PALM/fileio/@gifti/private/zstream.mex: invalid ELF header

Error in read_gifti_file_standalone>gifti_DataArray (/PALM/fileio/@gifti/private/read_gifti_file_standalone.m:125->read_gifti_file_standalone>gifti_Data)
Error in read_gifti_file_standalone (/PALM/fileio/@gifti/private/read_gifti_file_standalone.m:48->read_gifti_file_standalone>gifti_DataArray)
Error in gifti (/PALM/fileio/@gifti/gifti.m:71->read_gifti_file_standalone)
Error in palm_miscread (/PALM/palm_miscread.m:293->gifti)
Error in palm_takeargs (/PALM/palm_takeargs.m:1567->palm_miscread)
Error in palm_core (/PALM/palm_core.m:33->palm_takeargs)
Error in palm (/PALM/palm.m:81->palm_core)

The surface GIFTI has 2 data arrays, with the following metadata: darray 0

{'AnatomicalStructurePrimary': 'CortexLeft',
 'AnatomicalStructureSecondary': 'MidThickness',
 'Caret-Version': '5.65',
 'Date': '2012-10-01T21:47:15',
 'GeometricType': 'Anatomical',
 'Name': '#1',
 'UniqueID': '{cd0ccf14-19cb-4be9-955c-8b83a102116b}',
 'configuration_id': 'Unknown',
 'topo_file': 'Conte69.L.32k_fs_LR.topo.gii'}

darray 1

{'Caret-Version': '5.65',
 'Date': '2012-10-01T21:47:15',
 'Name': '#2',
 'TopologicalType': 'Closed',
 'UniqueID': '{e2ef4622-4b06-4612-9159-1d0b9802684a}'}

Any ideas?

• Mathias

[andersonwinkler]

Hi Mathias,

It should definitely work, and I suspect there is something uncommon (not necessarily wrong) with your files. May I ask you to send them to me? I'd like to see these three:

• combined133_gambling_reward_gtr_loss.dscalar.nii
• R_midthickness.surf.gii
• combined_133_gambling_reward_gtr_loss_R_area.func.gii

My email is like my username here, served by gmail. Please compress them before sending, or send via Dropbox or Globus.

Thanks!

All the best,

Anderson

[gllmflndn]

Hello,

I would also be curious to have access to the files as it should be possible to read all of the content from the CIfTI file using @nifti and @gifti.

I am also a bit surprised by this error message: /PALM/fileio/@gifti/private/zstream.mex: failed to load: /PALM/fileio/@gifti/private/zstream.mex: invalid ELF header Something went wrong during compilation.

[mgxd]

@gllmflndn I think the problem was related to this issue from the FSL mailing list - select functions needed to be recompiled for the system I was running in. After this, I was able to successfully run palm.

@andersonwinkler Is there any documentation on building palm (+ fileio functions) from source to avoid this problem in the future?

[andersonwinkler]

Hi Mathias,

There is, but it doesn't cover this particular issue. I'll include that (most likely have a shell script inside that private dir, as it's already the case with file_array).

Thanks @gllmflndn also for the paying close attention and being ready to assist.

Cheers!

Anderson

[gllmflndn]

OK, I just wondered whether we could use @nifti/@gifti instead of having to do a system call to wb_command.

The Octave ABI is frequently changing between versions so I would recommend to always compile the MEX files as part of the installation. There could be a single Makefile at the top of PALM.

[andersonwinkler]

Hi Guillaume,

Makes complete sense. Back then there wasn't CIFTI support. Do the current @nifti and @gifti read CIFTI directly? If yes, then sure, happy to remove the external dependency.

Thanks for all your support!

Anderson

[gllmflndn]

Hi @andersonwinkler ,

Possibly in part. I could try to have a closer look. @nicholst implied that I might be underestimating how much work would be required though. In #13, you also mention FreeSurfer: the latest version of @gifti supports a few of its file formats so it might be something to consider. It now also allows to read MZ3 files.

[andersonwinkler]

Hi Guillaume,

That is great, can give it a try, sure. This weekend I updated with the new i/o files that come with the latest SPM. For CIFTI, I started some two years ago but gave up after a few attempts. You are better positioned than I to have this done, though, because of similarities with GIFTI and the need to parse XML, for which you've a good tool developed for, and which (I believe) doesn't require Java.

Complete compatibility with Octave may be difficult, though, particularly for the case of the (ideal) implementation of memory mapped files, which are be useful for dense connectivity maps.

Cheers,

Anderson

[nicholst]

FWIW, I think the place to start with CIFTI is the code in the FieldTrip toolbox. That allegedly has excellent CIFTI support but is unusable by all but FieldTrip users as it always interpolates the mesh to the standard FieldTrip mesh. See https://wiki.humanconnectome.org/display/PublicData/HCP+Users+FAQ#HCPUsersFAQ-2.HowdoyougetCIFTIfilesintoMATLAB ?

However, if the base CIFTI i/o before interpolation could be identified in FieldTrip code, that could be a really great starting point.

[andersonwinkler]

I don't know now but some 3 years ago the FieldTrip code was very different than that of @gifti but I think it had memory mapping. Maybe some of the strategies used in FT could be ported to it...

In any case, the format is indeed quite tough, not trivial at all. Sometimes it feels it'd be easier to simply start all over from scratch, or maybe use the hdf5 existing infrastructure...

[nicholst]

I can only offer this in response: https://xkcd.com/927/

[andersonwinkler]

sad but true... ideally CIFTI (the 15th standard) would have been based on HDF5, or maybe still something else, but simpler, not mixing technologies. Anyway, I guess now we have to live with it and find ways...

[gllmflndn]

Hi Anderson,

There will be another update of SPM12 in the next couple of weeks and I would strongly recommend that you update @xmltree, @nifti, @gifti and @file_array again at that stage.

Concerning CIFTI support, I had forgotten how the specification looks like... I am not sure to understand your concern about memory mapping: @file_array is an implementation of memory mapping. Otherwise MATLAB has memmapfile and there is work in progress to have it implemented in Octave too. Does PALM work with any type of CIFTI files or mainly with .dscalar.nii only? I am asking to know whether it would be possible to implement a subset of CIFTI that would already cover most of your needs. Could I have access to an example of such file and what the output from wb_command looks like?

[andersonwinkler]

Thanks Guillaume, will do that. Please let me know when it becomes available. Thanks thanks! Anderson

[andersonwinkler]

Does PALM work with any type of CIFTI files or mainly with .dscalar.nii only? I am asking to know whether it would be possible to implement a subset of CIFTI that would already cover most of your needs. Could I have access to an example of such file and what the output from wb_command looks like?

Not any type unfortunately. It works with dtseries, ptseries, dscalar and pscalar. Since wb_command is the backend, updates on it may allow other types, although not always other types make sense for PALM. What really matters are the 4th dimension (over which permutations happen), and the surface geometry (for cluster statistics and TFCE).