Plans to support AFNI/SUMA?

[dipterix]

Hi, is there any plans to support AFNI/SUMA format?

[dfsp-spirit]

In general I think it would be cool to support some more file formats, as long as they are related to brain surface data.

I don't personally use AFNI/SUMA atm, but if you can link to a document with a spec, I'll check it out. Is there a specific format you're interested in?

[dipterix]

Thanks, it'll be cool to support niml and dset. AFNI/SUMA has some nice surfaces and the atlases and analysis data are often stored in these formats. Lots of people uses afni color maps so cmap would also be a good idea, but trivial.

Tricky point is they uses XML formats instead of binary data. XML is more flexible but might have less fixed patterns.

[dfsp-spirit]

One needs to use an XML library to parse it, and check whether they supply a schema for validation. As long as they do not use the flexibility of XML to have thousands of optional fields and different file format versions I'll give it a go when I find the time. I installed AFNI yesterday and had a short look. I noticed that it installed lots of R packages. I will double-check whether there is already GNU R support for reading the file formats hidden somewhere...

[dipterix]

Thanks. Yes they do, but most of their R scripts won't pass the R checks and even have errors. The main file to look at might be here.

[dfsp-spirit]

I've had a short look at some of the formats. They are not really pure XML. Many of them store data in other formats in inner elements (some even binary data, graphs, or whatever). E.g., if you download the AFNI example data for the course from their website and extract it in your $HOME, you can read a dset file containing a GIFTI image like this:

library(giftii);
ff = readgii(path.expand("~/AFNI_data6/FT_analysis/FT/SUMA/lh.sulc.gii.dset"));

You can see an example of a file containing binary data at ~/AFNI_data6/GroupAna_cases/3.anova/data/G2S01.niml.dset.

The dset format seems to be just a container, it contains a set of elements, that's it. The elements can include data, metadata (like history/log-like stuff) and GUI-specific stuff like visualization settings. I'm not 100% sure, but I got the impression that some fields even reference external files.

As such, I think there is no general way to parse something like a dset file. You have to know what's in there/what you want.

The only general option would be to have a general loader which loads the set of stuff:

library("xml2");
library("readr");
xmltree = read_xml(read_file("path/to/myfile.dset"));

then the user could iterate over the elements and check the name to see what data type is stored in an element. The API could provide parsers for specific element types, which would know how to read the raw data, and transform it into something (e.g., the binary graph data into an instance from some graph package, or a mesh into lists of vertices and faces), then return that.

Of course, one could have a helper function which would return all meshes/images/graphs/whatever datatype instances included in the dset.

Does this seem to make any sense to you?

[dfsp-spirit]

I've had a closer look and the files are not XML, not even the outer structure. The format is some homebrew-stuff which looks roughly similar to various markup languages (HTML/XML/...). It's more or less documented at https://afni.nimh.nih.gov/pub/dist/doc/misc/niml/NIML_base.html.

This means one cannot parse them with an xml lib. That's also the explanation why the code in the code file you linked is so long and ugly.

I'm not blaming the authors, the file formats are from the early 90s and that's how things were done back then. But still I don't feel like parsing that stuff unless I really need to I gotta say.

[dipterix]

You are right. AFNI exists for 30+ years and it's indeed very hard to be backward compatible. Still thanks for the awesome package you developed.

[dfsp-spirit]

I just wanted to say that by now (v0.1.10), there is GIFTI support for some file types in freesurferformats. GIFTI is a very versatile XML format, and as such, the data must be interpreted by the target software. E.g., some software packages seem to store morphometry data together with a surface mesh, while others require 2 different files. A list of dozens of different example GIFTI files from a number of packages is available at https://www.nitrc.org/frs/?group_id=75.

It should be possible in the near future to at least support some types of AFNI data stored in GIFTI files (e.g., surface meshes) in freesurferformats. I will have a look.