support for surface mesh?

[bendichter]

In the Chang Lab, we often visualize data spatially on the surface of brain. To do this, we take the MRIs of each subject and output surface mesh data, which has two variables, vertices and edges. Vertices gives 3D spatial coordinates and edges indicates the surfaces of a triangle mesh by referencing indices of the vertices (e.g. form a triangle face with vertex 1, 6, and 7).

We separate the mesh into different anatomical regions and save the in independent files so we can display specific regions (e.g. only show the left hemisphere) or apply different visual features to certain regions (e.g. make the temporal lobe red). The spatial coordinates are in the same space - if we plot electrodes and the mesh, the electrodes show up in the correct location on the brain. We visualize data by showing and coloring electrodes, or by creating color gradients on the surface of the brain. Both the surface calculation and of the data visualization are performed by the python library https://github.com/ChangLabUcsf/img_pipe, mostly calling freesurfer for processing and vtk for 3D visualization.

Is there a way to store this type of data in NWB? I see support for 2D images, but I have not seen support for these types of surfaces. This would really have to support external linking as well, otherwise we would have to copy anatomical data many times for a single subject.

Sometimes it is useful for us to store a 2D version of the brain as well, in which case we essentially take a screenshot of the 3D mesh and record the electrode positions in that image. The electrode positions in this new 2D space are different from the positions in the 3D space, so each electrode does not have a single position, but rather 2, one for each "space" or reference brain image. Is there support for having multiple positions for each electrode?

[ajtritt]

@bendichter There is no place for this type of data in the core NWB schema. You will need to create an extension

[bendichter]

I ended up creating this extension:

from pynwb.spec import NWBDatasetSpec, NWBNamespaceBuilder, NWBGroupSpec

ns_path = "img_pipe.namespace.yaml"
ext_source = "img_pipe.extensions.yaml"
surface = NWBGroupSpec(doc='brain cortical surface',
                       datasets=[  # set Faces and Vertices as elements of the Surfaces neurodata_type
                           NWBDatasetSpec(doc='faces for surface', shape=(None, 3),
                                          name='faces', dtype='int'),
                           NWBDatasetSpec(doc='vertices for surface', shape=(None, 3),
                                          name='vertices', dype='float')
                         ],
                       neurodata_type_def='CorticalSurface',
                       neurodata_type_inc='NWBContainer')
ns_builder = NWBNamespaceBuilder('img_pipe extensions', "img_pipe")
ns_builder.add_spec(ext_source, surface)
ns_builder.export(ns_path)