Q: plotting [source x RGBA] array on brain surface

[LauraGwilliams]

Hi there,

I am trying to plot an array of RGBA values (shape: [source x RGBA], [5124 x 4]) on a pysurfer brain. In other words, I want to manually specify the RGBA value at each vertex. The brain object can be from MNE-Python or Pysurfer natively, I don't mind. I have tried playing around with the LUT but I think I am making things much more complicated than they need to be.

Do you know of a way I could achieve this? Perhaps one method would be to access the RGBA values of the brain image itself, and simply add my RGBA values on top of it; but I can't find the cortex RGBA values to manipulate, and maybe you know a better method.

It's been suggested to me that @larsoner may be one of the people to ask - I hope you don't mind me flagging you here.

Thank you very much in advance. I really appreciate it! Laura

[mwaskom]

I don’t know off the top of my head how to do this, or even if it is possible. Hopefully Eric knows. If it turns out to be reasonably straightforward, it would be a good thing to add to the example gallery.

[larsoner]

You can more or less translate any VTK example to Mayavi using TVTK. Adapting this one, found by Googling "vtk mesh color per vertex":

https://www.vtk.org/Wiki/VTK/Examples/Cxx/PolyData/TriangleColoredPoints

I made this:

import mne
from mayavi import mlab
import numpy as np
from tvtk.api import tvtk
from tvtk.common import configure_input_data

rr, tris = mne.read_surface(
    mne.datasets.sample.data_path() + '/subjects/fsaverage/surf/lh.pial')
x, y, z = rr.T
alpha = z - z.min()
alpha /= alpha.max()
colors = np.array([np.linspace(0, 1, len(x)), np.zeros(len(x)),
                   np.linspace(1, 0, len(x)), alpha]).T
fig = mlab.figure()
mesh = mlab.pipeline.triangular_mesh_source(
    x, y, z, tris, figure=fig)
mesh.data.point_data.scalars.number_of_components = 4
mesh.data.point_data.scalars = (colors * 255).astype('ubyte')
mapper = tvtk.PolyDataMapper()
configure_input_data(mapper, mesh.data)
actor = tvtk.Actor()
actor.mapper = mapper
fig.scene.add_actor(actor)

I think this could in principle be adapted to add data (e.g., after interpolating the R/G/B values to all surface vertices using the morphing matrix, similar to how we interpolate scalar data to the high-resolution mesh before colormapping). There might be other considerations such as blending with other elements in the scene, but we can figure those out later if need be.

[larsoner]

... oh, and a screenshot after changing the code above to add an alpha that depends on the Z value:

[LauraGwilliams]

This is really helpful, thank you so much!

My only other question is whether it is possible to plot this mesh on top of the grey/black cortex surface? Right now I have something like the screenshot below, which is pretty much what I was after, but I feel like the alpha isn't properly interpretable unless it's covering up the background brain to a greater or lesser extent. Would this be difficult to add in?

Thank you again, Laura

[larsoner]

The easiest way to do that would be to use PySurfer to give you the cortical surface before doing the stuff above:

fig = mlab.figure()
brain = surfer.Brain('fsaverage', 'left', 'lat', figure=fig)
...

There are plenty of ways to configure the view of the cortical surface:

https://pysurfer.github.io/generated/surfer.Brain.html#surfer.Brain

[mwaskom]

PS when modulating the alpha of the overlay it's generally best not to show the curvature on the brain surface or else you get into problems interpreting low alpha over the lighter color vs high alpha over the darker color.

[LauraGwilliams]

ah, perfect! all solved -- thank you both so much for your help!

[larsoner]

@LauraGwilliams do you have time to turn this into a little example for the gallery?

[mwaskom]

It’s worth thinking about whether we want to have examples that primarily involve futzing with mayavi in the gallery, or If there is a better place (I.e. more of a “cookbook”)

[LauraGwilliams]

Sure, I can do that! Where do you want it to go?

[larsoner]

https://github.com/nipy/PySurfer/tree/master/examples

Maybe plot_custom_colors? I assume you had to plot a brain, take your ~5000 values and interpolate them (using PySurfer interpolation matrix) and plot? If so this seems generally pretty useful, and a good showcase of how to use PySurfer's computations for custom viz.

[larsoner]

ping @LauraGwilliams if you have time :)

[LauraGwilliams]

Yes I'm so sorry -- I haven't forgotten. I will try to get to this soon!!