MRG+1: allowing transparency in divergent colormaps

[sbitzer]

When analysing some of my data I came across the issue that I couldn't use a divergent colormap for, e.g., correlations or t-values and at the same time let small values in the middle of the colormap be transparent. So I have adapted Brain.scale_data_colormap such that it can now handle transparency in divergent colormaps properly. Also, I added the option to set the overall alpha-level of the colormap while maintaining transparency of a particular region in the colormap. The usage of these features is demonstrated in examples/plot_resting_correlations.py.

I believe that these changes make it easier to explore suitable visualisations of divergent data and hope that you find them useful!

[sbitzer]

Ok, these changes should implement the discussed functionality. I had some issues with Mayavi/VTK when changing the number of colors in the lookup table. So I now ensure that the new lookup tables have exactly 256 colors (see 59debb4). I'm most uncertain about the compatibility of the newly added arguments in add_data with existing code. Maybe you would like to comment on that. I have not further thought about dissociation of add_data and add_overlay.

[larsoner]

Also, could you rebase? That way the CIs can check everything.

[sbitzer]

I have resolved the few things you discussed except for the log message (see my comment above). The colorbar in your example looks strange, because I used the first color of the second half of the colormap to fill up the center of the colormap. This works nicely with most divergent colormaps as they smoothly transition from one to the other side of the colormap. That's not the case for the colormap in your example. I have now included some code which tries to identify these colormaps and then chooses gray as the fill color in the center for them.

A practical issue as I haven't done that before: I have rebased my branch with nipy/master locally and my state is now:

On branch divtrans
Your branch and 'origin/divtrans' have diverged,
and have 17 and 7 different commits each, respectively.
  (use "git pull" to merge the remote branch into yours)

I can't imagine that merging the old commits is the appropriate thing to do. What is?

[larsoner]

I can't imagine that merging the old commits is the appropriate thing to do. What is?

I would do:

git checkout -b divtrans-bak origin/divtrans  # make a backup
git checkout divtrans
git push origin --force divtrans

The colorbar in your example looks strange, because I used the first color of the second half of the colormap to fill up the center of the colormap. This works nicely with most divergent colormaps as they smoothly transition from one to the other side of the colormap. I have now included some code which tries to identify these colormaps and then chooses gray as the fill color in the center for them.

The problem is actually more general than this. Gray should be used anytime that transparent=True in the [-fmin, fmin] region, as the data values in that span will be gray. (Another way to say this is that the mayavi colorbar function does not implement transparency correctly in this case / wouldn't know what to put underneath the colorbar overlay to make transparency look correct.) Make sense?

Ideally the -fmid, -fmin and fmin, fmid regions with transparent=True would show up on the colorbar as linear mixes between gray and the actual colors, too. But I'm not sure if you can modify the LUT for the colorbar display and the LUT for the data separately, which seems like would be required to get this behavior. (Alternatively, if there is some way to get the colorbar display to actually show transparency, and pick a background color to blend with, all of this would become much simpler.)

[sbitzer]

Yes, I agree that it would make most sense, if the colorbar would somehow reflect transparency properly, but I'm afraid that this is a Mayavi or even VTK issue that I currently don't want to go into. At least for transparent=True I find the workaround of extending the region for the middle color quite intuitive, because it's clear then that the middle color is fully transparent. I have also seen in a talk last week a brain visualisation using this (don't know which software they used, but their colorbar was almost all black with some color gradients at the edges and the brain showed selected blobs). Only for transparent=False with fmin>0 confusion may occur, because you won't be able to see a representation of fmin on the colorbar although you might expect a sharp edge. Hm, to prevent this we could simply choose gray as center fill color whenever transparent=False? I still find it visually more pleasing to fill in the center color from the colormap for transparent=True, though.

[larsoner]

Yes, I agree that it would make most sense, if the colorbar would somehow reflect transparency properly, but I'm afraid that this is a Mayavi or even VTK issue that I currently don't want to go into

It looks like the mayavi.mlab.colorbar has an object property that allows you to choose a color an object to use to choose the colormap. So what we could do is:

1. Create two colormaps: one that is actually used for the data (you have this already) and
2. Another that is that same colormap mixed with a (0.5, 0.5, 0.5) background.
3. Apply the first colormap to the data (you have done this already), and
4. Apply the second colormap to a dummy object (single-triangle mesh at the origin of tiny size)
5. Set the dummy object visibility off
6. Call colormap with object=dummy

It's a bit of a hack but it should work, and I suspect will best represent what's happening with the data. I can give it a try if you're tired of messing with Mayavi :)

I find the workaround of extending the region for the middle color quite intuitive, because it's clear then that the middle color is fully transparent.

Hmmm... I'm not sure I agree that the transparent effect is currently made clear by the colorbar. With transparent=True, those middle colors are alpha=0., i.e., invisible / gray (on a standard brain), but show up as whatever the middle color is on the colorbar.

[sbitzer]

It's a bit of a hack but it should work, and I suspect will best represent what's happening with the data. I can give it a try if you're tired of messing with Mayavi :) It sounds like it could work. Sure, feel free to see how it goes!

I wondered more generally why Mayavi/VTK programmers did not include the possibility of having transparent colorbars and figured out that this is not true: You can switch on proper transparency of the colorbar with

surf = brain.data_dict['lh']['surfaces'][0]
lm = surf.module_manager.scalar_lut_manager
lm.scalar_bar.use_opacity = 1
lm.render()

This will apply transparency with respect to the figure window background (white in my example). For transparent=False this gives a result equivalent to add_overlay:

but with transparent=True and alpha=0.75 I'm not sure whether the result is a good representation of the colors on the brain:

I guess the question is what the background colour should be against which the transparency is plotted. Given that the colours apply to the brain, this should probably, in the ideal case be a texture containing the two colours of the inflated brain (greys in most cases). I wonder whether people would intuitively recognise a simple mixture between the colorbar and a particular grey to represent the transparency on the brain. Perhaps the transparency of the colorbar with the figure background is already a good approximation?

[larsoner]

For transparent=False this gives a result equivalent to add_overlay:

In this case I wouldn't have expected to be able to see any brain, i.e. the -fmin, fmin interval should have been black (or really -fmin, 0 the "just left of colormap midpoint" color and 0, fmin to be "just right of colormap midpoint, which in this case are both black).

but with transparent=True and alpha=0.75 I'm not sure whether the result is a good representation of the colors on the brain:

It looks better than before, but I agree still not quite right.

I guess the question is what the background colour should be against which the transparency is plotted. Given that the colours apply to the brain, this should probably, in the ideal case be a texture containing the two colours of the inflated brain (greys in most cases).

Yes this would be ideal, but I think just using middle gray (0.5, 0.5, 0.5) would already take care of 95% plus of use cases, and easy to describe in the docs (i.e., say we blend with this).

I wonder whether people would intuitively recognise a simple mixture between the colorbar and a particular grey to represent the transparency on the brain.

I think they would. In every example you've posted so far, and even in the examples we have online (even with the bone-colored brain) I think it would look okay, and probably more representative than anything else so far.

Perhaps the transparency of the colorbar with the figure background is already a good approximation?

I'd say it's better than what we have in master but not as good as mixing with gray (by default) would be for our specific use case. If people like mixing with the background instead of gray we could add an argument to scale_data_colormap to also allow that mode. But maybe we won't need it once we see the gray solution.

Did you have a chance to try my suggestion of using a dummy object for comparison? (Or alternatively, to see if there is a way to set the background color of the colorbar in addition to setting the opacity?)

[sbitzer]

I did not find an option to set the background only of the colorbar itself, but this is possible:

Not sure whether I like the box or whether the grey is better than the white. I did not try your dummy object approach.

[larsoner]

I think the box looks a bit worse than no box.

And can you look into this problem?

For transparent=False this gives a result equivalent to add_overlay:

In this case I wouldn't have expected to be able to see any brain, i.e. the -fmin, fmin interval should have been black (or really -fmin, 0 the "just left of colormap midpoint" color and 0, fmin to be "just right of colormap midpoint, which in this case are both black).

[sbitzer]

Ah right, I read this over. I interpreted fmin as the first value that you want to see displayed while transparent only controls whether there is a transparency gradient between fmin and fmid, but you're right, the add_data-interface defines this differently for sequential colormaps. I can fix that. Do you perhaps know how I can change the call

pipe = threshold_filter(pipe, low=thresh, figure=self._f)

to one in which all values except those in [center - thresh, center + thresh] are kept?

[larsoner]

Do we even need the thresholding filter? Maybe setting the limits properly (and scaling our colorbar properly) will already work.

[sbitzer]

I don't see how setting the limits and scaling the colorbar can implement the selective clipping of values at the centre of the colormap. The colormap used by Mayavi will always linearly interpolate colours given scalar values so you cannot simply skip values that lie in the centre. As I understood it, the thresh argument allows that kind of clipping of 'small' values independent of the transparency of the colormap and I had previously interpreted fmin to behave like that. As you pointed out, however, fmin is supposed to be the value from which all equal or smaller magnitude values are mapped to the leftmost colour in sequential colormaps or the centre colour(s) in divergent colormaps. So I don't see how the functionality of thresh could be implemented with colormap properties. Do I miss something?

The simplest solution would be to not adapt the behaviour of thresh for divergent colormaps, noting that with transparent=True a similar visualisation can be achieved.

[larsoner]

I don't see how setting the limits and scaling the colorbar can implement the selective clipping of values at the centre of the colormap.

My point is just that I don't think these values should be / need to be clipped, at the very least in the transparent=True case and maybe also in the transparent=False case.

[sbitzer]

Ok, I agree and, therefore, didn't touch the thresh functionality. I have corrected the behaviour of fmin though, updated the log message and introduced transparency of the colorbar with respect to the figure background in the latest commits.

[larsoner]

Looks like there are some PEP8 errors:

https://travis-ci.org/nipy/PySurfer/jobs/284137743#L1221

Can you rebase so that CircleCI runs the doc build properly?

[larsoner]

See https://github.com/sbitzer/PySurfer/pull/1 for non-hack gray fix

[larsoner]

Looks nice:

https://27-1551431-gh.circle-artifacts.com/0/tmp/circle-artifacts.YB9nYb3/html/auto_examples/plot_meg_inverse_solution.html

But something strange has happened here:

https://27-1551431-gh.circle-artifacts.com/0/tmp/circle-artifacts.YB9nYb3/html/auto_examples/plot_vector_meg_inverse_solution.html#sphx-glr-auto-examples-plot-vector-meg-inverse-solution-py

(it's probably a bug in the code I wrote, feel free to look or I can do it)

[sbitzer]

Nice!

(it's probably a bug in the code I wrote, feel free to look or I can do it)

As I wouldn't know where to start, except for the obvious that it has something to do with the computation of the background color, I would appreciate, if you could check it out first.

[larsoner]

I would appreciate, if you could check it out first.

I should be able to take a look tomorrow

[larsoner]

Fix up again at https://github.com/sbitzer/PySurfer/pull/1

[sbitzer]

Great, fine by me! :+1:

[larsoner]

Anybody else want to look? If not I'll go ahead and merge.

[larsoner]

Thanks @sbitzer !