recommendation for circular colormap?

nschloe commented 9 years ago

I need to visualize data that fits the circular color space, much like the wind directions in this talk. (My use case is the complex argument, i.e., the angle, in a complex-valued function.) Right now, I'm using the hsv color map aka the Comic Sans of circular color map.

Now that matplotlib will have four awesome colormaps for linear space, is there a recommendation you can give for circular color maps as well?

pelson commented 8 years ago

Great question! I'm also interested in this. Any thoughts @stefanv / @njsmith / @ngoldbaum?

ngoldbaum commented 8 years ago

One could construct something in viscm, although it would require a modification to allow the creation of spline paths that go vertically upward in the colorspace and then back down again. Right now (as far as I could tell when I was messing with it) it allows spline paths that start at the bottom of the colorspace and end at the top.

njsmith commented 8 years ago

That talk gives an example of how to do this in a few lines of code... I'd start by reconstructing that. It really isn't complicated :-) Basically, pick your favorite circle in CAM02-UCS space, and convert its points to rgb. (CAM02-UCS space is the volume shown in the lower left of the viscm analysis window.) I wouldn't try using the viscm tool for this; the point of the viscm tool is that for drawing arbitrary curves you really want a GUI since the alternative is to try and write some formula by hand. But for a circle the formula is very easy :-)

Some further elaborations to consider:

for an isoluminant circular colormap, write a function that takes the brightness component as an argument, and then automatically finds the largest radius colormap that can be made with that luminance. (Basically the algorithm is just binary search: pick a radius, generate the points, convert to sRGB, and if all the values are between 0 and 1, then this circle fit; try making radius larger. If any values are less than 0 or greater than 1, then this circle is too big; try making radius smaller.) For the talk we wanted simplicity so we just hard-coded a brightness and radius setting that worked, but this would be a better way to do it.
I guess the only way to make a colorblind friendly circular colormap is to use a vertical circle that keeps a* fixed while varying brightness (J) and b. Would be interesting to explore what such circles look like.
maybe a perfectly isoluminant circle is not the best; maybe a slightly tilted circle with some brightness variation is even better (at least for plots where you don't want to keep the brightness channel for another variable, like we did in that wind plot where darker = stronger winds). Could be interesting to experiment with.

The hardest question is what to call it/them ;-)

mycarta commented 8 years ago

Peter Kovesi has some excellent recommendations in this paper: http://arxiv.org/abs/1509.03700 I am slowly (snail pace) working towards building his entire tool in Python, I know he's been porting it to Julia but I doubt he's published it yet. You could use the Matlab tool http://www.peterkovesi.com/matlabfns/index.html#colour to make a circular colormap, then import it in Matplotlib, I think if properly referenced it'd be fine. Alternatively, you could desing an isoluminant circular colormap in HSL colorspace. Set L to constant, and linearly interpolate between yellow-magenta-blue-green-yellow

njsmith commented 8 years ago

(Sorry I didn't reply earlier... Apparently I somehow had github notifications turned off for this repo so I never saw it :-()

njsmith commented 8 years ago

Alternatively, you could desing an isoluminant circular colormap in HSL colorspace. Set L to constant, and linearly interpolate between yellow-magenta-blue-green-yellow

This is basically the same idea as the example given in the talk, except that the example in the talk should produce much better results :-). HSV and HSL don't even attempt to use real color science: they're designed to be cheap hacks for applications where you can't afford the cpu time of even attempting to be accurate. (I mean this as, that is literally why they were invented historically.) If you're using an Apple II then I recommend them, but otherwise...

njsmith commented 8 years ago

I hadn't seen Peter Kovesi's work before -- thanks! I just read the paper and it's really excellent; I learned quite a bit.

Returning to the topic of this thread, I'd certainly suggest anyone considering designing new circular colormaps to read section 4.6 of that paper (well, and the rest of it too ;-)).

nschloe commented 8 years ago

If anyone is able to recreate the color maps in Figure 14 of Peter Kovesi's article in any machine-readable format, that'd be great.

mycarta commented 8 years ago

Slowly chipping away at this, on and off. So far I've only been able to write an introductory article about why we need a new type of circular map at all (such as Peter's): https://mycarta.wordpress.com/2015/06/16/reinventing-the-color-wheel-part-1/ It is just after this that I discovered Nathaniel's and Stéfan's talk and tool, and thought it could be modified to make a colormap like Peter's, ut I've been distracted by trying to replicate Peter's equalization tool. I have a bunch of notes and ideas, I am open to collaborating on it, if anyone is interested.

mycarta commented 8 years ago

HSV and HSL don't even attempt to use real color science: they're designed to be cheap hacks for applications where you can't afford the cpu time of even attempting to be accurate.

I would have to agree on HSL and HSV, should have said LCH, sorry.

In Matlab I got decent results for circular colormap in Lch/Lab. I've tried in scikit-image, this is the best I got, and it was not good enough: the main colors are not opposed to one another in the compass positions NEWS as Peter suggess, but that could've been solved by changing the input angles: The real limitation is that there's a great deal of restriction in how large a Chroma circle you create without getting into non-gamut colors.

I tried also using the values for isoluminant colormap published in Kindleman's paper. Kindlmann, E et al. (2002). Face-based Luminance Matching for Perceptual Colour map Generation - Proceedings of Proceedings of the IEEE conference on Visualization, 299-306

The notebook has been sitting on my Mac at home for a while, I just uploaded it on GitHub here.

Not bad, but still not quite there. I really want to try using CAM02-UCS and your tool Nathaniel, to replicate Peter Kovesi's.

njsmith commented 8 years ago

That's weird that you're getting a different Lab / sRGB transform in Matlab and skimage... One of them is surely buggy, then. colorspacious also has an implementation of Lab that you could try too, I guess (and I'm fairly confident that it works correctly). It also does LCh to sRGB in a single step, so that's handy ;-)

I'd also be interested to see how your results compare using Lab versus CAM02-UCS. Theory says that the latter should be more perceptually smooth and better behaved, but it doesn't tell us whether the difference should be large enough to matter :-)

peterkovesi commented 8 years ago

Hi, Matteo has just alerted me to this thread.

I have a set of my colour maps already prepared in a wide range of formats, including CSV at http://peterkovesi.com/projects/colourmaps/index.html so from this you should be able to generate some circular colour maps to use for your purposes.

I have recently ported my Matalb code to Julia https://github.com/peterkovesi/PerceptualColourMaps.jl if time permits I may work on a Python port.

I have just had a very superficial look at CAM02-UCS. I have not checked properly but I presume it is based on the 10 degree standard observer. If this is the case it may not offer much advantage because we are typically interested in perceptual differences over much finer spatial frequencies. However it may be useful for low lightness contrast or isoluminant colour maps. it could be incorporated in the code as an alternate colour difference measure, probably something interesting to try.

Cheers Peter

njsmith commented 8 years ago

@peterkovesi: Yeah, CAM02-UCS should be thought of as being a "better Lab than Lab". It's based on essentially the same kind of experiments that were used to define Lab, so inasmuch as those are the wrong experiments, then it doesn't fix that problem. But everything Lab does, CAM02-UCS does better: on the experiments we have, it scores better at predicting perceptual difference judgements. These are not really the right experiments, as you note, but scoring well on them is probably better than not scoring well on them :-). Also, it has much better color constancy -- i.e., if you just change lightness or saturation, then the hue does not shift, which is very useful when trying to do things like use hue and lightness as independent channels. Lab/LCh is notoriously poor at this, esp. in the blue region (it will claim that a saturated blue and a less-saturated purple are both the same hue). I don't know that I would recommend trying to implement CAM02-UCS just so you can use it instead of Lab, because it's rather complicated to implement, but if you're working in an environment where CAM02-UCS is available then AFAIK there's basically no reason to ever use Lab for anything. (In Python, this is just a matter of typing 'pip install colorspacious'.)

MMesch commented 8 years ago

Hi,

for complex argument/magnitude, we need something like a 2d colormap. In the case of a 2d colormap we can't play much with luminosity, as in the @peterkovesi cyclic colormaps, because we need it for the magnitude axis unfortunately. I have played with colourspacious and the best I could come up with were these:

comparison

https://github.com/MMesch/mycolormaps

A smarter way to design these would be very nice indeed!

EDIT: Basically its the algorithm @njsmith suggested. I suggest 'orbit' as working name for a nice 2d colormap :)

peterkovesi commented 8 years ago

@MMesch that is a good point. It makes me think that all cyclic colour maps should probably be designed to be 2D from the start. Angular data is almost invariably associated with magnitude/confidence/coherence information in some form, and hence is 2D. Desaturating the colours to black or white to represent magnitude is the obvious thing to do. In effect the colour map is represented by a conical surface in Lab / CAM02-USC. One may be able to get away with some variation in the lightness of the colours but, as your example illustrates, the lightness variations in the HSV map are clearly too much. When the complex sine is rendered with that map the positions of the low magnitude black blobs shifts up and down with phase and some diagonal artifacts are apparent. In contrast your CAM02-USC maps have their low magnitude blobs positioned consistently across the image.

Another possibility might be to desaturate the colours to a grey value corresponding to the mean lightness of the colour map. Magnitude being represented purely by chroma/saturation rather than by a combination of lightness and chroma. Not sure how successful this might be but it may be more tolerant of lightness variations in the colour map. However I suspect using lightness to encode magnitude is probably the better way to go. Some experimentation needed.

Cheers Peter

MMesch commented 8 years ago

Agree fully. I think one should aim for 4 distinct hue regions at full magnitude, 4 - 2 distinct regions at half magnitude (playing with saturation) and then an even decay to black.

I have added a script that allows to design the colormaps in 3d with blender. Right now it works for 1d colormaps only but blender is very well adapted to design 2d surfaces such that it can be simply extended to 2d colormaps. PR's are very welcome :)

https://github.com/MMesch/cmap_builder blender_example

endolith commented 8 years ago

In effect the colour map is represented by a conical surface in Lab / CAM02-USC.

You can also do a football-shaped surface which goes from black to colored to white:

which is what I used to do these complex plots:

At each lightness it uses the largest circle in Lch (or JCh) space that just touches one of the walls of the RGB cube. In the above plot, black represents 0 and white represents infinity, neither of which has a phase, so the disappearance of color information at the extremes doesn't matter.

If instead you go right to the edges of the cube at each lightness, maximizing the chroma (so it's different chroma for different hues) it gets streaks in it:

which makes cooler-looking (but less informative) plots:

but I bet if you round the curves where the corners of the RGB cube are it would look better. I've been meaning to try that.

MMesch commented 8 years ago

@endolith looks great! The 'triplications' at blue etc.. make some problems. Also the colorsurface should be slightly tilted to allow higher saturations. As you said, it should look somewhat like a football. I have extended the blender script that one can put surfaces into the gamut. This allows to circumvent these issues and the resulting 2d colormaps look really nice and smooth!

https://github.com/MMesch/cmap_builder blender_example2

EDIT: I think saturation should be decreased a bit for intermediate lightness

nschloe commented 8 years ago

This all looks very nice!

The only thing that struck me as a little funny is the scaling from 0 to 100 here. "Why 100?", one might ask. In my application, for example, the magnitude always stays between 0 and 1, i.e., there are no poles. I wouldn't want to rescale the color map to [0,1] either since the information about the complex angle would then be lost at 1.

Perhaps a more generic idea would be to center the color map at 1 ("full saturation" or so), and log-scale to 10^{+-5} or so.

MMesch commented 8 years ago

I have designed two new colormaps in Blender. The complex function plot (below) looks clearly better than the HSV one I think, especially with the wheel2d colormap. I show log(magnitude), centered around 1. The darkwheel colormap is better for functions that don't have singularities. There are however still some minor problems. The surface grid should be less distorted around black and white (can be seen around the zeros in the darkwheel colormap). The transition to white and black should probably be a little bit more symmetric, that the size of poles and zeros corresponds rougly to their order. But this is merely an issue of playing a bit more with the surfaces in blender.

poles_and_zeros

wheel colormap: wheel2d

darkwheel colormap: darkwheel2d

nschloe commented 6 years ago

I'm coming back to this once in a while and would like to play around with the color maps.

@endolith @MMesch Do you have the code used to generate your plots?

endolith commented 6 years ago

@nschloe Yes, but it's extremely sloppy. :) I'll work on a cleaner version.

endolith commented 6 years ago

@nschloe Well here's the important part at least. It's very slow:

https://gist.github.com/endolith/b3f1b2202cce181fff594873867aa6b3

max_chroma

maximum_c-1

The constant-chroma uses the blue "min" line for all h values so it doesn't have streaks of high saturation (it does have a barely visible line at the J = 62 corner, though, if you tilt your laptop monitor or zoom in and scroll):

constant_chroma-1

To make the plots in previous posts I was generating each point using a function like this, but creating a lookup table once and interpolating might be a better idea.

endolith commented 6 years ago

Ok, I made a much better cplot() function (same gist), using fast interpolation of chroma values from the lookup table. (So only the C lookup table actually needs to be generated (though it's still 8 MB currently))

tan

figure_5

figure_1-35

@MMesch's function:

mmesch

nschloe commented 6 years ago

@endolith Excellent! For some Saturday afternoon programming, I'd like to cast this into a module; would you be okay with that?

endolith commented 6 years ago

@nschloe Ok I made a repo and added you https://github.com/endolith/complex_colormap

mathomp4 commented 6 years ago

I'm a lurker on this but interested. I've recently moved my Panoply install to use viridis by default (once I found a CPT file for it), so I'd be interested in essentially a "circular viridis" if anyone can ever distill all this "much smarter than me" into "list of RGB values" :)

endolith commented 6 years ago

@mathomp4 Something like this? https://flic.kr/p/DDxrgx

mathomp4 commented 6 years ago

@endolith Ooh. That's nice. If you can point me to them, I'll try my hand at converting to CPT (I figure if I use them for wind direction, I'll do 0->360...)

endolith commented 6 years ago

@mathomp4 I'll make a generator function. What do you mean by CPT?

mathomp4 commented 6 years ago

@endolith Apparently, it stands for "color palette table" and is a term from the Generic Mapping Tools (GMT). It seems a fairly simple format that can become complex!

Most of the ones I have for Panoply, I got from cpt-city including some that people on this thread might recognize.

On my todo list was a way to take the info from a Python palette write a script to CPT-ize it. In fact, I imagine something like that must already exist since cpt-city carries palettes in many formats. I can't see doing that by hand...though I did make one that way once (it went blue to white to blue; more of a tutorial).

BIDS / colormap

recommendation for circular colormap? #6