Brightness of 'plotly' template colorscale

[empet]

I tested 'plotly' template, with different traces and concluded that the colorscale used in this template is too bright.

In this notebook https://plot.ly/~empet/15032 I compared its brightness, respectively lightness function, with the same functions associated to jet colormap (a bad colormap that was replaced as the default colormap in matplotlib, matlab; seaborn displays an error when a user chooses this colormap), as well as to four perceptual uniform colorscales (viridis, magma, cmocean ice and deep). The final conclusion is that plotly template colorscale is much more like the jet colormap than the perceptual uniform ones.

How was derived/defined this colorscale? Why it has 13 entries, instead of 11 (with 11 colors, the plotly scale is [0, 0.1, 0.2, ..., 1.]? With 13 colors the scale contains floats with many decimals.

[jonmmease]

Hi @empet,

Thanks for caring about the new plotly themes 🙂 And apologies to all of our color conscious users for not laying out the technical background for this choice of colormap. I'll try to remedy that now.

This colormap is based on CET-L7 (aka linear_bmw_5-95_c86_n256) from Peter Kovesi's work on perceptually uniform colormaps (https://peterkovesi.com/projects/colourmaps/index.html).

As you noticed in your analysis, this family of colormaps is not uniform in components of the hsl or hsv colorspaces (and also as your analysis shows, neither is viridis). Instead, they are uniform in the Lightness (L*) component of the CIELAB colorspace (https://en.wikipedia.org/wiki/CIELAB_color_space). From the wikipedia link

The nonlinear relations for L, a, and b are intended to mimic the nonlinear response of the eye. Furthermore, uniform changes of components in the Lab color space aim to correspond to uniform changes in perceived color, so the relative perceptual differences between any two colors in Lab can be approximated by treating each color as a point in a three-dimensional space (with three components: L, a, b) and taking the Euclidean distance between them.[13]

I say it's "based on" this colormap because we've clipped to upper and lower ends slightly so that the extremes aren't so close to black and white. This doesn't alter the perceptually uniform characteristics, but it improves the contrast of the extremes against both a white and black background, at the cost of a little bit of dynamic range.

Here is a comparison of the plotly and viridis colormaps in CIELAB lightness space.

# Imports
import plotly
import plotly.io as pio
import plotly.graph_objs as go

from colorspacious import cspace_convert
import numpy as np

# Get colormaps
plotly_cmap = pio.templates['plotly']['data']['bar'][0]['marker']['colorscale']
viridis_cmap = plotly.colors.PLOTLY_SCALES['Viridis']

# Build figure with subplots
fig = go.FigureWidget(plotly.tools.make_subplots(rows=1, cols=2, shared_yaxes=True, subplot_titles=['Plotly', 'Viridis']))

# Configure layout
fig.layout.yaxis.title = 'CIELAB lightness'
fig.layout.xaxis.title = 'Scale value'
fig.layout.xaxis2.title = 'Scale value'
fig.layout.showlegend = False

# Plot colormaps in CIELAB lightness space
for col, (cmap_name, cmap) in enumerate(zip(['plotly', 'viridis'],
                                            [plotly_cmap, viridis_cmap])):
    scale_vals, colors_hex = list(zip(*cmap))
    colors_rgb = np.array([tuple(int(h[i:i+2], 16)/255.0 for i in (1, 3, 5)) for h in colors_hex])
    colors_cielab = cspace_convert(colors_rgb, 'sRGB1', 'CIELab')
    lightness_cielab = colors_cielab[:, 0]
    fig.add_scatter(x=scale_vals,
                    y=lightness_cielab,
                    mode='markers+lines',
                    marker={'color': colors_hex, 'size': 20},
                    line={'color': 'black', 'width': 1},
                    name=cmap_name,
                    row=1,
                    col=col+1)

# Display figure
fig

So you can see that they both scale linearly in this lightness space, and you can see that viridis has a bit more dynamic range due to the clipping discussed above.

The actual calculation of the plotly colormap leverages the excellent colorcet library (https://github.com/pyviz/colorcet). Note that since plotly.js interpolates colorscales automatically it's not necessary to keep every single color value computed by colorcet.

https://github.com/plotly/plotly.py/blob/a8ae062795ddbf9867b8578fe6d9e244948c15ff/templategen/definitions.py#L216-L223

In terms of why we chose this colorscale rather than any of the other perceptually uniform options, that was purely a matter of taste. To my eye, it looks really nice within the context of the plotly brand colors that this theme is based on (https://brand.plot.ly/).

The biggest limitation of this family of colorscales is that, as far as I understand, they don't take colorblindness into account. So I would still like to develop a new built-in theme that is more robust to various forms of color blindness. Volunteers welcome!

I hope this helps shed some light on the rationale. Thanks again for taking the time to share your feedback.

[empet]

Thanks @jonmmease for the details. I referred to the brightness that is exaggerated in comparison with all known and used colormaps. That's why I converted each color to HSV. Otherwise I knew from BIDS explanation (when viridis, magma and inferno were proposed for matplolib use) how is devised a perceptually uniform colormap.

[jonmmease]

Hi all, just wanted to post an update that we are planning to use the 'plotly' colorscale by default for plotly.py version 4. And we're taking another look at the template colorscales.

I'm proposing that we change the default sequential scale from the linear_bmw scale described above...

To the plasma colorscale.

This colorscale was developed alongside the more well know Viridis (http://medvis.org/2016/02/23/better-than-the-rainbow-the-matplotlib-alternative-colormaps/), and it shares the same perceptual uniformity properties.

I like this scale because the bottom end has a very similar dark blue to purple transition to linear_bmw, which was chosen to fit nicely with the plotly brand colors (https://brand.plot.ly/). But it has a larger dynamic range as it includes a nice purple to orange to yellow transitions above that. What do you think @empet?

[empet]

I plotted comparatively the Mt Bruno, with lighting and lightposition like in the lines below:

lighting=dict(ambient=0.5,
                      diffuse=1,
                      fresnel=4,        
                      specular=0.5,
                      roughness=0.5),
lightposition=dict(x=100,
                           y=100,
                           z=2000)

and comparing with Magma and Inferno colorscales, too, https://plot.ly/~empet/15162 I decided to vote for Plasma.

Update: I've just realized that the cover photo of @plotlygraphs https://twitter.com/plotlygraphs is a streamtube trace with Plasma colorscale.

[jonmmease]

Great, thanks for weighing in @empet!

[empet]

More arguments that the linear_bmw_5-95_c86_n256 colorscale does not meet the standards for a good colorscale.

To get the first image of a blackhole an EHT team worked on searching the best color space and parameters that define a good colormap. The ehtplot library provides tools to inspect/analyze the existent colormaps and build new perceptually uniform colormaps (PUCs).

I cloned its repository https://github.com/liamedeiros/ehtplot, and calling a few functions I plotted comparatively the lines representing the basic parameters in the CAM02-UCS color space, that influence the quality of a colormap, for the following colormaps and their reversed versions: hot linear_bmw_5-95_c86_n256 (i.e. plotly-template coloscale), plasma, inferno, and cmocean deep colormap:

The colorful line represents the lightness J', the dashed line is chroma C', and the dotted one is the hue, h'.

A PUC should have a linear lightness with respect to data values. We can see that the lightness of the hot colormap is far from being linear. It is also nonlinear for Kovesi's colormap, but it is linear for the perceptually uniform colormaps, plasma, inferno and cmocean deep_r.

In order to illustrate how non-uniform colormaps can lead to faulty feature extraction, ehtplot provides a function that defines the so called pyramid, i.e. a heatmap plotted with the inspected colormaps. In the case of a PUC, the viewer's eye should pick out only a cross X connecting the heatmap oposite corners, and obviously the color gradient.

The pyramids associated to the above cmaps and their reversed versions are illustrated in this image:

Following the comments in the ehtplot notebook, COLORMAPS.ipynb, for hot and hot_r, plotly and plotly_r we artificially see red and yellow squares, respectively blue and magenta squares in the representative pyramids.

These squares are caused by the nonlinearity of lightness, J'. In plasma and plasma_r pyramids we can identify some squares only because of the color transitions.

The pyramids for other PUCs:

exhibit similar patterns like plasma and plasma_r.

To correct the lightness non-linearity (or non-uniformality) ehtplot implements an algorithm for colormap uniformization as well as for the so called lightness "symmetrization" (details in COLORMAPS.ipynb)