Add a `ChaseSpectral` colormap

[mgrover1]

Based on discussion on Twitter see discussion here, it would be great to have a Spectral colormap, with added colors on the high end of spectrum (common when visualizing reflectivity, where higher values indicate colors corresponding to hail)

Example image

@wx4stg mentioned interest in helping, and I think input from @dopplerchase would be helpful too!

[dopplerchase]

some initial thoughts out loud

[mgrover1]

Good points @dopplerchase

[wx4stg]

so the way I've been doing this:

specR = plt.cm.Spectral_r(np.linspace(0, 0.95, 200))
pink = plt.cm.PiYG_r(np.linspace(0.75, 1, 40))
purple = plt.cm.PRGn_r(np.linspace(0.75, 1, 40))
cArr = np.vstack((specR, pink, purple))
cmap = pltcolors.LinearSegmentedColormap.from_list("cvd-reflectivity", cArr)
vmin=10
vmax=80

I think the grey scale is a good idea, the "going back up in lightness" could be done by reversing the pink part...

[wx4stg]

specR = plt.cm.Spectral_r(np.linspace(0, 0.95, 200))
pink = plt.cm.PiYG(np.linspace(0, .25, 40))
purple = plt.cm.PRGn_r(np.linspace(0.75, 1, 40))
cArr = np.vstack((specR, pink, purple))
cmap = pltcolors.LinearSegmentedColormap.from_list("cvd-reflectivity", cArr)

This produces the following:

[mgrover1]

specR = plt.cm.Spectral_r(np.linspace(0, 0.95, 200))
pink = plt.cm.PiYG(np.linspace(0, .25, 40))
purple = plt.cm.PRGn_r(np.linspace(0.75, 1, 40))
cArr = np.vstack((specR, pink, purple))
cmap = pltcolors.LinearSegmentedColormap.from_list("cvd-reflectivity", cArr)

This produces the following:

I like this... you have three sections like @dopplerchase mentioned. It's a bit more clear here.

[wx4stg]

specR = plt.cm.Spectral_r(np.linspace(0, 1, 200))
pink = plt.cm.PiYG(np.linspace(0, .25, 40))
purple = plt.cm.PRGn_r(np.linspace(0.75, 1, 40))
cArr = np.vstack((specR, pink, purple))
cmap = pltcolors.LinearSegmentedColormap.from_list("cvd-reflectivity", cArr)
vmin=10
vmax=80

using 0->1 for the spec_r part makes the lightness match. it makes the break softer but I think this is probably worth the trade-off

as for grayscale less than 10 dBZ... I almost want to suggest we have a standard colormap that only works for 10+ and a "low filter" version with grayscale similar to how radarscope... not sure what to think there

[wx4stg]

alright, last suggestion, then I'll let the experts talk haha. For the lowfilter colormap...

from matplotlib import pyplot as plt
from matplotlib import colors as pltcolors
from metpy.plots import ctables
import numpy as np

lowfilt = plt.cm.bone_r(np.linspace(0, 0.6, 80))
specR = plt.cm.Spectral_r(np.linspace(0, 1, 200))
pink = plt.cm.PiYG(np.linspace(0, .25, 40))
purple = plt.cm.PRGn_r(np.linspace(0.75, 1, 40))
cArr = np.vstack((specR, pink, purple))
cArrLowFilt = np.vstack((lowfilt, specR, pink, purple))
cmap = pltcolors.LinearSegmentedColormap.from_list("chaseSpectral", cArr)
cmapLowFilt = pltcolors.LinearSegmentedColormap.from_list("chaseSpectral_LowFilt", cArrLowFilt)

_, nwsrefcmap = ctables.registry.get_with_steps("NWSReflectivity", 5, 5)

[jthielen]

Seeing this in my notifications, I got nerd sniped and came up with a spectral colormap based on @dopplerchase's annotations, similar to @wx4stg but with fading to black on the low end and tweaking lightness in CIELab space to be piecewise linear. Here's a sample figure of it:

Link to RGB txt file.

This can be compared to a couple other colormaps:

Code for all this is a bit hacky, but if folks would find it useful, I could probably clean it up and share it too.

[dopplerchase]

@jthielen great work! Can you post the code for this? I think it is time we play around with it and see how it looks on all weather (e.g., tropical cyclone, midlatitude cyclone etc.). It might be the case where we need 2 colormaps, one for the cold season and one for the warm season.

Adding to the discussion:

We should do our best to practice good communication with colors (e.g., https://www.nature.com/articles/s41467-020-19160-7)

That being said determining quantitative values from a radar image is also important, and having no breaks in luminosity (e.g., reference points) can be limiting. So the key here is to have motivation for the luminosity changes. For example, we can motivate the 30-35 dBZ as the convection-no convection threshold (but we might want to re-check some papers on the exact threshold. Then we can also motivated 60 dBZ as a 'hail' threshold (again need to check papers here).

This will ensure we have a meaningful colormap that has evidence for the breaks, of which I think is currently laking with the legacy NWS map.

[mgrover1]

@jthielen Thank you so much for putting this comparison together! I noticed in that first image that the lowest part of the scale is not showing up... do we need that part?

[jthielen]

@jthielen great work! Can you post the code for this? I think it is time we play around with it and see how it looks on all weather (e.g., tropical cyclone, midlatitude cyclone etc.). It might be the case where we need 2 colormaps, one for the cold season and one for the warm season.

Sure thing! I cleaned it up a little bit (including making the control parameters very clear), and put it up here: https://nbviewer.org/urls/dl.dropbox.com/s/nvnav3m9x5w8xf1/spectral_reflectivity_cm.ipynb. Note that I have a mode-dependent random image selection for the preview. The choices are (similar to Gallus et al. 2008):

• "BE": bow echo
• "NS": linear with negligble stratiform
• "PS": linear with parallel stratiform
• "LS": linear with leading stratiform
• "TS": linear with trailing stratiform
• "NL": nonlinear
• "TR": tropical
• "SY": synoptic
• "DC": discrete cell
• "IC": isolated cells
• "CC": cluster of cells
• "BL": broken line of cells

Feel free to play around with it and report back with any improved sets of parameters! I'd be particularly curious how low we can go on that "resolution factor", since that'd determine the color table size in the code base (unless pyart wants to add needed lab/rgb conversions and be able to procedurally generate it?)

Adding to the discussion:

We should do our best to practice good communication with colors (e.g., https://www.nature.com/articles/s41467-020-19160-7)

That being said determining quantitative values from a radar image is also important, and having no breaks in luminosity (e.g., reference points) can be limiting. So the key here is to have motivation for the luminosity changes. For example, we can motivate the 30-35 dBZ as the convection-no convection threshold (but we might want to re-check some papers on the exact threshold. Then we can also motivated 60 dBZ as a 'hail' threshold (again need to check papers here).

This will ensure we have a meaningful colormap that has evidence for the breaks, of which I think is currently laking with the legacy NWS map.

For convection-no convection threshold, papers vary considerably (while it's more for MCS criteria, I typically refer to the papers in Table 1 of Haberlie and Ashley 2018 for this), so having the inflection point be roughly 35 dBZ shouldn't be an issue!

I'm less acquainted with papers on no hail-hail thresholds (probably because it is far less well-delineated!), but choosing round values in the general range like 60 and 70 seem quite sensible.

@jthielen Thank you so much for putting this comparison together! I noticed in that first image that the lowest part of the scale is not showing up... do we need that part?

That's an artifact of the sample data I used (it is clipped to remove values below 0 dBZ)...I still think it would be useful to extend into the negatives, though exactly how far is a choice to be determined.

[mgrover1]

Great! Thanks @jthielen - is someone interested in moving this over to PR land? @jthielen @wx4stg @dopplerchase

[wx4stg]

I would love to but yesterday I realized I have no idea how the colormap formatting in py-ART works... if someone else does it I'll definitely be learning something new haha.

[jthielen]

PR created here: https://github.com/ARM-DOE/pyart/pull/1325. I switched back to -10 rather than -20 as minimum...subjectively thought it looked better. Also, came up with a related colormap that is simply divergent in lightness (rather than multiple local extrema) while I was fiddling around with parameters, so included that one as well.

[mgrover1]

Closed with #1325