Distinguishable colors not working?

[KristofferC]

I saw the discussion here https://github.com/JuliaGraphics/Colors.jl/issues/18 about generating distinguishable colors. However, for me, this does not seem to work right now. For example:

Here there are for example two blue identical colors and two quite similar red/orange ones.

[KristofferC]

Manually using the colors from Colors.jl gives better result:

[tbreloff]

I changed the way it works... it doesn't use distinguishable_colors anymore. I changed it to pick colors out of an arbitrary color gradient (I didn't love the colors generated before, and keeping a finite list actually broke Gadfly with too many series on one plot). Right now it's using the "rainbow" color gradient, but I could easily do the same thing with a gradient generated from distinguishable_colors. I need to make a couple small changes, and then you could pick your own gradient and help me settle on a good default. I'll keep you posted.

On Tue, Oct 20, 2015 at 5:20 AM, Kristoffer Carlsson < notifications@github.com> wrote:

Manually using the colors from Colors.jl gives better result:

[image: selection_020] https://cloud.githubusercontent.com/assets/1282691/10603108/80df813a-771c-11e5-8ce5-2b536ba82d76.png

— Reply to this email directly or view it on GitHub https://github.com/tbreloff/Plots.jl/issues/51#issuecomment-149489324.

[tbreloff]

I put together an IJulia notebook on colors in Plots: https://github.com/tbreloff/Plots.jl/blob/dev/examples/palettes.ipynb. I'd love input, as well as some suggestions for alternative palette generation.

[tbreloff]

cc: @timholy Tim: I'd love your opinion on color generation as well.

[timholy]

I'm probably not the best person to comment, but I'll give a shout-out to three who know more than me.

The colors output by distinguishable_colors, in the end, come down to how different pairs of colors are judged by colordiff. The algorithm is:

• generate a long list of possible colors (the candidate variable in distinguishable_colors)
• start with an initialized list of chosen colors (the seed input to distinguishable_colors)
• from the candidates, pick the one that is most-distant from all previously chosen colors (this is where colordiff is crucial)
• iterate the step above this one until you've generated as many as have been requested.

So there are really only a few things that determine the returned colors: the seed, the candidates, and the colordiff algorithm. @dcjones has done some nice work in limiting the candidate colors, e.g., to be of a restricted range of luminance (see the optional inputs to distinguishable_colors), and that improves the aesthetic appeal of the choices.

But if you're worried about "near-duplicates," perhaps the core issue is the colordiff algorithm. Over the last 100+ years, a lot of work has gone into measuring differences between colors, and sadly, I'm not a colorimetry expert. Other than some type-stability fixes, I've made no contributions whatsoever to colordiff.

According to git, it looks like @glenn-sweeney wrote most of Colors/src/differences.jl, with some contributions from @m-lohmann. I know from past conversations that both of them know a lot more about color than I do, so perhaps they can offer some insight.

[tbreloff]

Thanks @timholy. I see the code in Gadfly here. I'll be honest that I don't fully understand what each of those arguments change... I'll try to dig into it more.

[m-lohmann]

I think CIELAB or the DIN color spaces provide the most “euclidean” metrics for a good evaluation of color differences. CIEDE2000 would work, too, but it might be computationally too expensive. So, LCHab should be uniform enough to provide a good metric for color difference computations; it’s just CIELAB using cylindrical coordinates. The actual problem might come from the conversion via the deuteranopic simulation. As far as I know, also from reading other papers on color deficiency algorithms, the implemented algorithm is probably not the best way to guarantee a good equidistribution of colors. Color deficiency is a tricky topic, and I guess that this could be the main problem here. Would distinguishable_colors without the deuteranopic step produce better results (to color normal persons)? If that’s the case, then maybe it would be a good idea to make the color deficiency simulation optional until a better solution for color deficiency friendly palettes is found.

Edit: I just had a look at how RGB values are handled. The only treatment we get for out-of-gamut colors in sRGB is clamping values to the border of the range. But if you generate LAB values, then a lot of colors that are generated are outside the sRGB color space. So, you might generate proper colors in LAB. But even if the difference in chroma is large, there is a certain class of colors that still have similar hue and lightness values. And all these out-of-gamut colors get clamped to very similar colors on the gamut boundary of sRGB. I might not be entirely correct about the exact location the colors get clamped to, but that doesn’t matter for the actual problem. So, to prevent the clamping it must be guaranteed that all colors generated in LAB must also lie within the sRGB gamut boundaries from the get-go. The “harsh” treatment of pretty much cutting out and ignoring half of the LAB gamut during conversion to RGB always bugged me. You need RGB values outside the range of [0..1] to perform proper conversion, even between ARGB and sRGB, even more so during conversion from device independent color spaces like CIELAB, DIN99 etc. Clamping is pretty much the worst solution.

Combine the clamping/hue/lightness problem with the projection of 3D color coordinates to a 2D color space (a plane in XYZ) that’s done in the color deficiency simulation, and you get all kinds of strange “distinguishable” palettes.

[tbreloff]

I think it defaults to skipping the deuteranopic step (since transform = identity by default). I played around some more in that palette notebook... I'm really curious which palettes you guys prefer...

Do you have any experience with the "husl" color space?

[m-lohmann]

I had a look at the HUSL website, and from what I can tell, it’s also quite non-uniform. It’s basically LUV pressed into a number scheme to guarantee the colors stay inside the RGB gamut, if I understand the Python code properly. I don’t see anything that would guarantee perceptual uniformity. LUV is not particularly uniform in the first place, it’s just a slightly more uniform substitute for the Yxy chromaticity diagram. And it’s about the best you can get in terms of perceptual uniformity if you use only linear transformations, as MacAdam put it in his book. HUSL would still not deal with the out-of-gamut problem properly, because it’s just mapping the straight RGB gamut borders to a circle, disregarding everything else, unless I completely misunderstood the inner workings of this color space.

Nevertheless, it would be interesting to test it as an alternative for distinguishable_colors, if there would be any noticeable improvement. And having another color space in the tool belt can’t be bad.

[timholy]

@m-lohmann, good catch. Here's what I get:

with the current Colors, for which the line that generates the candidates is

    for h in hchoices, c in cchoices, l in lchoices
        candidate[j+=1] = LCHab(l, c, h)
    end

Here's what I get:

if I first make sure that every color is representable as RGB:

    for h in hchoices, c in cchoices, l in lchoices
        rgb = convert(RGB, LCHab(l, c, h))
        candidate[j+=1] = convert(LCHab, rgb)
    end

To my eye, the second group looks much more distinguishable. Do others agree?

[tbreloff]

I agree that it's better with that change, but I don't think it's quite ideal. Using white as the seed (and then throwing it away, since it's the background color), I see:

I like this one, which uses a predefined gradient with my "zvalue pattern" algorithm to pick infinitely from the gradient:

I'm thinking that could be good for the default "white-enough" background, and if it's close to black, just lighten those colors. If it's a strong hue (what's a good way to determine that?) then it could default to using the new distinguishable_colors with the background as the seed.

Thoughts? Do you think a non-greedy distinguishable_colors algorithm would be much better? What if the algorithm treated contrast with a background color differently than it treated hue-difference with the other colors?

[timholy]

I don't think it's quite ideal

Aesthetically or in terms of their distinguishability?

I like this one, which uses a predefined gradient with my "zvalue pattern" algorithm to pick infinitely from the gradient

Those do look like good color choices. Roughly speaking they seem to be of similar luminance and relatively high chroma; if you want to develop a bit more intuition for those parameters, this page has a widget you can play with (set the number of stops to 1 and choose Lch).

But certainly your algorithm seems great, and might be much faster than distinguishable_colors. I'd think it would be fine to add that algorithm to Colors, if you'd like.

[tbreloff]

Aesthetically. My needs are a little more, since I want high contrast with a background, good looking initial colors, and I want each color to have similar "pull" to the eye (ie I don't want any colors to blatantly stand out... Not sure the correct terminology)

I'll look at that link and see if I can generalize those goals as part of the algorithm.

As for contributing to Colors... Would you like to look through src/colors.jl and let me know if any of that looks Colors-worthy?

On Oct 21, 2015, at 7:45 AM, Tim Holy notifications@github.com wrote:

I don't think it's quite ideal

Aesthetically or in terms of their distinguishability?

I like this one, which uses a predefined gradient with my "zvalue pattern" algorithm to pick infinitely from the gradient

Those do look like good color choices. Roughly speaking they seem to be of similar luminance and relatively high chroma; if you want to develop a bit more intuition for those parameters, this page has a widget you can play with (set the number of stops to 1 and choose Lch).

But certainly your algorithm seems great, and might be much faster than distinguishable_colors. I'd think it would be fine to add that algorithm to Colors, if you'd like.

— Reply to this email directly or view it on GitHub.

[KristofferC]

I know nothing about theory of colors but in my subjective view the one that I liked most aesthetically is this one:

To me, none of the colors there really stick out. However, I do think that the 3 blue ones and the orange/redish ones are quite similar to each other. More similar than any of the ones in for example:

For some reason it always to me seems like I think the blue colors are similar to each other. Maybe I am just bad at distinguishing blue nuances...

[tbreloff]

A couple things:

• I think it's a fact that blues are harder to distinguish, which is why distance in RGB space will never be as good as distance in other spaces.
• The last 5 colors in the second example are really hard to see on a white background, so it's not enough that they're different from the first ones.

I think the best of all worlds might be a distinguishable colors algorithm that can account for background color, seed colors, as well as some parameter to determine how much the colors can "stand out" from each other. I'm not sure how much effort that is...

[timholy]

Again, just constrain luma and chroma and I suspect you'll get most of these things you want. High luma + low chroma = white, which is what you're trying to avoid.

But also again, plenty of room for additional algorithms. I took a very brief glance through your src/colors.jl. IIRC there's some redundancy with stuff in Colors.jl, but it also looks like there are some good tools in there. Two brief comments:

• not sure where you got the "color lists" or palettes from, are there any license concerns?
• names like getPaletteUsingFixedColorList leave a little to be desired.

I'd encourage you to put together a PR.

[tbreloff]

are there any license concerns?

That's a good question. Honestly I don't know anything about "color licensing". Where do I begin? Are there general rules to follow, or some way that licenses are registered? If I can't remember where I saw something, but think I probably got inspiration from an online source, should I just delete the colors to be safe?

names like getPaletteUsingFixedColorList leave a little to be desired

no argument here!

[m-lohmann]

The problem is partly because the blue region is this weird thing poking out like a tail in CIELAB color space. Blue colors aren’t the strength of CIELAB, there is a considerable lack of hue “stability” in the blue region. In CIELAB the blues are drawn out into this “tail”. Maybe cutting off the stronger chromas due to sRGB constraints, the area to choose from is too small, so the amount of blue color variation is not large enough. You can see in the LAB diagram that maximum chroma of colors varies a lot around the full hue angle.

The dashed line is the border of the sRGB gamut:

The same in DIN99o/DIN99c, showing a lot less variation in maximum chroma:

Maybe a grid like system like the OSA-UCS cuboctahedral arrangement of colors could be helpful. http://jimlaiplus.appspot.com/OSA-UCS/index.jsp

Another very important reason for the perception of similarity or difference of colors are the neighboring colors in a scale. The effect can lead to drastic misinterpretations of the hue and lightness of colors.

[tbreloff]

I took in all your comments, thought on it, and came up with a good solution that uses both the distinguishable_colors algo and my zvalues algo to generate an infinite palette that fits my requirements. Here's most of the new code:

function adjust_lch(color, l, c)
    lch = LCHab(color)
    convert(RGB, LCHab(l, c, lch.h))
end

function lightness_from_background(bgcolor)
  bglight = LCHab(bgcolor).l
  0.45bglight + 55.0 * (bglight < 50.0)
end

function gradient_from_list(cs)
    zvalues = Plots.get_zvalues(length(cs))
    indices = sortperm(zvalues)
    sorted_colors = map(RGB, cs[indices])
    sorted_zvalues = zvalues[indices]
    ColorGradient(sorted_colors, sorted_zvalues)
end

function generate_palette(bgcolor = colorant"white";
                         color_bases = [colorant"steelblue", colorant"indianred"],
                         lightness = lightness_from_background(bgcolor),
                         n = 20)
    seed_colors = map(c -> adjust_lch(c,lightness,50), vcat(bgcolor, color_bases))
    colors = distinguishable_colors(20,
          seed_colors,
          lchoices=Float64[lightness],
          cchoices=Float64[50],
          hchoices=linspace(0, 340, 20)
        )[2:end]
    gradient_from_list(colors)
end

What this does is take in the background color, and a list of initial seed values, figures out a good lightness level, converts the seed colors and limits the candidate colors to that lightness, then generates a custom ColorGradient with the exact z-values which will perfectly match the first n generated colors. After that it will infinitely pull from the gradient using my zvalue algorithm.

Here's some results:

Thanks for all the help... what do you think?

(ps - I deleted a few of those pre-defined gradients that I wasn't sure about licensing, since I don't think I'll be using them anyways)

[tbreloff]

I'm going to close this, but please keep the conversation going if there's more to say. Thanks again for the insight.