Calling hsl function with a hex code is slightly off

[ndreas]

I stumbled upon this while tweaking https://github.com/npxbr/gruvbox.nvim which is based on lush.nvim. Using hsl('#aabbcc') results in a slightly different color in the highlight.

It was fairly easy to reproduce, I just ran :LushRunQuickstart, followed by :Lushify and changed the highlight of the search, followed by running :hi Search to see the difference:

As you can see, #cc241d becomes #cd231d in the highlight.

Is this by design? I'm no color expert but I seem to recall converting between RGB and HSL can be lossy. To me the difference is not so big that I notice it, but I kind of expect that colors set by hex RGB stays the same since :hi takes hex RGB codes.

[rktjmp]

Hmm... Yeah converting between RGB and HSL is sort of fuzzy because there's some amount of int -> float -> round -> int conversion conversion going on.

As you say, the difference is tiny, probably nearing imperceptibility for all but the lucky people on $40k displays?

#cc241d -> rgb(204, 36, 29)
#cd231d -> rgb(205, 35, 29)

I imagine if I were to find a way for it to be lossless here, it would just end up being lossy for another color (i.e clamp up vs clamp down).

I will leave this open until I can find time to check how much drift is in the conversion, maybe some colors have more. I imagine they are nearly all within a point of each other though.

[rktjmp]

Sorry meant to post back on this. I did investigate further. The nature of converting between RGB and HSL often leads to fractional HSL values (i.e. 32.42% saturation), which end up getting munged.

Even the HSLuv library, which at least gives the impression of being made by someone pretty "into colors" has the same effect.

I don't think it's actually a problem in use though, here's a random set of colors pushed through rgb -> hsl -> rgb:

#47FF35 -> #44FF33 #AA8B09 -> #AA8909 #B9B7FC -> #B8B6FC #67CE69 -> #69CE6B #172A54 -> #172A54 #FBBCC9 -> #FBBCC8 #CC7973 -> #CD7A74 #9BACA1 -> #9BABA1 #E1FDFB -> #E2FDFB #F7980F -> #F8960D #74EBCB -> #75EBCC #0F8477 -> #0F8577 #06D970 -> #06DB70 #D97295 -> #DA7294 #C4FD0C -> #C5FD0D #709C59 -> #709B59 #82D1A0 -> #80D09F #35EBF6 -> #37ECF6 #6D11D4 -> #6C11D4 #384325 -> #374224 #CDFD51 -> #CCFD4E #A1759D -> #A2769E #A0D99F -> #A1D9A0 #229537 -> #229637 #CA79C2 -> #C978C1 #76D64F -> #74D54D #62F184 -> #5FF181 #C529F9 -> #C529FA #707C96 -> #6F7B95 #094830 -> #094930 #74C932 -> #73C832 #C76516 -> #C56516 #C05292 -> #C15393 #DE9823 -> #DE9921 #72513E -> #74523E #661163 -> #641162 #F8C96F -> #F8C76D #473CE6 -> #483DE6 #A8E187 -> #A9E189 #298C44 -> #288A42 #F4FB1A -> #F4FB18 #F7E1F9 -> #F7E1F9 #3BAF07 -> #3DB007 #132DBF -> #132DBE #23E45B -> #25E45B #7B1F9F -> #7A1F9E #04CB15 -> #04CD15 #26E0B6 -> #24E0B4 #5DBB8D -> #5DBB8E #85BF45 -> #84BF45 #B138E2 -> #B236E2 #9A6671 -> #996671 #D907AC -> #DA07AC #C6A450 -> #C7A652 #FAF7BC -> #FAF7BD #3DB0E8 -> #3BAEE8 #885B80 -> #8A5C81 #229240 -> #22913F #BFC5E3 -> #BFC5E3 #04087F -> #040881 #5A63A8 -> #5B64A9 #6C37C3 -> #6D37C3 #BD41C8 -> #BD41C8 #FE5559 -> #FE5255 #377032 -> #387133 #CD8A4A -> #CE8A4B #84A599 -> #84A498 #8BC6DE -> #8CC7DE #A9025C -> #AC025C #F5C602 -> #F2C202 #4FDCEA -> #4DDDEA #E15E18 -> #E25F18 #9995A8 -> #9994A8 #13E8CB -> #13E7CB #F20B23 -> #F40B23 #743F8C -> #743F8D #5E84C6 -> #5D83C6 #6B0E32 -> #6C0E32 #4E24CF -> #5025D0 #C1486D -> #C1496D #337C24 -> #327B24 #EF4413 -> #EF4415 #240F39 -> #240F38 #A44DF0 -> #A44DEF #F6F2F4 -> #F7F3F5 #E96D6D -> #E96D6D #8F9FE5 -> #8F9FE5 #BB3F80 -> #BB3E7F #0A965B -> #09955B #BE168F -> #C01690 #609452 -> #609451 #CB0335 -> #C90335 #E15FDF -> #E160DF #7D2B67 -> #7D2B67 #D55C68 -> #D55D69 #FB5808 -> #FB5A09 #6372BA -> #6372BB #26EC10 -> #26EA10 #5C9251 -> #5D9451 #6D4447 -> #6E4547

And a comparison of the first pair, I can't really spot the seam between the two, it will be monitor and eye dependent though. Maybe the bottom one is darker, maybe sort of? Maybe?:

Pretty interestingly, if you convert both RGB colors to HSL, you get the same values, so they are identical in the HSL colorspace:

They're actually equal in HSL, HSB and drift by 1 in LAB and CMYK.

I think you're more likely to notice differences when swapping terminals or monitors than the slight drift in RGB.

Going to close this as I think it's:

1) Maybe unsolvable? (If HSLuv has the same behaviour, I doubt I can come up with a better solution). 2) Not really impactful in use?

It is an annoying quirk though.

If you (or anyone) finds a colour that is obviously incorrect, definitely reopen.