"Colorant"

[rennis250]

We once had this discussion briefly before in another issue, but I didn't fully complete my thought at that time. I would argue that the base type should not be named "Colorant". Since this package is about colors, which are products of one's visual system, not elements of the environment, then the base type probably shouldn't be named after a term used for solely pigments, especially when a monitor doesn't produce colors by mixing colorants, with monitor presentation probably being the most frequent usage scenario of this package (plus, a pigment by itself is assigned no color by a person, since they can't see it, and there are circumstances where its color will be dependent on the actual light source). I would argue that the base type should simply be named "Color", which is the actual phenomenon we try to model here, encompassing surface colors, illuminant colors, volume colors, etc. Then, "Color"'s two subtypes could be something like "Opaque" and "Transparency", although "Opaque" is a bit unappealing to me. "Transparency" is at least commonly used in the field (EDIT: I got a bit ahead of myself here and was wrong in making claims about perceptual transparency). Although this might seem a bit strange to some users at first, it is no more difficult to accept than the current naming scheme and I feel it more closely reflects the phenomenon of colors and the terminology used in color science and research.

Best, Rob

[timholy]

You probably know that Color already exists as a subtype of Colorant. Can you sketch out the hierarchy you are proposing?

As you know from previous discussions, this was one of the candidates but was rejected (https://github.com/JuliaLang/Color.jl/issues/101#issuecomment-133521830). For me this ship sailed in https://github.com/JuliaLang/Color.jl/issues/101, but I won't object to changes if others are enthusiastic (and are willing to do the work). I also sincerely apologize for not CCing you on that discussion when it was being decided.

These kinds of changes might be easier now that we have Base.@deprecate_binding, but if you're re-using an old name for a new meaning, you might need two rounds of deprecation: Colorant->ColorFuture, Color->Opaque, wait for people to update their packages, then ColorFuture->Color.

[rennis250]

No need to apologize, as I've been away for some time and I don't think I've never really made it explicit that I work in color vision research. I realize that I'm arriving to the party very late here and causing trouble when I usually don't like to rock the boat. There were a few reasons why I had to pull back from Julia for some time, but hopefully I can more frequently partake now. Anyway, since that's the case, I would be perfectly fine with taking on the potential work, depending on what people agree to.

Anyway, yes, I am aware that Color is a subtype of Colorant, but to me at least, a colorant is a physical cause of color, not a color in and of itself. Rather, it should be listed as something that, when illuminated, gives rise to the percept of a surface color (however, this can be influenced by a few psychological factors). One could argue that I am being pedantic or overly technical here, but this really is how colors work, so I am just suggesting that the Colors type hierarchy reflects that.

This argument does get a bit sticky, though, because lights also have no color (going back to Newton's old saying, "The rays are not colored"), yet there is a very strong historical tradition of specifying colors as a mixture of the lights that produced them (e.g., the RGB space). You can bend the rules a bit here, though, because light is the most frequent and direct physical cause of seeing color. Plus, it's also a bit hard to push back against the legacy of Newton, Young, and Maxwell. ;)

[rennis250]

Right... a sketch. Only the top two layers would be affected. Colorant would become Color, TransparentColor could remain as is, and the present Color type could become something like SolidColor (SurfaceColor would directly reflect terminology from color vision research) or the previously proposed OpaqueColor (the previous arguments against this choice didn't really convince me). Nothing more than that.

[timholy]

On the technical point, I thought it would be safe to say that a laser emitting at 525nm is "green." That is, color is determined by the spectrum of the light, and that material properties are important only as a means to altering the spectrum.

[rennis250]

Color is interestingly not that directly tied to physics. The perceived color of a surface can be altered by simply placing a piece of paper of a different color around it. This is known as simultaneous color contrast and is a perceptual effect that is easily illustrated:

http://colorusage.arc.nasa.gov/Simult_and_succ_cont.php

Nothing has changed about the original physical stimulus, yet to the observer it appears to have a different color. The effect is rather striking when one displays a gray disk on a screen and sets the rest of the screen to modulate sinusoidally between black and white over time. You will see a substantial change in the color of the central "gray" disk, although it is not physically changing.

There are a number of other effects in the literature that I can reference, if necessary. The point is that color is actually a psychological phenomenon: one that is initiated by physical processes, but which is shaped by psychological/neural responses and interactions.

[m-lohmann]

@rennis250 Absolutely, color is just the perception of a physical stimulus. Another striking example of that is metamerism. The effects you describe, like simultaneous color contrast, Hunt effect, Stevens effect, scotopic and mesopic vision, complex chromatic and luminance adaptation, tone compression etc. are handled by image appearance models like iCAM06 that are much more complex than the color package we have here.

@timholy Apart from color being just a perception (quale), every deuteranomalous or protanomalous (red-green “colorblind”) person would disagree about the greenness of your laser ;) And, as I mentioned above, individual color perception actually is determined by the reflectance/transmittance spectrum of an object and the spectrum of the illuminant, but that’s only half the truth. Actually, every perceived color can be caused by an infinite amount of different spectral stimuli, the reason for the existance of metamerism. The reason is that an infinite dimensional range of wavelengths gets mapped to a mere 3-dimensional space (we only have 3 types of cones). If that weren’t the case, we couldn’t perceive a scene in nature, on a monitor, and printed on a piece of paper as looking the same because the spectral composition of colors on an LED monitor is very different from the spectral composition of the same colors on a printed photo or the actual physical object/scene.

[timholy]

These context effects are not the job of the Colors package, however; in the end what we're modeling is the equivalent of the projection onto LMS that the cone photoreceptors in that patch of retina will receive.

However, I now understand your point above better: you're saying that a "colorant" is the physical object which requires illumination to produce the color. I originally misread your statement as saying that only physical objects could have color. (Looking back, you were very clear, I was just misinterpreting what you meant.)

So, I agree that Colorant still isn't an ideal name, but I think it's closer than Color. The biggest problem I see is that transparency isn't really color either: it's a rendering property, and has meaning only in the context of any other "colorants" that end up at the same screen position. Alpha compositing simulates the physical process of the Beer-Lambert law governing the propagation of light through absorbing materials. In that sense, an RGBA value really is closer to the "physical material that produces color," i.e., a colorant.

[rennis250]

That's the thing, though. We aren't only modeling quantal cone catches. We include the LAB space and LUV space, both of which model color opponency and chromatic adaptation: one represents a multipicative adaptation for direct viewing of the illumination and the other represents a subtractive adaptation to the illuminant for the viewing of surface colors (or the other way around... I can never remember which is which off the top of my head because I don't use them actively in my work). In addition, both LAB and LUV seek to be perceptually uniform by ensuring that MacAdam's discrimination ellipses (as represented in the xyY space) form circles when plotted in the LAB/LUV spaces. Perceptually uniform color spaces are a beast of a subject that ultimately try to model perceptual similarity, which has recently been found to be influenced by one's mental representation of the surfaces before them. None of these processes are accounted for by cone activity or physics. The subtractive adaptation processes and color opponency, at the least, are occurring in the retinal ganglion cells.

Also, we include the HSV/HSL spaces, which are definitely not accounted for by cone activity or physics.

More importantly, an accurate explanation of color is incomplete without context effects, even in the most basic of colorimetry experiments and even at the level of basic retinal activity, as was found to the dismay of some (even in historically important cases, such as the previously mentioned MacAdam ellipses and the development of the LAB/LUV spaces). The point is that there is only one situation in which you can view a color in complete absence of (spatial) context. It is known as a Ganzfeld and is constructed by placing an observers head in a sphere that is illuminated at all locations. A few experiments have been done in these situations, but it is so unnatural that observers see strange epiphenomena.

Regarding transparency, that is a subject that I am still not fully decided on myself, regardless of what I said in my earlier post. The perception of transparency is also a heavily context dependent effect and transparent objects can be rendered to appear solid, even when physics says that they aren't. Wittengenstein in his "Remarks on Color" returned to this point often, as it seemed to be an important point for him (rightfully so, in my opinion). What I want to say is that even the RGBA space admits that no colorants are involved in the process and its effects are dependent on context. We merely mix some lights in an appropriate spatially-dependent context and then the observer thinks a transparent object is before them, when nothing of the sort is before them.

Also, the RGB and RGBA spaces and their cousins stick around for historical and technical purposes, not because they are accurate models of color (in fact, they aren't models of color at all, but specifications of how to mix lights - this misunderstanding has been the source of some ruined years of hard work for others). Basically, once you have a color in mind, you need to tell your device to produce it somehow, and this is currently only possible by adjusting the primaries on your monitor, so RGB and RGBA stay around in color packages for their utility. I would personally organize them under an engineering-style color management package for monitor presentation and not include them in a color package, proper. However, historical tradition is sometimes hard to break from.

This current package models both some perceptual processes and some physical processes. Regardless, for a package called "ColorTypes", the physical cause should not dominate the actual color, which is a psychological phenomenon. I would argue that we stick with tradition and parsimony with the rest of the field, as well as being as accurate about the color perception process as possible. In other words, we allow the RGB and RGBA spaces and their cousins as "Colors", and appropriately represent the rest of actual colors as "Colors".

If we do leave "Colorants" at the apex of the hierarchy, then we must remove the LMS, LAB, LUV, HSL, and HSV spaces from that hierarchy, as none of them model colorants or optics and we are being more inaccurate in our message than by following tradition.

[timholy]

My head is starting to hurt.

My personal bottom line: I think Color isn't right for anything that includes transparency, and you've pointed out problems with Colorant. So it sounds like there needs to be a new name.

I'm fine with correctness, but it has to be done in a way that doesn't resoundingly trump practical utility. I doubt it's viable to have RGB living in one object hierarchy, LAB in a separate hierarchy, and LMS in a third hierarchy, even if you have conversion functions between them---it seems unavoidable that it will complicate the lives of people who just want to use "colors."

Maybe there needs to be a Colorimetry.jl package which is the minimal correct foundation, and then ColorTypes.jl can live on top of that and provide the same features it currently does, warts and all.

[rennis250]

I need to take some time away right now, but I will provide references that will hopefully show that the concept of color does include transparencies, although in more complex circumstances, transparency becomes the subject of material perception (in some sense, this is the modern name for present-day color perception studies). Transparency is an active research topic in the field and is the subject of a few posters and talks at international conferences on the subject of color. Wittengenstein put it this way: find a piece of paper or paint with the same color as a piece of colored glass. You basically can't do it, even if a spectrophotometer says that the light coming from the glass and the light coming from the paper have the same exact chromaticity coordinates. Then, he proposed the next task: take a painting with a transparent object and remove a small square from that painted transparent object. You will find that it no longer looks transparent, even though it was painted to render the physical process of transparency. Transparency starts as a physical process, yes, but the color it is assigned (sometimes known as 'veiled' or 'volume' colors, depending on how you frame it) is dependent on the observer's visual system and on context and is fundamentally different from the color assigned to papers and other such opaque surfaces; not because it is all physics, but because this is how our brains are built. So, yes, while we have a light mixing system (RGBA) that mimics what happens in the case of physical transparency, the result is actually an optical illusion.

LMS and LAB would not be in separate packages. I merely forgot to list LMS in my list at the end of my previous post. Sorry about that. Both LMS and LAB model different stages of the color perception process on the psychological side, while the RGB space models the physical side of initiating that process. In my ideal world, I would place the physical side and the psychological side of color in two separate hierarchies (not necessarily two separate packages, as I suggested earlier). The basic distinction would always be between what optics says about the physical distribution of the current light field and what the observer says they see. Then, you work with colors as the psychological entities that they are in a perceptual/physiological space and when you are done, you can ask the computer for a physical display that produces the effect you desire by converting to one of the physical spaces. You could even build in a metadata system for perfect and accurate color conversion for your monitor or display apparatus. This keeps things clean and coherent and would directly match up with what we currently know about color. Lastly, you could still always work in a pure RGB space or one of its cousins, without any loss of generality or utility, and convenience functions could be provided for those that just want "colors", if this proposed structure is deemed too complicated.

Ultimately, I think the current organization of ColorTypes as a foundation makes sense and fits nicely with Julia's type system. For educational purposes, it is better to start with the colorimetric experiments that gave rise to the different color spaces, but for practical work, it is better to start with the spaces that one obtains, which is basically how things are currently laid out.

EDIT: Just to be clear, what I meant by that last paragraph is that I like the general layout of the color packages and how the types work with each other. I just think the type hierarchy itself needs some alteration. Splitting the colorimetry aspects into a separate package is probably not necessary.

[rennis250]

Ok, I realize I'm getting rather dense and probably distracting. In principle, I agree with you that RGBA is a physical space and it models a process of physical transparency, so it should be listed under a hierarchy of physical causes of color. However, perceptual transparency also exists and is accompanied by its own set of colors, which we could call the "transparent colors". Anyway, listed below are some references for the issue of perceptual transparency, in which we find that the notion of a transparent color can be developed. I can provide more, if necessary.

https://en.wikipedia.org/wiki/Perceptual_transparency

http://psycnet.apa.org/journals/rev/109/3/492/

http://electronicimaging.spiedigitallibrary.org/article.aspx?articleid=1098143

http://www.uni-kiel.de/psychologie/psychophysik/faul/text_preprint.pdf

https://www.staff.uni-giessen.de/~g61088/pdf/34.transp.pdf

http://poseidon.sunyopt.edu/Zaidi/pubs/Alldocuments/Khang,%20Zaidi%20-%202002%20-%20Accuracy%20of%20color%20scission%20for%20spectral%20transparencies.pdf

http://download.springer.com/static/pdf/283/art%253A10.3758%252FBF03204137.pdf?originUrl=http%3A%2F%2Flink.springer.com%2Farticle%2F10.3758%2FBF03204137&token2=exp=1456740669~acl=%2Fstatic%2Fpdf%2F283%2Fart%25253A10.3758%25252FBF03204137.pdf%3ForiginUrl%3Dhttp%253A%252F%252Flink.springer.com%252Farticle%252F10.3758%252FBF03204137*~hmac=2bf1a1f75055af294cb28e766958e03d6f85e324c60a51b0a0eb0f86d6136d4e

EDIT: And, @m-lohmann, I didn't say thanks for providing the additional information. Thanks!

[rennis250]

?

No worries if you're busy with other things; I fully understand that.

@dcjones, @JeffBezanson, @StefanKarpinski, @yurivish, @SimonDanisch, @stevengj, @lobingera, @timothyrenner, @shashi, @nolta, @vtjnash, @jiahao, @kmsquire, @AndrewGibb, @KelSolaar, @glenn-sweeney

Sorry if I left anyone out, but just grabbing these people from those that were active in previous discussions on these sorts of things.

[rennis250]

Also, more than happy to be proved wrong here. It's not the first time I've been wrong or completely misinterpreted/misrepresented something.

[timholy]

I agree that there is such a thing as perceptual transparency, I guess I'm just questioning whether that's called "color."

If you're keen, I'd say the best bet is to submit a PR to ColorTypes. Then this will be more concrete, and you'll also have a better idea how hard it will be to port other packages (perhaps starting with Colors.jl and Images.jl) to use it.

[rennis250]

Fair enough. I was considering doing that anyway to move discussion along. Actually, I've been asking a few people and peoples' responses range from "yes, that is a color", to "no, it's a transparent object with a color", to "it's a transparent object, but the colors that can appear there are in their own separate class" to "not sure... we need to do more experiments." With more perusal of the literature, I'm finding a similar range of expressions, with people splitting into more or less two main camps. Admittedly, material perception is something I've only recently become active in, so I should have thought a bit more before spreading potential nonsense. :/ Apologies for the noise...

[StefanKarpinski]

@rennis250, if you can come up with a better/more correct term and propose a better/more correct hierarchy of names, we can certainly consider making another transition, but it would have to be fairly compelling at this point, as we already went through this disruptive process quite recently. But do make a proposal and if it gets traction, we can consider whether a PR makes sense.

[lobingera]

Hello colleagues,

i somehow like @timholy s comment:

My head is starting to hurt.

i've always been on technical side of "Color" like in the PDFspec https://wwwimages2.adobe.com/content/dam/Adobe/en/devnet/pdf/pdfs/PDF32000_2008.pdf where Color is a tuple of values applied to the underlying rendering system of certain properties. I fear that @StefanKarpinski s

better/more correct term

will lead to more discussions. From the handling side - putting a color into further APIs - i already mentioned, a 4-tuple including transparency would be nice.

[KelSolaar]

Just read the convo in diagonal, so excuse my poor attempt :)

Right now if I get it properly it is like that:

Colorant ---> Color

What about a generic and broad Stimulus object?

Stimulus ---> Color

The API works with quantities, which imply that they were measured / generated in the first place. The HVS perceives colour when radiation interacts with the eye and that the signal gets further processed. The radiation in this context is defined as a colour stimulus, this is what Colorimetry measures (based on a set of conventions).

[timholy]

a 4-tuple including transparency would be nice

We already have that:

julia> using Colors

julia> c = RGBA{U8}(0.8,0.6,0.2,1.0)
RGBA{U8}(0.8,0.6,0.2,1.0)

julia> sizeof(c)
4

julia> c.alpha
UFixed8(1.0)

julia> C = [c]
1-element Array{ColorTypes.RGBA{FixedPointNumbers.UFixed{UInt8,8}},1}:
 RGBA{U8}(0.8,0.6,0.2,1.0)

julia> reinterpret(NTuple{4,UInt8}, C)
1-element Array{Tuple{UInt8,UInt8,UInt8,UInt8},1}:
 (0xcc,0x99,0x33,0xff)

julia> reinterpret(NTuple{4,UInt8}, [ARGB{U8}(0.8,0.6,0.2,1.0)])
1-element Array{Tuple{UInt8,UInt8,UInt8,UInt8},1}:
 (0xff,0xcc,0x99,0x33)

(Without the array wrapper, julia seems to complain about bitstypes. I'm not sure that complaint makes sense, might be a bug.)

[timholy]

Without a massive PR I don't think we gain much by leaving this open.