Fixing Coloramity

[timholy]

In an ideal world, one would be able to:

• load an image from a disk file
• visualize it using using OpenGL
• perform some calculations on it using Image's algorithms
• plot the results in Gadfly

Unfortunately, currently we are far from living in this world. It seems that the biggest problem is the separation between this package (Color) and ColorTypes.jl. I'm running into this in the course of trying to split out functionality in Images into the JuliaIO packages. Overall I agree that splitting this functionality is a good thing: for example, it would be great to largely divorce Images.jl from ImageMagick. The problem is that the ImageMagick.jl package started by Simon (using code lifted from Images) uses ColorTypes, while Images (and essentially all other packages) uses Color, so the GL&JuliaIO world is completely incompatible with the rest of the package ecosystem.

@rant begin
    I'd like to propose that henceforth if one snips out a chunk of functionality
    (especially, types) from one popular package into a standalone package, it's
    incumbent on the programmer to issue pull requests rebasing the
    "parent" package on the new standalone package. This includes getting
    all the parent's tests working. If you're not willing to do that, it's better to use
    the parent package unmodified.
end

I have a proposal for fixing this, and I'm (grumpily) willing to put in the (considerable) time needed to make it happen:

• @simondanisch presumably needs to base color types on FixedSizeArrays.jl for smooth interoperability with GL. Consequently, I propose we keep his design of having a standalone ColorTypes package that only encodes the basic color type definitions, and that it be based on FixedSizeArrays.
• We make some changes (see below) to the design of ColorTypes to make it a little more like Color.jl. We also pick names that are less awkward than our current names.
• We rebase Color on top of ColorTypes, naming the new package Colors.jl. We issue a deprecation warning for using Color.
• To ensure that people actually use the new package, I submit the necessary pull requests against Cairo, ColorBrewer, Escher, Gadfly, Gtk, Images, ImageView, ProfileView, PyPlot, Tk, and Winston. There are a few other packages that use Color, but I can't fix everything, and these are the main ones I've heard of. It's up to @simondanisch to update the GL packages.

CC @lobingera, @timothyrenner, @shashi, @dcjones, @vtjnash, @stevengj, @nolta, @simondanisch, @jiahao. I'd like feedback from most/all of you before undertaking this course, since it will be a fair amount of work and I'd like to have some confidence that the PR I submit will be accepted.

Proposed type hierarchy sketch for the new ColorTypes:

using FixedSizeArrays

# Paint means either Color or Color+Transparency
abstract AbstractPaint{T, N} <: FixedVector{T, N}

# AbstractColor means just color, no transparency
abstract AbstractColor{T, N} <: AbstractPaint{T, N}
abstract Color{T} <: AbstractColor{T, 3}
abstract AbstractRGB{T} <: Color{T}

# Types with transparency
abstract Transparent{T,N} <: AbstractPaint{T,N}
# Because storage order matters for some applications, we have both
# "AlphaColor" and "ColorAlpha"
abstract AbstractAlphaColor{T,N} <: Transparent{T,N} # storage order alpha-color
abstract AbstractColorAlpha{T,N} <: Transparent{T,N} # storage order color-alpha

# Concrete types
immutable RGB{T} <: Color{T}
    r::T
    g::T
    b::T
end

# ...and so forth

immutable AlphaColor{C<:Color,T} <: AbstractAlphaColor{T,4}
    alpha::T
    c::C   # C really has to be Color{T} (use inner constructor here)
end

immutable ColorAlpha{C<:Color,T} <: AbstractAlphaColor{T,4}
    c::C
    alpha::T
end

# Grayscale
immutable Gray{T} <: AbstractColor{T,1}
    val::T
end

immutable AlphaGray{G,T} <: Transparent{T,2}
    alpha::T
    c::G
end

immutable GrayAlpha{G,T} <: Transparent{T,2}
    c::G
    alpha::T
end

[timholy]

While it might need some tidying up in terms of spacing, here's the proposed new naming scheme:

It's a little ironic that Color is not in the hierarchy, given that it was a motivation for renaming to Colors, but there you have it. And it might reduce bugs. (I got a weird precompilation error where it looked like the typename Color was being confused with the module name Color.)

[StefanKarpinski]

You could make the apex Color without too much confusion, but I'm not sure if it's worth it.

[timholy]

I thought about that and could be persuaded. But I worry it might encourage folks to assume it means OpaqueColor{T,3}. It's likely that I'm strongly influenced by code history, though, so I'm not sure I have the best judgement.

[stevengj]

+1 for making the apex Color

[kmsquire]

+1 for making the apex Color

[timholy]

That's enough "yea"s for me, Color it will be. Tomorrow morning :smile:.

[shashi]

The 3 packages keep things extremely neatly layered now. And awesome - Color is finally happening!

Really appreciate the Tour de force!

[dcjones]

I'm quite happy with this. Nice job Tim and Simon!

I'm not a big fan of Color as the apex type nor the name OpaqueColor, but admittedly for pedantic reasons. RGBA, for example, is a color plus an alpha channel, which is not a color per se. And the correct term to describe what OpaqueColor encompasses currently is "color". The proposed type hierarchy feels slightly like it's shoving the colorimetry (which was the original purpose of this library) in the corner.

My preference would be an apex type called Colorant (Paint was ok too), then rename OpaqueColor to Color.

[StefanKarpinski]

Is "colorant" a somewhat standard term? It is in the dictionary: "a dye, pigment, or other substance that colors something." That seems apt.

[dcjones]

It's not standard as far as I know. It's hard to find prior art because I don't know of a comparable library; most are thoroughly graphics oriented or thoroughly colorimetry oriented.

[StefanKarpinski]

That's actually a very good thing. I suspect that custom types and external dispatch are quite necessary to satisfy both sets of needs with a single hierarchy. Maybe what is needed is a non-technical term since all the other terms are taken and mean something else.

[lobingera]

@dcjones

The transparentColor vs. OpaqueColor makes some sense in compositing, as opaque colours replace what is already 'painted' and transparent colours mix with the paint on the canvas. This differentiation is seen e.g. in the set_source_rgb vs. set_source_rgba calls for cairo.

For plotting i sometimes like to be able to choose between opqaue and (half-)transparent colours. I understand you like to deal in Colors with color perception and currently the discussion (see ColorTypes) is more from how to technically deal with values.

[timholy]

@dcjones, I will change those names. I more than half-expected this to be your view of it, and I think getting the names technically correct is a good thing.

I do wish more of the colorimetry experts would chime in early in the game. I'm CCing two other folks, @glenn-sweeney and @m-lohmann. Along with @dcjones, would you please click "watch" on both https://github.com/JuliaGraphics/ColorTypes.jl and https://github.com/JuliaGraphics/Colors.jl, since those are where an increasing amount of the discussion is/will be happening?

[StefanKarpinski]

Despite the reputation that bikeshedding has, getting names right once and for all is very important.

[timholy]

Perhaps the most important part of this transition, in fact. (But not the most work; that honor is easily captured by the traits system.)

[AndrewGibb]

I've been lurking around Julia a while now. I work for BBC R&D and whilst colour isn't my area of expertise, I spend a lot of time discussing all aspects of video with those who are, so perhaps my input would be valuable here.

First, sincere thanks to Tim and Simon for the work. It is the continued enthusiasm by the core Julia developers to do the dirty work which makes it so appealing as a platform.

I would support @dcjones recommendations for the names in the type hierarchy. (I see now this change is already reflected in ColorTypes.jl). Of all those discussed, those names are the clearest to me.

I also support the decisions about separating ColorVectorSpace. The relevant READMEs make it very clear what the different packages are for.

Rec709 (from which sRGB derives its colour primaries) defines a 10 bit representation of colour value, where the interval 0 to 1 maps to 64 to 940. As this is a somewhat specialised use case, perhaps I should add it myself.

One suggestion for an enhancement: Might it be useful to have a guide to adding additional colour space? Obviously Tim has captured many, but I know feature film cameras use different colour spaces, for example. I can't think of any other use cases, however.

[timholy]

Thanks for commenting, @AndrewGibb. If you want to add Rec709, the best approach would probably be to add the corresponding number type to FixedPointNumbers.

The idea of adding an extension guide is a good one.

[timholy]

https://github.com/JuliaGraphics/ColorTypes.jl/pull/5

[timholy]

Done.

[SimonDanisch]

:100:

[rennis250]

I haven't had a chance yet to read through this discussion, but I'm not sure I would put colorant at the apex of the type hierarchy, as pure light sources also produce color percepts, as does pressure on the surface of the eye. Not to mention that some colors are produced without the presence of a physical stimulus (e.g., percepts of blue at the fovea). The current choice seems to put too much weight on the idea that colors are the result of surfaces alone, which is not really the case. Indeed, a colorant alone will not produce a color percept and requires the presence of an emitting light source, the choice of which can have a large influence on the perceived color, as light is essential and color is ultimately a perceptual process, rather than an external physical entity.

[timholy]

This has been merged, many packages are now using it, and I am off working on other things. So at this point, anyone who wants further redesign has to do the work him/herself :smile:.

[rennis250]

I have no problem with helping out and implementing changes, in face I am happy to do so. I just wanted to put that out there to see what people think.

Best, Rob

[timholy]

Hard to say without a more concrete sketch of your idea of a redesign---my reaction could range anywhere from "great!" to "more appropriate for a LightPerception package."

But glad to hear that you're willing to help make julia better, as always.

[rennis250]

Actually, I should have started my post with a huge thank you, rather than being pedantic, especially when I haven't been active in the Julia community for so long. So, my apologies; that was rude of me. I really like the work that has gone into the color packages since I've had a time-out.

I will try to post something within the next days. I've always felt that it is unfortunate that there has been such a heavy split of color terminology over the years in industry, physiology/psychophysics, and phenomenology research. Each of these subfields shares some terminology, but not all, and it can be difficult within psychophysics/physiology, where researchers sometimes introduce new terms for concepts that phenomenologists already labeled and made clear in the past. The huge divide between the science of color and the arts also doesn't help.

Best, Rob