Open LeaVerou opened 1 year ago
Yes, we need to have blending and compositing available.
The W3C Compositing spec is old, RGB-only, and does compositing in a broken way which happens to be compatible with what browsers do and Adobe Photoshop does, or used to do.
It works in sRGB (or technically any RGB space) but luminance uses the old NTSC primaries (yes, really)
Lum(C) = 0.3 x Cred + 0.59 x Cgreen + 0.11 x Cblue
That means we need to calculate a separate "CSS blend luminance" which is not Y in CIE XYZ /facepalm
https://drafts.fxtf.org/compositing-1/
Be careful when choosing names, because we want to be able to add the blend-modes and source-over
is the usual Porter-Duff name for what CSS blending calls normal
which is the initial value of mix-blend-mode
.
Is there any reason for it to take more than 2 arguments? My understanding is that the operator is associative.
Like mixing, compositing is defined on two arguments and compositing more than that requires specifying and order of operations for several two-argument steps.
In addition to all that broken compositing in gamma-encoded sRGB space we would also want to make available compositing in linear-light xyz-d65
which is what is used for commercial 2D and 3D graphics.
Let's focus on alpha blending for now please. Sure, we shouldn't make any decisions that make it harder to implement the other operators and blending modes later on, but the immediate need is for alpha blending, that is far more common than any other operators, and we don't need to figure out the API for blending modes to add alpha blending.
The W3C compositing spec is old, sure, but surely this is a solved problem in the literature? So we convert to XYZ and do it there, then convert back to the color space of the first color?
surely this is a solved problem in the literature? So we convert to XYZ and do it there, then convert back to the color space of the first color?
Yes, that is the correct way.
It doesn't predict the actual contrast you will get in a browser, of course.
the immediate need is for alpha blending, that is far more common than any other operators
All of the blend modes are doing alpha blending.
Sure, we shouldn't make any decisions that make it harder to implement the other operators and blending modes later on
Exactly. So the correct approach is to implement the General Formula for Compositing and Blending:
Apply the blend in place
Cs = (1 - αb) x Cs + αb x B(Cb, Cs)
Composite
Co = αs x Fa x Cs + αb x Fb x Cb
and then provide useful defaults.
For the blend operator B, the default is normal
which is simply
B(Cb, Cs) = Cs
and for source-over compositing,
Fa =1 Fb = 1 – αs
Let's focus on alpha blending for now please.
As someone that has already gone down this road, if you add support for source-over
compositing and normal alpha blending, you have everything you need to do the others, you can just swap out different compositing and blend methods. So the key is just to make sure the compositing step and the blend step is generic. I think that is the main point that is being conveyed.
Though I had some missteps in the beginning 😅, I can pretty much mimic what browsers are doing now.
@svgeesus But in which color space? Will XYZ work well here? How does this change once we support CMYK spaces?
@facelessuser I see your point, what I meant was, from an API design pov, alpha compositing should be quick and easy to specify, not just a parameter of a more general compositing API.
Though I had some missteps in the beginning 😅, I can pretty much mimic what browsers are doing now.
What missteps did you have?
But in which color space? Will XYZ work well here?
For browser-compatible simple alpha source-over blending, premultiplied sRGB will work (oog values may need to be clipped, needs investigation of how the equations react to extended values)
For browser-compatible other blend modes, sRGB plus a bogus NTSC-luma will work; the luma is only needed for the non-seperable blend modes:
For higher quality physical light normal blending and source-over compositing, premultiplied XYZ-D65 will be fine
For compositing in CMYK then a) are you crazy and b) read the PDF spec for the full horror and c) have a drink with @faceless2 to get extended horror. Also to actually do compositing that is fully general and includes some CMYK then we would need ICC support to be able to get the Lab values. So, not for now.
From an API design pov, alpha compositing should be quick and easy to specify, not just a parameter of a more general compositing API.
An API design would have two methods, blend and composite. Each would take the source and destination colors as parameters plus an optional options object where one could specify the blend mode and the compositing operator and the blend colorspace, and these would default to normal
and source-over
and either xyz-d65
or srgb
respectively.
For browser-compatible simple alpha source-over blending, premultiplied sRGB will work (oog values may need to be clipped, needs investigation of how the equations react to extended values)
For browser-compatible other blend modes, sRGB plus a bogus NTSC-luma will work.
For higher quality physical light normal blending and source-over compositing, premultiplied XYZ-D65 will be fine
Let's use XYZ-D65 then, and allow for a parameter if people want to produce shittier results that are browser-compatible (or not, what's the use case?).
For compositing in CMYK then a) are you crazy and b) read the PDF spec for the full horror and c) have a drink with @faceless2 to get extended horror. Also to actually do compositing that is fully general and includes some CMYK then we would need ICC support to be able to get the Lab values. So, not for now.
Of course we'd need ICC support to do CMYK properly, but that's in scope for the future, and wanting to overlay a CMYK value with transparency over another is not a crazy ask. Once we do have Lab values, does alpha blending work the same way?
Once we do have Lab values, does alpha blending work the same way?
For the proper way: yes. Lab-D50 → XYZ-D50 → XYZ-D65 and then as normal
For the web-compatible way, Lab-D50 → XYZ-D50 → XYZ-D65 → srgb-linear → sRGB and hope
What missteps did you have?
EDIT: Just to clarify, clamping αo is specifically related to premultiplication as well, and in my case, I was using it to undo premultiplication on each channel without first clamping it to a realistic 0 - 1 range. Basically, all my issues were related to undoing premultiplication.
From an API design pov, alpha compositing should be quick and easy to specify, not just a parameter of a more general compositing API.
An API design would have two methods, blend and composite. Each would take the source and destination colors as parameters plus an optional options object where one could specify the blend mode and the compositing operator and the blend colorspace, and these would default to
normal
andsource-over
and eitherxyz-d65
orsrgb
respectively.
color1.composite(color2)
is not an intuitive way to do alpha blending, which is by far the most common kind of compositing. I do think we should have a separate method for alpha blending (which can internally be a shortcut to compose
or whatever once we have that). I laid out some options in the first post for what that function can be named.
I personally always liked the name overlay
. I figured if enough people ever complained that compose
was too obtuse for basic alpha blending, that is what I'd add. For me, it creates a good mental picture of what is being attempted, but I can certainly understand the idea of using over
with its tie to the operator name. If it were me, I'd vote for overlay
or overlayOn
.
I started a draft PR yesterday, so we can iterate: #231 Still needs tests
I personally always liked the name
overlay
.
Note that overlay
is the name of a specific blend mode defined in CSS which has some unusual/strange non-linear properties.
To avoid confusion, the name "overlay" should not be used for a function which performs normal alpha blending.
... the name "overlay" should not be used for ... normal alpha blending....
I agree. "over" is usually the correct term. Overlay is a specific blend mode. I believe the PR is using "over".
Alpha in video, and mattes in film, were oft treated like black-magic potions with arcane setups and workflows... in the transition from chemical imaging to digital, the old voodoo methods were dragged along, and strange monsters under-the-bed grumbled as flows shifted to linear space blendings...
If you are doing everything "in house" with no inter-facility interchange of files, then alpha issues are less likely to bite you -- however, interchange is common, and understanding what/how alpha is interpreted in various use cases across color spaces is key. (yea, that pun was intended).
Also, I hope this is the right thread for this post, feel free to move if it isn't.
The statement "alpha has no gamma" is technically true, but also not correct.
While the alpha channel just specifies a percentage of transparency, a specific percentage of transparency will not have the same perceptual effect among different color spaces and most especially, gamma vs linear tone response curves.
The reason for the history lesson in the above twisty, was to show that compositing operations were handled in non-linear spaces for a very long time, and people became accustomed working that way. Linear does have advantages, but it's also not always the ideal working space, depending on the specific task.
Linear is good for a lot of compositing, but not so good for gradients, and of questionable utility for color grading, where you'd rather have controls that are perceptually uniform as opposed to illumination uniform.
The fact that compositing is good in linear, but perceptual spaces are better for some other things, brings us to the actual problem of "how best to handle alpha blends", which is answered with "it depends".
If you are preparing an element in linear workspace, and that element is going to be exported with an alpha channel, it is important to know the kind of space it is going to be used in. The element is perhaps going to the DI, and what space are they grading in? P3? $X^{\prime}Y^{\prime}Z^{\prime}
$ ? Rec709/2.4?
Let's say you are sending text elements with alpha. How you handle the alpha, and how the recipient handles the alpha, depends on your working space gamma, the interchange file format, and the working space & gamma of the recipient. For these examples we'll assume premultiplied alpha.
Okay, the tidbit to gleen for the foregoing is that:
Put another way, there is no "best" default for the alpha in a multi-space environment, there is only the "it needs to be handled like this" where "this" is the space the alpha was created or adjusted for.
The following table does not imply best practices, only common cases and potential flows. | Source Working Space |
Intermediate File Specs |
Destination Working Space |
Alpha Requriements |
---|---|---|---|---|
$\gamma 2.6 $ |
ProRes4444 | $\gamma 2.6 $ |
Same alpha as adjusted in source workingspace | |
Linear $\gamma 1.0 $ |
EXR (lin) | Linear $\gamma 1.0 $ |
Same alpha as adjusted in source workingspace | |
Log | DPX + TIFa | Log | Same alpha as adjusted in source workingspace | |
Linear $709 \gamma 1.0 $ |
ProRes4444 $\gamma 2.4 $ |
$\gamma 2.4 $ |
Alpha adjusted to work with the $ \gamma 2.4 $ transformed image data |
|
Linear $P3\ \gamma 1.0 $ |
ProRes4444 $\gamma 2.2 $ |
$DCI\ P3\gamma 2.6 $ |
Alpha adjusted to work with the $\gamma 2.2 $ transformed image data in the ProRes, and then likely adjusted further for the final comp 2.6 |
...Be careful when choosing names...
over
is appropriate, and common term for the basic alpha "sticking things on top of other things".
...So we convert to XYZ and do it there, then convert back to the color space of the first color?...
As I tried to illustrate above, it depends. What space is the alpha intended for? If the alpha is attached to a gamma encoded image, then usually that alpha is intended for providing transparency for that image data at its current gamma. Usually.
And of course, this applies only to alphas with partial transparency. It does not apply to alpha 0 or 1. Text for instance has antialiasing, and this is partially transparent. It needs to be composited using image data and alpha at the gamma it was built for.
The color space that the image+alpha was built/intended for first, or if the image data is transformed to another space, then a transform on the alpha as well to match perceptual intent.
XYZ is essentially an RGB space with imaginary primaries, so long as fg/bg are both in XYZ, I suyppose it should work—but keep in mind that as linear spaces with imaginary primaries, it's easy to exceed the gamut of the actual destination space.
Opinion: I like working spaces that are the same as the destination space, or at leas the intermediate space. For additive chromatic spaces like RGB, they can easily be linearized, if linear blending is desired.
...How does this change once we support CMYK spaces?...
Not being snarky: convert CMY to RGB to blend, then back. K is reserved or used in some blend modes. You might like this paper on blend modes.
Some simple blend modes might (big maybe) be emulated while staying in CMY. I've tried doing subtractions for instance, though trying to find a workable CMYK blending space may be a rich-black alley.
RGB is already an additive space, and three primaries where each brandishes a share of the luminance. CMYK as a subtractive space with 4 colors including one that modulates luminance much more than the others adds non trivial complexity. As a result, Grassman's laws apply to RGB but not so much to CMYK. I.e. you can get straight-line mixes in RGB, but you don't get straight line mixes in pigments or inks.
And you need a perceptually uniform space to transform CMYK to RGB... so now. thinking out loud, could a perceptually uniform CMYK mixing space be created... and is there a compelling reason to do so?
For those that read this, thank you for joining me in this trip down memory lane...
I was looking for this feature, missed it, and then found this issue. As far as I can tell, right now Color
objects represent alpha, but don't do anything with it.
I have to admit that I don't really understand the intricacies of different methods, but for my direct practical application I would like a method like color.blendAlpha(background)
, which transforms a color with alpha < 1 into a color with alpha = 1, simulating the way the browser would do it if an element with color
is on top of an element with background
.
But I guess that isn't easy either, because different browsers do it differently, and have done it differently historically?
We should have a method for alpha blending. This is required to be able to do contrast calculations properly (see https://github.com/w3c/csswg-drafts/issues/7358 ).
Things to decide:
overlay()
orover()
? OroverlayOn()
or juston()
which is more clear as aColor
method, but not so much in the procedural API (SeeoverlayOn(color1, color2)
vscolor1.overlayOn(color2)
). Whatever we come up with should make sense in both APIs, so I’m leaning towardover()
which is also the operator name.The math seems pretty straightforward. @svgeesus are there any considerations when applying this to other color spaces? What color space do we use for the result? I guess the color space of the color being overlaid?