MaterialX implementation

[Transcribing here a long previous thread of discussion, for reference].

I wrote some initial notes about the current form of the MaterialX reference implementation:

To summarize, I was concerned that the way the physical layering is expressed in MaterialX doesn't quite capture the intent of the OpenPBR model. The main issue is that the layer operations don't explicitly account for the presence of the volumetric medium inside the layer.

If we look in detail at the sheen, coat, and specular layers, there's some problems representing each with the current MaterialX layer node:

Coat In standard surface, this is represented as (changing names slightly for clarity):

coat_lobe = coat * coat_brdf(...) + lerp(white, coat_color * (1 - reflectance(coat_brdf)), coat) * base

This is actually supposed to represent the coat as an "intermittent" layer on top of the base, where the coat weight is just the presence/coverage weight of the coat, since the above unpacks as:

coat_lobe = (1-coat)*base + coat*coated_base

where

coated_base = coat_brdf + coat_color*(1 - reflectance(coat_brdf))*base

which is like the albedo-scaling top + base*(1-reflectance(top)) form for the coat+base layer. In other words coat_lobe is a statistical mix between uncoated base, and coated base.

In OpenPBR we just write that as

layer(base-substrate, coat, coat_weight)

where base-substrate and coat are slabs of material, and coat_weight is the coat presence weight (i.e. the fraction of surface which is coated).

The coat_color here is approximating the effect of the volumetric absorption in the coat layer. In OpenPBR we say that the color should be the "observed tint color of the underlying base at normal incidence", after accounting for all the light transport effects. In standard surface, the approximation of this is just this multiplication of the base by the coat_color tint.

In MaterialX, the coat is represented as:

coat_layer = layer(top = coat_bsdf,
                   base = thin_film_layer * (coat_color*coat + (1-coat)))

where coat_bsdf has a "weight" parameter equal to coat, which presumably is just multiplied into the BSDF.

First, the operation "layer BSDF A on top of BSDF B" doesn't strictly make sense to me as a physical operation, as BSDFs are not physical things you can layer. In OpenPBR we are careful to define the layering as placing a slab of material, which is the combination of (interface, medium), on top of another such slab. It doesn't make as much sense physically to talk about layering one BSDF on top of another, unless that is just a shorthand for the approximate albedo scaling combination of the BSDFs.

Then the way the volumetric absorption of the coat is accounted for in this formula, i.e. the (coat_color*coat+(1−coat)) factor, is rather artificial. This basically assumes the albedo scaling approximation is being used. Also the presence weight of the coat layer is rolled into the coat BSDF as a multiplicative weight, which also seems artificial as BSDFs don't generically have a multiplicative "weight" factor.

Sheen

In standard surface, this is written down as the combination:

sheen_layer = sheen * sheen_color * sheen_brdf(...) + (1 - sheen * reflectance(sheen_brdf)) * base_mix

This looks similar to the usual base*(1-reflectance(top)) + top albedo scaling approximation, except it isn't quite of that form since top = sheen * sheen_color, but the reflectance term doesn't include sheen_color. In fact this specific combination is supposed to represent reflecting fibres/flakes which produce a colored reflection but do not tint the base. Regular albedo scaling can't do that, as if top is colored this will produce a complementary color tint of the base lobe. Really this combination is supposed to be some loose approximation of a microflake volume with colored flakes but gray transmittance.

In OpenPBR we said that:

any light not reflected after multiple scattering is assumed to transmit to the lower layers (because the microflake volume has gray extinction, the transmitted light will not be tinted by the fuzz)

And suggest the explicit layer combination (matching standard surface):

To represent this as a MaterialX layer operator would require some generalization then, e.g. perhaps a Boolean to specify "whether the base layer is tinted by the complementary color of the coat layer BRDF".

Specular

The specular lobe in MaterialX is represented as:

specular_layer  = layer(top =  specular_bsdf,
                        base = transmission_mix)

transmission_mix  =  mix(fg = transmission_bsdf,
                         bg = opaque-base,
                         mix = transmission)

but (in OpenPBR anyway) this is supposed to be representing a dielectric interface (where specular_bsdf is the BRDF, and transmission_bsdf is the BTDF). It's not physical in general to think of this as an actual layer, the form above only really makes sense in the albedo scaling approximation, where it is then just roughly approximating the balance of energy between the dielectric lobes. If people took this layer seriously as a physical description, it would be unclear what it means except as a shorthand for albedo scaling, which defeats the purpose of trying to define a layering operation as something more general than albedo scaling. I think this specular reflection lobe should actually be written explicitly as the sum of BRDF and BTDF lobes, not artificially as a layer operation.

Overall I think it's quite difficult to map the formal layer/mix structure in OpenPBR into corresponding abstract layer operations that produce a well-defined implementation. In my view it's much easier to work with something like the standard surface form (or the analogous form of OpenPBR) where one is just evaluating a mixture of closures/lobes, with well-defined mixture weights, which is explicitly implementing a certain (actually perfectly acceptable, for VFX) approximation. Then there is no need to figure out how to carefully craft the layer API so that implementers will be able to reproduce the mixture model you want them to, you just give them the mixture model.

Alternatively, if we must go the route of using layer operations, this needs to be generalized appropriately, though as described it would probably have to be designed quite carefully in order for implementers to be able to make sense of it. (Or you tell implementers how to do this in a reference implementation, which then probably just has the form of the mixture model, which makes the intermediate layer description a bit redundant).

My thought about how we could possibly generalize the layer operation to do what is needed is allow that:

the layer can have a coverage/presence weight in [0,1] defaulting to 1.
the layer transmittance (or optical depth) can be specified, thus tinting the base. I think using a depth value would be quite artificial as these layers are conceptually infinitesimally thin (or at least, we don't care about the actual depth, we care only about the optical depth).
- we specify whether the base layer is tinted by the complementary color of the coat layer BRDF (e.g. true for specular layer, not true for fuzz layer). This is a proxy for the details of the scattering/absorption processes inside the layer. Though you can see this is not fully general, and basically just designed to recover the mixture model we are looking for in the end.

AcademySoftwareFoundation / OpenPBR

MaterialX implementation #86