More general colorSpace support for Basis?

[spiffmon]

Hello, and kudos on the fantastic design and performance of basis universal!

I represent Pixar's Universal Scene Description (USD) project, and we would very much like to support basis textures in the USDZ package format, and in USD more broadly. While there aren't any barriers to us doing so, there is a "standards complexity" that we, and some of our strategic partners share.

We strongly believe that the 3D community will best be served in the long-run if 3D formats for content delivery and for high-quality content creation are at least easily convertible, if not actually the same format. In content-creation pipelines it is common to create textures in multiple colorSpaces, and those colorSpaces represent meaningful data that we may not want to lose, even for final asset packaging.

Conceptually, in M&E, at least, colorSpaces standardize around ACES, and software support via OpenColorIO, which does provide some facilities for GPU transformations. But I am not an expert in this domain, and will leave it to others to discuss.

Without the ability to specify colorSpaces in basis, it is likely that USD will need to support basis for glTf compatibility, but also something like AVIF for good compression that accommodates generalized colorSpaces... but which lacks (currently) GPU decompression support. And that means consumers (and creators) of USDZ will need to be prepared for either format, also.

Thanks for listening! --spiff

[MarkCallow]

Take a look at the KTX v2 file format. It supports Basis Universal encoding and stores color space information (primaries and transfer function).

[spiffmon]

Thank you, @MarkCallow , I see. Since the doc states that transfer function must be either srgb or linear, we are then limited to using the colorPrimaries to encode any more particular transformation, correct? So if a source image were encoded in a general OCIO colorSpace, e.g., there may be no way to preserve that encoding, requiring to first transform the image into a colorSpace whose display transform can be represented by a matrix multiply?

Secondarily, is there any potential for standardizing Key/Value pairs for (eg. OCIO) color Configuration and ColorSpace, when one is representable without pre-transformation? This would be useful if one were trying to span delivery and interchange with one set of textures.

Cheers and thanks!

[MarkCallow]

@spiffmon, the reason for the transfer function limitation is because only those are currently supported by GPUs. Others could be supported by doing decoding and encoding in shaders but it is impossible to get correct filtering and blending. The latter is possible if you have one of the rare GPUs that has programmable blending or framebuffer fetch. The former may be possible with texture gather and a shader but I haven't thought about it in depth.

Given the way the HDR market is moving it seems likely that something like HLG support may be added to GPUs in the near future. Because of this market we have an open issue to relax the transfer function limitation. I expect this relaxation to happen soon.

Yes there is potential to standardize key/value pairs for color Configuration and ColorSpace. You can provide your own specification, just don't prefix the names with KTX or ktx. Alternatively you can work with Khronos and we can create KTX prefixed names. Initially you can file an proposal as an issue in the KTX spec. repo which we can discuss in the 3D Formats WG. It is also possible to set up a liaison between your group and Khronos if in depth discussions between us will be needed.

[spiffmon]

The rationale is clear and sensible, given KTX’s primary mission, @MarkCallow . Thanks for the pointers to possible future direction; I’ll take this information back to the interested folks in the USD community, and get back to you. It looks like KTX2 has alot to offer!