sRGB colors

[lilleyse]

Hey @wallabyway - I've been digging more into glTF point clouds myself and I'm wondering if you came across this issue.

Most of the LAS files we get seem to be in non-linear color space, I assume because the color is really coming from non HDR photographs that are captured at the same time as the lidar. The LAS spec doesn't mention color spaces specifically but in my experience 99% of LAS files are like this.

Meanwhile in glTF vertex colors are expected to be in linear space. Before I read that, I was hoping that we could store 8-bit sRGB colors in the glTF and render them directly to the screen, which seemed like a win-win for small file sizes and fast shaders. Now it's looking like we'll need to store colors as 16-bit fixed point or 32-bit float in linear space and convert back to sRGB in the shader. I thought about storing 8-bit linear space colors but that seems like a bad idea due to quality loss.

In https://github.com/KhronosGroup/glTF/issues/1638 there was some talk about introducing an extension for sRGB-encoded vertex colors and that would be very relevant for point clouds. What do you think?

[wallabyway]

@lilleyse - Hi Sean. I've been mulling over this, sorry for the late response.

Here's my initial thoughts: For the AEC use-cases, the visual difference ('banding artifacts' or 'crushed blacks') between 8bit linear-RGB and 8bit sRGB color space, will be overlooked, because of the sparse nature of the points and the EDL effects applied.

However, I do like the idea of an sRGB extension, where LAS 8bit sRGB, is stored inside the glTF vertex buffers "as is". It reduces loss, conversion processing and can simplify the 'unlit material point-cloud shader'.

I'm not a big fan of converting to linear RGB float32 (haven't tried 16bit fixed pt), even though my testing with MeshOpt compression, seems to find the redundancy and produce a file size, that's about the same size as an 8bit uchar version.

So, for the AEC use case, of the three options...

1. sRGB-8bit extension (lossless)
2. convert sRGB to 8bit-linear RGB (lossy)
3. convert sRGB to high bit precision (16/32) and rely on compression

... I'm leaning towards number 2.

If the glTF folks have plans for an sRGB extension anyway, then I'd definitely switch to 1 (but I'll leave @emackey to comment)

[emackey]

I could see a case for such an extension when there's unlit, untextured points. If a texture is in use, things get more complicated because there may be an sRGB hardware decoder on the texture. Perhaps the app would turn that off for unlit cases?

I think there may have been casual talk, but I'm not aware of any proposal for an 8-bit sRGB-encoded vertex color extension. Sounds like the GIS community could propose one though.

[lilleyse]

Once I add meshopt / draco to the pipeline I'll report back with numbers to compare.

My main hesitation with proposing an extension is not getting enough adoption, and then colors look wrong in certain engines. Will have to weigh the tradeoffs...

[emackey]

and then colors look wrong in certain engines

Fragmentation is a good reason to hesitate. But if you do go forward with an extension, we have an established pattern that I recommend you follow: The extension body supplies the (replacement) ID of the accessor with the sRGB vertex colors.

This is a little trickier with vertex attributes than with other accessors, but in this case I think it would mean replacing COLOR_0 with something like _COLOR_SRGB. The leading underscore indicates a custom attribute, and the extension would call it out as the sRGB vertex colors to use. Engines that don't support it would either fall back to a linear COLOR_0 if one was included as a fallback, or would revert to white.

[emackey]

Probably it's worth checking how bad option 2 from the original post would be for typical point clouds, before going down this rabbit hole. Given that these are separate points anyway, banding might not be as noticeable.