search

atteneder / KtxUnity

Load KTX and Basis Universal textures at runtime
Apache License 2.0
215 stars 42 forks source link

HDR (RGBD PNG) lightmap transcoding #65

Closed daverin closed 2 years ago

daverin commented 2 years ago

Hi Atteneder

This may lead to a really cool feature for KtxUnity, but mostly its a request for unity help.

My team is currently trying to get HDR lightmaps into unity through glTFs. The way we're going about this is to encode hdr/exr files into RGBD PNGs. In summary, the alpha value of the PNG is used as a divisor to get larger HDR values during transcoding. RGBD is the recommended approach to IBL as mentioned in EXT_lights_image_based.

I have here the HDR lightmap, its corresponding RGBD PNG, and its wrapped KTX2 image that I have been testing with.

I figured that the best place for this logic would be here:

https://github.com/atteneder/KtxUnity/blob/88edcafac793ee5234af6441ca83aa3b98916b9e/Runtime/Scripts/KtxNativeInstance.cs#L178

My problem comes when trying to transcode this into an HDR supported texture format in Unity. My options are between RGB9e5Float and RGBAHalf. I am able to load my own data into an RGBA32 texture pretty easily with the Color32 struct...but I cannot find a corresponding struct or sample code to load data into a RGB9e5Float or RGBAHalf texture. I feel like I am missing something trivial and the unity docs aren't making things clearer for me.

Any help would be greatly appreciated 🙌🏾

daverin commented 2 years ago

Managed to resolve this by using setting the ktx transcode format to RGBA32, then transcoding that data into a RGBAFloat texture using a Color struct.

atteneder commented 2 years ago

Hi @daverin. Thanks for sharing and I'm glad you found a solution.

I hope KTX will be utilized for HDR textures (and ideally in a compressed manner) soon and I'm certainly interested to add support for it to KtxUnity.

daverin commented 2 years ago

Hi @atteneder

I switched from RGBAFloat -> RGBAHalf and managed to reduce my memory footprint by half, though I still wasn't satisfied with it.

My final solution was to do the division operation at the shader/material level. I figured that this way I fully capitalise on the magic of KTX compression. This way also allows me to do texture streaming with the lightmaps. Works like a charm 🧙🏽‍♂️