RGB images get converted to RGBA images before getting uploaded to the GPU,which is causing a huge performance drop

[zrezke]

Description

Displaying RGB images for the first time is very expensive as they get converted to RGBA before being sent off to the GPU. I did some profiling with a 4k video stream where this issue really becomes apperant:

Suggestion to resolve this issue

Instead of using the TextureFormat::Rgba8Unorm, maybe we could use a texture format like TextureFormat::R8Unorm and upload the RGB data directly to the GPU inside of this texture, then do some extra work in the shader to correctly sample an R8Unorm as an rgb texture.

If this performance issue get's resolved this would be really useful especially for live streaming data, where you really want to minimize the time wasted loading and converting the data...

[emilk]

This looks surprisingly slow to me - is this a debug build?

[zrezke]

Not sure what I was running at the time, I tried it again today with a fresh pip install:

rerun --version
rerun_py 0.6.0 [rustc 1.69.0 (84c898d65 2023-04-16), LLVM 15.0.7] x86_64-unknown-linux-gnu release-0.6 643dea9, built 2023-05-25T20:35:24Z

The times for pad_rgb_to_rgba range from ~6.5 ms to ~13 ms.