Single channel textures in .obj and .glb / .gltf files not working as expected

[roym899]

Describe the bug When opening these meshes, the obj file has no texture and the glb / gltf files either show up red or give an error. In Open3D all meshes show up red. So maybe something with the meshes is off? However, they work fine in f3d / Blender / MeshLab (all meshes were exported from a DAE / COLLADA file via Blender; there were similar issues when using trimesh to do the conversion; source mesh is also included in the zip):

Format	Rerun	O3D	MeshLab	f3d	Blender
obj with mtl and png texture	white w/o texture	red	works	works	works
glb	red	red	no support	works	works	works
gltf + bin + texture	error	red	no support	works	works
gltf embedded	red	red	no support	works	works

The error for gltf + bin + texture is

Failed to load mesh "/\\//home/leo/rerun_material/meshes/gltf\\+bin\\+textures/mesh.gltf": external reference in slice only import

To Reproduce Drag & drop the attached files into the Rerun Viewer.

Expected behavior Ideally, show the textured mesh in all cases. If there's an issue with the mesh leading to missing texture and red material (?) it would be nice to get some information about the problem.

Screenshots No texture (obj in Rerun)

Red (glb in Rerun)

"Correct" (obj in MeshLab)

Desktop (please complete the following information):

• OS: Ubuntu 20.04

Rerun version

rerun_py 0.12.1 [rustc 1.74.0 (79e9716c9 2023-11-13), LLVM 17.0.4] x86_64-unknown-linux-gnu release-0.12.1 bef0d78, built 2024-01-17T16:50:34Z

[roym899]

I think the red color has to do with the texture for this mesh being single channel.

[Wumpf]

To extend on that, the issue is concrete that we're using TextureFormat::R8Unorm as a format here which as the name implies is a red channel texture.

To make matters worse, we actually also should probably take the OETF (gamma curve ;)) into account: Gltf specifies that any information in the texture data should be ignored and we should always assume the data is OETF encoded https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#metallic-roughness-material It specifically talks about RGB values though, so it's a good bet this applies. The tricky bit of all this is that we quickly approach the the complexity of our tensor decoding and I believe we don't want to be that well-supporting on the mesh for the time being until we're in a situation where we can easily convert "tensors to textures" on the gpu as a separate, but still on-the-fly step. I.e. in the limit we'll need the ability to map all of our tensor decoding to a texture that also includes mipmapping, allowing us to have a very classic render pipeline from that step on and separating these concerns!).

That said, it shouldn't be too bad if for this unfortunate case of single-channel textures we just do the texture sampling in gamma space and convert it afterwards.

[jleibs]

Not sure if it's covered by the above, but it's also been noted that compressed meshes don't work.

If you try it yourself, take any GLB file, compress it on https://optimizeglb.com/ and try to load it again in Rerun it will fails, but work fine on any web glb viewer

[Wumpf]

created a separate issue for that here since it's orthogonal to the texture problems described in the ticket here https://github.com/rerun-io/rerun/issues/6365

[emilk]

Very related to:

• https://github.com/rerun-io/rerun/issues/7056