Transparent objects are rendered incorrectly depending on lights and order they are added to a scene

[Qgel]

I am trying to render two transparent objects which are 'behind' each other, such as a transparent cube withing another transparent cube with the top cut off, see example below.

However, depending on the order I add the meshes to the scene, and also somehow on the lights, the back object (from camera perspective) sometimes incorrectly does not render through the transparent front object.

The following code can be used to re-produce the issue:

import trimesh
import pyrender
import trimesh.transformations

cube_outer = trimesh.creation.box(extents=[1.2, 1.2, 1.2])
cube_cutout = trimesh.creation.box(extents=[1.0, 1.0, 1.0])
cube_cutoff = trimesh.creation.box(extents=[2.0, 2.0, 2.0]).apply_translation([0,0,1.5])

cube_hollow = trimesh.boolean.difference([cube_outer, cube_cutoff, cube_cutout])

mat_outer = pyrender.MetallicRoughnessMaterial(alphaMode="BLEND",
                                          baseColorFactor=(1.0, 0.0, 0.0, 0.2))
p_cube_hollow = pyrender.Mesh.from_trimesh(cube_hollow, material=mat_outer, smooth=False)

cube_inner = trimesh.creation.box(extents=[0.3, 0.3, 0.3])
mat_inner = pyrender.MetallicRoughnessMaterial(alphaMode="BLEND",
                                          baseColorFactor=(0.0, 1.0, 0.0, 0.5))
p_cube_inner_transparent = pyrender.Mesh.from_trimesh(cube_inner, material=mat_inner, smooth=False)

light = pyrender.PointLight(color=[1.0, 1.0, 1.0], intensity=1)

scene = pyrender.Scene(ambient_light=[0.1, 0.1, 0.1, 1.0], bg_color=(1.0,1.0,1.0,0.0))
scene.add(p_cube_hollow)
scene.add(p_cube_inner_transparent)
scene.add(light).matrix = trimesh.transformations.translation_matrix([0, 1, 2])

viewer = pyrender.Viewer(scene, run_in_thread=False)

For illustraction, this is the scene rendered in wireframe mode:

However, in solid mode with the code as written above, I get the following result:

Reordering how the meshes are added to the scene fixes the problem and I get the expected result. However, this is not a general solution for more complex scenes. The following is a gif with the above code, but the lines scene.add(p_cube_hollow) and scene.add(p_cube_inner_transparent) are swapped.

Increasing the light intensity also fixes the problem, but only with large intensity (and it does not work for my actual scene, but does in this testcase):

Moving the meshes from their initial (0,0,0) position (even by as little as 0.0001) sometimes also seems to fix the problem.

[Qgel]

I've looking into this in a bit more detail, and it seems the culprit is the depth sorting function in renderer.py:710.

After some more research, I think I understand that there is no general 'always works' sorting function (especially for whole meshes, but also for individual triangles). However, the current function only considers the distance of the meshes centroid from the camera.

The following distance function would also consider the meshes extents. This fixes the problem in the above case, and also in the scene I'm actually trying to render:

def dist(node):
    d_centroid = node.mesh.centroid - cam_loc
    d_centroid_norm = np.linalg.norm(d_centroid)
    d_extents = node.mesh.extents * (d_centroid / d_centroid_norm)
    return -(d_centroid_norm - np.linalg.norm(d_extents))

What would be your thoughts on this? If this is not something that could be used in general, maybe it could be made possible to provide a custom sort function to the renderer or add ordering data to a mesh?

[Qgel]

Another thought: Why can't we just set glDepthMask(GL_FALSE), i.e. disable depthbuffer writing, after we have drawn all solid objects in renderer._forward_pass()? This would get rid of the problem, and in my opinion also renders more correctly in general:

We can see that now the inner cube also has transparency to the back wall of the outer cube, wheres before with any ordering one of them would always appear solid.

[sadaszewski]

Qgel's suggestion is good. It should be made available as default or perhaps as a renderer option or material property?