mmatl / pyrender

Easy-to-use glTF 2.0-compliant OpenGL renderer for visualization of 3D scenes.
http://pyrender.readthedocs.io/
MIT License
1.31k stars 226 forks source link

pyrender offscreen render with the SKIP_CULL_FACES flag outputs inconsistent results #250

Open Jianghanxiao opened 1 year ago

Jianghanxiao commented 1 year ago

In this setting, the results are not deterministic. Do you have some idea on where does the randomness come from? And is there some way to avoid such randomness?

Below is an example code snippet to see the problem. When you run it multiple times, you will get inconsistent results. (Here I hardcode the camera pose and light pose here to avoid some additional library to calculate it, but you can use any pose)

You can use this model from ShapeNet to test https://drive.google.com/file/d/1M0ncX1IXSxcpGijlDm58MCr36CaGts2S/view?usp=sharing

import pyrender
import trimesh
import numpy as np
import os
import imageio
os.environ['PYOPENGL_PLATFORM'] = 'egl'

np.random.rand(52)
mesh_path = 'example/models/model_normalized.obj'
mesh_tri = trimesh.load(mesh_path)
scene = pyrender.Scene.from_trimesh_scene(mesh_tri)

camera = pyrender.PerspectiveCamera(yfov=np.pi / 4.5, aspectRatio=1.0)
camera_pose = [[-1.00000000e+00,  6.12323400e-17, -1.06057524e-16, -4.24230095e-16],
 [ 0.00000000e+00,  8.66025404e-01,  5.00000000e-01,  2.00000000e+00],
 [ 1.22464680e-16,  5.00000000e-01, -8.66025404e-01, -3.46410162e+00],
 [ 0.00000000e+00, 0.00000000e+00,  0.00000000e+00,  1.00000000e+00]]
light_pose = [[ 2.22044605e-16, -1.00000000e+00 , 0.00000000e+00 , 0.00000000e+00],
 [ 1.00000000e+00 , 2.22044605e-16 , 0.00000000e+00 , 0.00000000e+00],
 [ 0.00000000e+00 , 0.00000000e+00 , 1.00000000e+00 , 3.00000000e+00],
 [ 0.00000000e+00 , 0.00000000e+00 , 0.00000000e+00  ,1.00000000e+00]]
scene.ambient_light = np.full(shape=3, fill_value=0.3, dtype=np.float32)
scene.add(camera, pose=camera_pose)
light = pyrender.DirectionalLight(color=np.ones(3), intensity=6)
scene.add(light, pose=light_pose)

r = pyrender.OffscreenRenderer(800, 800)
color, _ = r.render(scene, flags=pyrender.RenderFlags.SKIP_CULL_FACES)
# color, _ = r.render(scene)

imageio.imwrite('temp.png', color)

Here are the two inconsistent results: temp temp

eamonn-zh commented 1 year ago

Same problem, in particular, it occurs in the case of double face