Processing time is increasing

[TheFloHub]

Hey, Very nice tool you build here =) I noticed the processing time for sample_sdf_near_surface is increasing for every call when I run it in a loop. Besides this it seems to work perfectly for me. It also crashes every single time in the 100th call. Is this normal? What could be the reason for this? I'm running it on windows. The mesh I'm loading here is the standford bunny.

This is my simple code:

mesh = trimesh.load(Path("D:/Code/ClassReconstruction3d/meshes/bunny.obj"), force='mesh')
mesh = scale_to_unit_sphere(mesh)
for i in range(200):
    print(i)
    startTime = time.time()
    points, sdfs = sample_sdf_near_surface(mesh, number_of_points=30000)
    print(time.time() - startTime)

Cheers, Flo

[TheFloHub]

sorry I forgot the output^^

0 10.621185541152954 1 11.258548974990845 2 11.667204856872559 3 11.957420825958252 4 11.998848676681519 5 12.05840802192688 6 12.533770322799683 7 13.034075498580933

... 67 111.58820295333862 68 114.63963675498962 69 120.29056310653687 70 120.93348836898804 71

...

points, sdfs = sample_sdf_near_surface(mesh, number_of_points=30000) File "D:\virtualenv\tensorflow2\lib\site-packages\mesh_to_sdf__init.py", line 59, in sample_sdf_near_surface surface_point_cloud = get_surface_point_cloud(mesh, surface_point_method, 1, scan_count, scan_resolution, sample_point_count, calculate_normals=sign_method=='normal' or return_gradients) File "D:\virtualenv\tensorflow2\lib\site-packages\mesh_to_sdf__init__.py", line 17, in get_surface_point_cloud return surface_point_cloud.create_from_scans(mesh, bounding_radius=bounding_radius, scan_count=scan_count, scan_resolution=scan_resolution, calculate_normals=calculate_normals) File "D:\virtualenv\tensorflow2\lib\site-packages\mesh_to_sdf\surface_point_cloud.py", line 168, in create_from_scans z_far=bounding_radius * 3 File "D:\virtualenv\tensorflow2\lib\site-packages\mesh_to_sdf\scan.py", line 61, in init color, depth = render_normal_and_depth_buffers(mesh, camera, self.camera_transform, resolution) File "D:\virtualenv\tensorflow2\lib\site-packages\mesh_to_sdf\pyrender_wrapper.py", line 58, in render_normal_and_depth_buffers renderer = pyrender.OffscreenRenderer(resolution, resolution) File "D:\virtualenv\tensorflow2\lib\site-packages\pyrender\offscreen.py", line 31, in init self._create() File "D:\virtualenv\tensorflow2\lib\site-packages\pyrender\offscreen.py", line 149, in _create self._platform.init_context() File "D:\virtualenv\tensorflow2\lib\site-packages\pyrender\platforms\pyglet_platform.py", line 52, in init_context width=1, height=1) File "D:\virtualenv\tensorflow2\lib\site-packages\pyglet\window\win32__init.py", line 134, in init super(Win32Window, self).init(*args, **kwargs) File "D:\virtualenv\tensorflow2\lib\site-packages\pyglet\window\init.py", line 648, in init__ self._create() File "D:\virtualenv\tensorflow2\lib\site-packages\pyglet\window\win32\init.py", line 285, in _create self.switch_to() File "D:\virtualenv\tensorflow2\lib\site-packages\pyglet\window\win32\init__.py", line 348, in switch_to self.context.set_current() File "D:\virtualenv\tensorflow2\lib\site-packages\pyglet\gl\win32.py", line 243, in set_current super(Win32Context, self).set_current() File "D:\virtualenv\tensorflow2\lib\site-packages\pyglet\gl\base.py", line 334, in set_current gl_info.set_active_context() File "D:\virtualenv\tensorflow2\lib\site-packages\pyglet\gl\gl_info.py", line 91, in set_active_context self.vendor = asstr(cast(glGetString(GL_VENDOR), c_char_p).value) File "D:\virtualenv\tensorflow2\lib\site-packages\pyglet\gl\lib.py", line 107, in errcheck raise GLException(msg) pyglet.gl.lib.GLException: b'Der Vorgang ist ung\xfcltig.'

[GTO2013]

I am not 100% sure this is the problem, but for every scan this is called, which creates a new scene with the mesh and camera parameters for every scan. I am not sure if this gets cleaned up properly. It would probably be better to setup the scene with the mesh and camera once and just update the camera pose for each scan.

def render_normal_and_depth_buffers(mesh, camera, camera_transform, resolution):
    global suppress_multisampling
    suppress_multisampling = True
    scene = pyrender.Scene()
    scene.add(pyrender.Mesh.from_trimesh(mesh, smooth = False))
    scene.add(camera, pose=camera_transform)

    renderer = pyrender.OffscreenRenderer(resolution, resolution)
    renderer._renderer._program_cache = CustomShaderCache()

    color, depth = renderer.render(scene, flags=pyrender.RenderFlags.SKIP_CULL_FACES)
    suppress_multisampling = False
    return color, depth

[liangpan99]

I am not 100% sure this is the problem, but for every scan this is called, which creates a new scene with the mesh and camera parameters for every scan. I am not sure if this gets cleaned up properly. It would probably be better to setup the scene with the mesh and camera once and just update the camera pose for each scan.

def render_normal_and_depth_buffers(mesh, camera, camera_transform, resolution):
    global suppress_multisampling
    suppress_multisampling = True
    scene = pyrender.Scene()
    scene.add(pyrender.Mesh.from_trimesh(mesh, smooth = False))
    scene.add(camera, pose=camera_transform)

    renderer = pyrender.OffscreenRenderer(resolution, resolution)
    renderer._renderer._program_cache = CustomShaderCache()

    color, depth = renderer.render(scene, flags=pyrender.RenderFlags.SKIP_CULL_FACES)
    suppress_multisampling = False
    return color, depth

I have tested and found that the renderer cannot be cleaned up properly. When the function render_normal_and_depth_buffers is done, the clean up process will get stucked at this position (see the image) in file pyrender/renderer.py. Because the mesh_to_sdf use a custom shader and it doesn't define a clear() function.

To clean up properly, i add a clear() function like this:

# Render a normal buffer instead of a color buffer
class CustomShaderCache():
    def __init__(self):
        self.program = None

    def get_program(self, vertex_shader, fragment_shader, geometry_shader=None, defines=None):
        if self.program is None:
            shaders_directory = os.path.join(os.path.dirname(__file__), 'shaders')
            self.program = pyrender.shader_program.ShaderProgram(os.path.join(shaders_directory, 'mesh.vert'), os.path.join(shaders_directory, 'mesh.frag'), defines=defines)
        return self.program

   # For clean up
   def clear(self):
        self.program.delete()
        self.program = None

Hope it works.