How to speed-up rendering?

[MrBoriska]

Checklist

• [X] I have searched for similar issues.
• [X] For Python issues, I have tested with the latest development wheel.
• [X] I have checked the release documentation and the latest documentation (for master branch).

My Question

I am trying rendering Triangle Mesh object to image with o3d.visualization.rendering.OffscreenRenderer(640, 480)

But 10Hz is maximum frequency for my system (Jetson TX2).

Start code:

msg_in = self.color_sub.message

self.fx = msg_in.proto.pinhole.focal.x
self.fy = msg_in.proto.pinhole.focal.y
self.cx = msg_in.proto.pinhole.center.x
self.cy = msg_in.proto.pinhole.center.y

self.height = msg_in.proto.pinhole.rows
self.width = msg_in.proto.pinhole.cols

self.channels = msg_in.proto.image.channels

intr_matrix = np.asarray([self.fx, 0,  self.cx,
                        0,  self.fy, self.cy,
                        0,  0,  1 ], dtype=np.float64)
intr_matrix = intr_matrix.reshape(3,3)
self.render = o3d.visualization.rendering.OffscreenRenderer(self.width, self.height, headless=False)
self.render.scene.show_axes(False)
# Pick a background colour (default is light gray)
self.render.scene.set_background([0, 0, 0, 1.0])  # RGBA
self.render.scene.set_lighting(self.render.scene.LightingProfile.NO_SHADOWS, (0, 5, 0))

extrinsics = np.eye(4,4)
#extrinsics[0:3,0:3] = o3d.geometry.TriangleMesh.get_rotation_matrix_from_xyz((0, 0, 0))
#extrinsics[3,0:3] = (0,0,0)
self.render.setup_camera(intr_matrix, extrinsics, self.width, self.height)

In loop for each input image (AR tags detections):

self.render.scene.clear_geometry()

fds_msg = self.fiducials_sub.message
for fd in fds_msg.proto.fiducialList:
    if fd.id and fd.id in self.objects.keys() and fd.cameraTTag:
        quaternion = np.asarray([
            fd.cameraTTag.rotation.q.w,
            fd.cameraTTag.rotation.q.x,
            fd.cameraTTag.rotation.q.y,
            fd.cameraTTag.rotation.q.z
        ])
        R = o3d.geometry.TriangleMesh.get_rotation_matrix_from_quaternion(quaternion)

        P = np.asarray([
            fd.cameraTTag.translation.x,
            fd.cameraTTag.translation.y,
            fd.cameraTTag.translation.z
        ])

        self.objects[fd.id].obj.rotate(R @ np.linalg.inv(self.objects[fd.id].R))
        self.objects[fd.id].R = R
        self.objects[fd.id].obj.translate(P, relative=False)

        self.render.scene.add_geometry(self.objects[fd.id].name, self.objects[fd.id].obj, self.objects[fd.id].material)

t1 = time()
img = self.render.render_to_image()
t2 = time()

Where (t2 - t1) ~ 0.1 sec

How to speedup rendering ?

[errissa]

I am not very familiar with the performance characteristics of the Jetson TX2 so it's hard to answer this question. Obviously, performance partly depends on scene complexity. For render_to_image it will also depend on the GPU memory read back speed since it is a synchronous call and waits for the rendered image to be delivered back. To get a feel for the best performance you can expect from render_to_image on your Jetson I would recommend taking that same measurement using the examples/python/visualization/render_to_image.py example. It sets up about as simple a scene as possible.

One more thing to try if you don't need the post-processing effects (such as SSAO, tone mapping, color grading, etc.) is to disable post processing:

self.render.scene.view.set_post_processing(False)

[MrBoriska]

Errissa, thank you for reply. I am trying set_post_processing. This method not found in view. AttributeError: 'open3d.cpu.pybind.visualization.rendering.View' object has no attribute 'set_post_processing'

For perfomance, i am using VTK for python and get ~0.03s render time. But open3d is more convenient for me...

VTK and Open3d using OpenGL for rendering and using offscreen mode.

[errissa]

Please try the latest development wheels of Open3D available here. The set_post_processing is a relatively new feature.

[MrBoriska]

I am using latest ARM64 build (14.1 version).