o3d.t.io.RSBagReader() with libRealSense Depth Post-Processing

[jrecasens]

Checklist

• [X] I have searched for similar issues.
• [X] For Python issues, I have tested with the latest development wheel.
• [X] I have checked the release documentation and the latest documentation (for master branch).

Proposed new feature or change

When reading a .bag file we would like apply Depth Post-Processing as implemented in the Intel RealSense SDK 2.0 (libRealSense)

The existing o3d.t.io.RSBagReader() class reads a .bag file as-is but we cannot add the libRealSense post processing such as decimation or temporal filters to the depth data.

When using o3d.t.io.RSBagReader() we get a RGBDImage Open3D format from bag_reader.next_frame(). This means there is no space to actually apply depth post-processing.

This work involves adding additional code to o3d.t.io.RSBagReader() to apply additional filters to the depth variable. For example (Python Pseudo Code):

depth_frame = frames.get_depth_frame(); decimation_filter = rs.decimation_filter() decimation_filter.set_option(filter_magnitude, decimation_magnitude) filtered_frame = decimation_filter.process(depth_frame)

Filters could be passed as json or a dictionary of filters

References

https://github.com/oclis/ROS-EYE/blob/ec30974957be471ef60da47f173dfbbffcfc555d/src/realsense_device_manager.py#L49

Additional information

No response

[jrecasens]

Hello, Is this something possible to implement in Open3D? any recommendations on how to implement this?

[ssheorey]

Hi @jrecasens this is not supported in Open3D at the moment. We would be happy to accept PRs implementing this feature. I believe it should be straightforward to add support for this. As a workaround, you can directly use librealsense python bindings for now of you need more advanced functionality not supported by Open3D.