Generating Pointclouds from Numpy arrays

[GitwellAnyohub]

Hello,

Using the d415 camera, I'm able to generate a pointcloud starting with rs2::frames. However, rather than generating the pointclouds in realtime, I'd like to take to aligned numpy arrays (depth aligned to color) and THEN create point clouds. It'd be perfect if the pc.calculate() and the points.export_to_ply() functions worked on Numpy arrays and not just realsense frames. Has this been done before?

this is the code that works:

pc=rs.pointcloud()
points=rs.points()

#start pipeline

frames=pipeline.wait_for_frames()
depth_frame=frames.get_depth_frame()
color_frame=frames.get_color_frame()

pc.map_to(color_frame)
points=pc.calculate(Depth_frame)
points.export_to_ply('pointcloud/%06.ply'%(tick), color_frame)
'''```

[dorodnic]

Hello @GitwellAnyohub In general in the SDK we provide software_device to convert data into SDK frames. It's a bit more complicated than just two arrays, since frames carry a lot of extra information (intrinsic and extrinsic calibration and depth units are all required to generate a point-cloud). As far as I know, the software_device has not yet been ported to python (but is available for C++ and C#) You could generate point-cloud not in real-time if you save the scene into a *.bag file using the built-in recorder. We support fairly seamless playback of such files.

[GitwellAnyohub]

I'll try that at some point. Any idea when software_device will be ported into python?

[lramati]

Work has started in that direction, but making sure python doesn't delete the buffer too early is proving trickier than originally expected

[RealSense-Customer-Engineering]

[Realsense Customer Engineering Team Comment] @Git'wellAnyohub Any other quesitons for this ticket? Please update. Thanks!

[RealSense-Customer-Engineering]

[Realsense Customer Engineering Team Comment] @GitwellAnyohub Considering the question resolved, can we close the ticket? Please update. Thanks!

[binflo13]

Hello, has there been any progress in this topic? It would be very useful to get this.

[jsheo96]

is there any updates on this issue? cause I'm eagerly looking forward to a way to mask depth frame using numpy array.

[MartyG-RealSense]

Hi @jsheo96 I am not aware of any progress that has been made on numpy conversion, unfortunately.