panda3d / panda3d

Powerful, mature open-source cross-platform game engine for Python and C++, developed by Disney and CMU
https://www.panda3d.org/
Other
4.5k stars 784 forks source link

Expose Native Buffers ID for OpenGL-CUDA Interoperability #1468

Open QuanyiLi opened 1 year ago

QuanyiLi commented 1 year ago

Description

As discussed in https://discourse.panda3d.org/t/allowing-torch-tensorflow-to-directly-access-rendered-image-in-gpu/29119/7, it is possible to do interoperation between OpenGL and CUDA with native buffer ID. However, the related Python API lacks in Panda. Could you please provide such an interface?

Use Case

I am testing OpenGL and CUDA interoperability with pure Python interfaces provided by Panda3D and CuPY, respectively. Such a feature allows nowadays deep learning frameworks, like Tensorflow/PyTorch to manipulate the rendered content on GPU, which could improve the efficiency of visual data collection and model training by a large margin.

rdb commented 1 year ago

We just need to add a virtual get_native_id method to BufferContext, similar to how we have it in TextureContext, and implement it in the various buffer-related GL subclasses.

This could then easily be supported for vertex buffers, index buffers, and SSBOs.