Support GPU Accelerated FrameCapture on all OpenGL supported platforms.

PhotonVision / photonvision

PhotonVision is the free, fast, and easy-to-use computer vision solution for the FIRST Robotics Competition.

https://photonvision.org

GNU General Public License v3.0

264 stars 183 forks source link

Support GPU Accelerated FrameCapture on all OpenGL supported platforms. #1339

Open srimanachanta opened 3 months ago

srimanachanta commented 3 months ago

Currently, we use libcamera + OpenGL for GPU acceleration but we limit it to Raspberry Pi only. OpenGL is supported by ARM Mali GPUs as well so we should be able to adapt the current driver to support both VideoCore and Mali GPUs by leveraging OpenGL. If the issue lies with libcamera, we could revert to V4l2.

mcm001 commented 3 months ago

We have a prototype using JOGL for HSV, but without zero-copy, it was slower than on-CPU. And my understanding is that's only possible with CSI cameras anyways. Unless v4l lets you put USB camera frames straight into shared memory?

ramalamadingdong commented 3 months ago

can I see this prototype? I am doing benchmarks for a GPU based robotics kit and would like to run those benchmarks using Photonvision. You can make v4l usb camera into shared memory using something like this V4L-RS

Or maybe gstreamer.

I am really happy to help do some / most of the dev work required to get this going but would like to get plugged in to current efforts first.

Last question: If I use libcamera why is it limited to RPI?