search

Kode / Kha

Ultra-portable, high performance, open source multimedia framework.
http://kha.tech
zlib License
1.48k stars 174 forks source link

Zero copy hardware accellerated video on Raspberry Pi #1055

Open mateli opened 5 years ago

mateli commented 5 years ago

This is examples for Raspberry Pi that renders video both on a rotating cube and a rotating teapot.

https://www.youtube.com/watch?v=-y3m_HFg4Do github.com/raspberrypi/userland/tree/master/host_applications/linux/apps/hello_pi/hello_videocube github.com/raspberrypi/userland/tree/master/host_applications/linux/apps/hello_pi/hello_teapot

This is done by allocating a shared buffer using EGL, upload encoded video to OpenMAX and decode it directly by the buffer that are then used by the OpenGL code to render the frames. To not download encoded data and re upload it to OpenGL is the key to get this performance. As OpenMax are used on most ARM platforms as well as the Mesa3D drivers for desktop cards this solution works on most platforms. Something similar can also be done using WPF. It would be preferred if Kha supported a consistent API for this across platforms so that above examples could be ported to Kha and run on Rasperry Pi, Android, x86 Linux, Windows etc.

I am also submitting video encoding as a separate issue where both can be combined in order to for example transcodeing video as well as adding various effect to a video, above and beyond what ffmpeg can do. #1056

RobDangerous commented 5 years ago

I edited your title because this is already implemented for Windows, macOS, iOS and Android. Also on Linux AFAIK different APIs have to be used for different hardware.

mateli commented 5 years ago

Most Linux hardware support OpenMAX with recent versions of Mesa 3D. Mesa previously only implemented support for VAAPI but now has OpenMAX support as well. Using OpenMAX probably works a bit differently on the RPi frame buffer, X11 and Wayland. I believe that Android use OpenMAX as well so perhaps that code can be adapted.

Does the current code support playing on a 3D surface?

RobDangerous commented 5 years ago

Of course it does. The shader compiler supports a special video sampler type and you can use it for whatever you want. The Android code is here: https://github.com/Kode/Kinc/blob/master/Backends/System/Android/Sources/Kore/Video.cpp