selkies-project / selkies-gstreamer

Open-Source Low-Latency Accelerated Linux WebRTC HTML5 Remote Desktop Streaming Platform for Self-Hosting, Containers, Kubernetes, and HPC
Mozilla Public License 2.0
261 stars 43 forks source link

[META] ARM, mobile/embedded devices, and IBM POWER ppc64le #50

Open ehfd opened 1 year ago

ehfd commented 1 year ago

GStreamer has encoders and screen capturing interfaces from various embedded operating systems, including Raspberry Pi and NVIDIA Jetson. gst-omx is an example of an encoder usable for Raspberry Pi and other ARM SoCs. v4l2codecs in gst-bad enables encoding support for Video4Linux as well. While support for Android or iOS is probably out of our scope, ARM Linux support should be useful.

Relevant files: https://github.com/selkies-project/selkies-gstreamer/blob/master/src/selkies_gstreamer/gstwebrtc_app.py and to add omxh264enc and/or v4l2h264enc, then to add aarch64 capabilities to GStreamer build workflows.

The most critical part here is the native build pipeline (emulation is prohibited because of a 7-8x overhead and 6-hour time limit for GitHub Actions).

ehfd commented 1 year ago

https://github.com/m1k1o/neko uses v4l2h264enc for Raspberry Pi.

ehfd commented 1 year ago

@ndufresne, maintainer of the v4l2 plugin of GStreamer Good Plugins:

yeah, so I'd say, stick with VP8/H264 (and HEVC perhaps)
I've seen some activity (of VP9 encoders) on Exynos side, after half a decade of silence, might be good news
as for stateless/rk (Rockchip), I expect one or two more years before that is ready
Media CODEC in Linux are still in catchup mode

VP9 encoders are out of the table for at least a year. Thus, I am focusing on H.264 and VP8, with H.265 to be back on the table if we land WebTransport.

https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/-/commit/040e25218ac4d7cf50bcfd4651300b4bdfa629cb

ehfd commented 1 year ago

From https://github.com/selkies-project/docker-nvidia-egl-desktop/issues/23: @brianlmerritt

NVIDIA Jetson:

  1. Compiling GStreamer for AARCH64 (addons/gstreamer)
  2. Add nvv4l2h264enc/nvv4l2vp9enc and test
  3. Verify all the pipeline works Would be trivial for newbies.

https://docs.nvidia.com/jetson/archives/r35.2.1/DeveloperGuide/text/SD/Multimedia/AcceleratedGstreamer.html

Nicholas Dufresne (ndufresne) at #gstreamer IRC oftc.net should be experienced in this regard.

ehfd commented 9 months ago

GST-OMX is deprecated. Stick with v4l2.