WebRTC Support

[s-ol]

Using gstreamer because it seems that's mandatory even if you use a more lightweight WebRTC signalling library. For WebRTC, needs the gst-plugins-rs package. Test program in make test/gstreamer shows a PNG at http://localhost:4273/gst-image/.

• [X] basic gstreamer playback into images on DB
• [x] start a gst-webrtc-signalling-server instance
• [ ] figure out what happens when a stream fails to start or ends
  • does it just silently retry until it starts working (again)? tie into folk's error system somehow?
  • could use some help with this
• [x] vendor the gstwebrtc-api JS bundle and implement a web route that allows streaming

stretch goals:

• [ ] gstreamer seeking (for non-live streams)?
• [ ] gstreamer publishing (or specifically webrtc publishing)
  • of a region?
• [ ] replace a big chunck of camera.folk with gstreamer
  • uridecodebin uri=v4l2:///dev/video0 has strong screen tearing for me, need to investigate

[s-ol]

How to use at the moment:

When the gstreamer pipeline "videotestsrc" frame is /frame/ at /ts/ { ... }

The syntax of the pipeline string is gst-launch-1.0, but gstreamer.folk will automatically add a sink (and videoconvert) at the end.

For webrtc:

• install gst-plugins-rs
• run gst-webrtc-signalling-server
• clone/download this directory https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/tree/main/net/webrtc/gstwebrtc-api
• npm i && npm start
  • npm run build creates a bundle of the library code that can be vendored
• start sharing video in the browser window that opens, take note of peer-id
• use pipeline string uridecodebin uri=gstwebrtc://127.0.0.1:8443?peer-id=$PEER_ID

[osnr]

(slightly concerned about the long-term goal of replacing camera.folk/v4l2 with gstreamer -- it feels like we should try to use the lowest-level reasonable interface in general to avoid latency and give us configuration power. But otherwise this seems good/useful, continuing to look at it now)

[cwervo]

Okay here are some thoughts after thinking about this for a day, seeking :

• @s-ol what do you think about carving this out into its own repo that we vendor? I'm kind of concerned about mixing dependencies that include npm for Gstreamer
• I'm also interested in using just the Gstreamer C core for local video playback
• So, I guess my delineation is: folk.js stays in this repo, everything that depends on npm goes into a separate public repo and is basically Folk WebRTC middleware, does that make sense?

[s-ol]

• @s-ol what do you think about carving this out into its own repo that we vendor? I'm kind of concerned about mixing dependencies that include npm for Gstreamer

Oh, I didn't update the instructions above anymore, npm was necessary to build the gstreamer webrtc demo page for initial testing, but later I compiled the gstwebrtc-api library into a standalone JS bundle and vendored it. Now only the gstreamer distro packages are dependencies and if they aren't detected webrtc.folk errors with a message pointing out what to install.

the flow now is:

• wish for a webrtc feed (or trigger the When when), see test/webrtc.tcl
• open http://localhost:4273/webrtc and click the link for the feed to broadcast to
• start camera or screenshare