With WebRTC, we can serve individual users different tracks based on their needs. A good example of this is if the user only supports a specific video technology during the SDP negotiation, we can only serve that particular type of video.
For transcoding, it should be able to work the same way. Right now Waveguide is the offer, and the browser is the answer. This should allow the browser to automatically tell Waveguide what it supports, however nothing immediately tells us about their bandwidth requirements. For figuring out their bandwidth requirements we have two potential options:
Use the rtcp.ReceiverEstimatedMaximumBitrate to automatically downgrade the stream to what the user can handle.
Force the user to choose a lower bitrate stream via the UI and communicate that to Waveguide.
For now with Glimesh / Waveguide, we can handle only one type of transcoding and that is to bring the video down to under 2000Kbps. In effect this would give our users two options, Source (whatever the streamer is putting out) and something similar to a 720p 30fps stream.
Where does transcoding fit into Waveguide's architecture?
Not a super clear answer with the current architecture. We'll need to transcode a video stream at most once, so likely it'll happen on whatever Waveguide server the streamer is connected to, and then add the transcoded track to the Control.
With WebRTC, we can serve individual users different tracks based on their needs. A good example of this is if the user only supports a specific video technology during the SDP negotiation, we can only serve that particular type of video.
For transcoding, it should be able to work the same way. Right now Waveguide is the offer, and the browser is the answer. This should allow the browser to automatically tell Waveguide what it supports, however nothing immediately tells us about their bandwidth requirements. For figuring out their bandwidth requirements we have two potential options:
For now with Glimesh / Waveguide, we can handle only one type of transcoding and that is to bring the video down to under 2000Kbps. In effect this would give our users two options, Source (whatever the streamer is putting out) and something similar to a 720p 30fps stream.
Where does transcoding fit into Waveguide's architecture?
Not a super clear answer with the current architecture. We'll need to transcode a video stream at most once, so likely it'll happen on whatever Waveguide server the streamer is connected to, and then add the transcoded track to the Control.
How to transcode video?
There's a couple of different options to explore. It seems Pion offers a couple of bindings in the MediaDevices package that may help support interacting with h264. An example: https://github.com/pion/mediadevices/blob/master/examples/rtp/main.go
We could also use ffmpeg / gstreamer bindings directly from Go, Pion provides a couple of examples for that here:
https://github.com/pion/example-webrtc-applications/blob/master/gstreamer-receive/main.go#L50-L62
https://github.com/pion/example-webrtc-applications/blob/master/twitch/main.go#L141-L147