search

microsoft / MixedReality-WebRTC

MixedReality-WebRTC is a collection of components to help mixed reality app developers integrate audio and video real-time communication into their application and improve their collaborative experience
https://microsoft.github.io/MixedReality-WebRTC/
MIT License
913 stars 283 forks source link

H264 stream as ExternalVideoTrackSource #664

Open Awaisryk opened 3 years ago

Awaisryk commented 3 years ago

I have a UWP app that uses data channels to send h264 stream (from android) to browser where I am using Broadway H264 decoder to decode this stream and displaying in on canvas. I want to use WebRTC video tracks instead of data channels. I looked into ExternalVideoTrackSource but I don't know how to create I420AVideoFrame from those bytes[] I am receiving from android. I would really appreciate if someone can point me to some directions. Thanks.

djee-ms commented 3 years ago

You can't. We don't support feeding arbitrary video content into the WebRTC engine, only I420 raw uncompressed ones. That would be quite difficult I think to implement because the underlying Google implementation only deals with compressed video streams internally, and has no API to manipulate compressed ones (they only flow internally).