Closed at-wat closed 3 years ago
io.ErrClosedPipe
? In the past users were confused why video wasn't showing up. They would create a Track
and just expect it to work. By returning an error it helped users debugging a lot.I do think we need to revisit the Media API entirely. This could be a good kicking off point. Stuff like we shouldn't be story the PayloadType/SSRC on tracks at all.
https://github.com/pion/webrtc/issues/1020#issue-563147171 is a kind of a workaround to do it on v2 API.
https://github.com/pion/webrtc/issues/1025 may be a better solution to implement ReplaceTrack
easily.
Are there any plans to implement this in v2 (or is v3 coming anytime soon)? I am considering adding it myself and have not dug into the specs, but any idea what ReplaceTrack
look like? I see the spec but I fear details like checking whether renegotiation is needed and what "switch seamlessly" entails.
@cretz I don't have specific plan yet. It's very helpful if you can dig into this.
checking whether renegotiation is needed
MDN said: https://developer.mozilla.org/en-US/docs/Web/API/RTCRtpSender/replaceTrack
The new track must be of the same media kind (audio, video, etc) and switching the track should not require negotiation.
I need this for an impl I'm working on, so I started a toying w/ an impl at https://github.com/cretz/webrtc/tree/replace-track. It's not very cleaned up by intention as I'm still working bugs out, but right now when it tries to switch tracks in the example code, it just ends up pausing the track, but if switched back, resumes just fine. I will be digging more into why, but if there's anything obvious, pointers welcome.
After further investigation, the only way I can get track replacement to work is to mess w/ the RTP packets. Specifically, besides setting the ssrc, I also have to keep my own sequence number, keep my own timestamps, and send an out-of-band RTCP PLI on change. I have tested with h264 and vp8 (no audio atm) and that procedure works well. There are various issues if any of those 4 things are not done which I can explain on request.
So, I don't think we can really support RTPSender.ReplaceTrack
here since it requires tighter control over all of the outbound RTP/RTCP params. If we did want to add a helper, it'd be some kind of pseudo track that relayed a track and allowed it to be switched.
I will open a PR a bit later with an example for "swapping" tracks, but I think there is no value in RTPSender.ReplaceTrack
at the moment because basically we'd need a pseudo track in the sender that is piped and alters RTP stuff.
PR opened at #1166
In Web API,
Track
connected toRTPSender
can be replaced without renegotiation. For example, it is used to switch between front/rear cameras. https://developer.mozilla.org/en-US/docs/Web/API/RTCRtpSender/replaceTrackIn the currently implementation,
Track
which is not bound toRTPSender
returns error. https://github.com/pion/webrtc/blob/2b212f8a7c4feed4621272bf8309637827e695cd/track.go#L155-L157ReplaceTrack()
toRTPSender
which removes itself from the currently boundTrack
and add to the newTrack
Track.WriteRTP()
to be called onTrack
withoutRTPSender
(It will be useful when using with https://github.com/pion/mediadevices)