Support RTPSender.ReplaceTrack()

[at-wat]

In Web API, Track connected to RTPSender can be replaced without renegotiation. For example, it is used to switch between front/rear cameras. https://developer.mozilla.org/en-US/docs/Web/API/RTCRtpSender/replaceTrack

In the currently implementation, Track which is not bound to RTPSender returns error. https://github.com/pion/webrtc/blob/2b212f8a7c4feed4621272bf8309637827e695cd/track.go#L155-L157

• Add ReplaceTrack() to RTPSender which removes itself from the currently bound Track and add to the new Track
• Allow Track.WriteRTP() to be called on Track without RTPSender

(It will be useful when using with https://github.com/pion/mediadevices)

[Sean-Der]

• 👍 from me on ReplaceTrack, that would be really helpful
• Can we just have the caller ignore io.ErrClosedPipe? In the past users were confused why video wasn't showing up. They would create a Track and just expect it to work. By returning an error it helped users debugging a lot.

I do think we need to revisit the Media API entirely. This could be a good kicking off point. Stuff like we shouldn't be story the PayloadType/SSRC on tracks at all.

[at-wat]

https://github.com/pion/webrtc/issues/1020#issue-563147171 is a kind of a workaround to do it on v2 API. https://github.com/pion/webrtc/issues/1025 may be a better solution to implement ReplaceTrack easily.

[cretz]

Are there any plans to implement this in v2 (or is v3 coming anytime soon)? I am considering adding it myself and have not dug into the specs, but any idea what ReplaceTrack look like? I see the spec but I fear details like checking whether renegotiation is needed and what "switch seamlessly" entails.

[at-wat]

@cretz I don't have specific plan yet. It's very helpful if you can dig into this.

checking whether renegotiation is needed

MDN said: https://developer.mozilla.org/en-US/docs/Web/API/RTCRtpSender/replaceTrack

The new track must be of the same media kind (audio, video, etc) and switching the track should not require negotiation.

[cretz]

I need this for an impl I'm working on, so I started a toying w/ an impl at https://github.com/cretz/webrtc/tree/replace-track. It's not very cleaned up by intention as I'm still working bugs out, but right now when it tries to switch tracks in the example code, it just ends up pausing the track, but if switched back, resumes just fine. I will be digging more into why, but if there's anything obvious, pointers welcome.

[cretz]

After further investigation, the only way I can get track replacement to work is to mess w/ the RTP packets. Specifically, besides setting the ssrc, I also have to keep my own sequence number, keep my own timestamps, and send an out-of-band RTCP PLI on change. I have tested with h264 and vp8 (no audio atm) and that procedure works well. There are various issues if any of those 4 things are not done which I can explain on request.

So, I don't think we can really support RTPSender.ReplaceTrack here since it requires tighter control over all of the outbound RTP/RTCP params. If we did want to add a helper, it'd be some kind of pseudo track that relayed a track and allowed it to be switched.

I will open a PR a bit later with an example for "swapping" tracks, but I think there is no value in RTPSender.ReplaceTrack at the moment because basically we'd need a pseudo track in the sender that is piped and alters RTP stuff.

[cretz]

PR opened at #1166