search

pion / webrtc

Pure Go implementation of the WebRTC API
https://pion.ly
MIT License
13.88k stars 1.66k forks source link

pion sends RTCP RR for offered rtx streams #2197

Open atoppi opened 2 years ago

atoppi commented 2 years ago

Your environment.

What did you do?

I'm sending an offer from Janus that contains rtx for video

SDP offer (janus) ``` v=0 o=- 1651154116641094 1 IN IP4 192.168.1.174 s=Mountpoint 10000 t=0 0 a=group:BUNDLE a v a=ice-options:trickle a=fingerprint:sha-256 82:8E:EA:3D:0D:52:F1:3B:02:0C:A7:5F:10:E1:DD:9D:4C:40:1D:13:8C:AE:19:FA:86:C7:DF:6A:A3:3E:23:12 a=extmap-allow-mixed a=msid-semantic: WMS janus m=audio 9 UDP/TLS/RTP/SAVPF 111 c=IN IP4 192.168.1.174 a=sendonly a=mid:a a=rtcp-mux a=ice-ufrag:LrNZ a=ice-pwd:RewF3LVilXQs06C9gOjVUk a=ice-options:trickle a=setup:actpass a=rtpmap:111 opus/48000/2 a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid a=msid:janus janusa a=ssrc:2759908030 cname:janus a=candidate:1 1 udp 2015363327 192.168.1.174 53372 typ host a=end-of-candidates m=video 9 UDP/TLS/RTP/SAVPF 100 101 c=IN IP4 192.168.1.174 a=sendonly a=mid:v a=rtcp-mux a=ice-ufrag:LrNZ a=ice-pwd:RewF3LVilXQs06C9gOjVUk a=ice-options:trickle a=setup:actpass a=rtpmap:100 vp8/90000 a=rtcp-fb:100 nack a=rtcp-fb:100 nack pli a=rtcp-fb:100 goog-remb a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time a=rtpmap:101 rtx/90000 a=fmtp:101 apt=100 a=ssrc-group:FID 1010778191 3393424866 a=msid:janus janusv a=ssrc:1010778191 cname:janus a=ssrc:3393424866 cname:janus a=candidate:1 1 udp 2015363327 192.168.1.174 53372 typ host a=end-of-candidates ```

Since pion seems not to support rtx over a different SSRC, I reject the rtx by not registering the codec in the media engine.

SDP answer (pion) ``` v=0 o=- 2205701793287732906 1651154116 IN IP4 0.0.0.0 s=- t=0 0 a=fingerprint:sha-256 53:CB:E8:41:73:7D:79:DF:A7:B0:40:B3:0D:22:FF:55:10:52:CB:EA:33:47:5B:6A:5B:B2:74:9C:D6:AC:02:6C a=group:BUNDLE a v m=audio 9 UDP/TLS/RTP/SAVPF 111 c=IN IP4 0.0.0.0 a=setup:active a=mid:a a=ice-ufrag:UMqimwTZaeCbwgxK a=ice-pwd:XXJzevSCqkILXAaBakEipRSxZHVJtsMt a=rtcp-mux a=rtcp-rsize a=rtpmap:111 opus/48000/2 a=recvonly m=video 9 UDP/TLS/RTP/SAVPF 100 c=IN IP4 0.0.0.0 a=setup:active a=mid:v a=ice-ufrag:UMqimwTZaeCbwgxK a=ice-pwd:XXJzevSCqkILXAaBakEipRSxZHVJtsMt a=rtcp-mux a=rtcp-rsize a=rtpmap:100 vp8/90000 a=rtcp-fb:100 nack a=rtcp-fb:100 nack pli a=rtcp-fb:100 goog-remb a=recvonly ```

Janus acknowledges the rejection and switches to same-ssrc retransmissions

What did you expect?

pion ignores the rtx SSRC

What happened?

pion starts sending periodic RTCP Receiver Reports about the rtx stream and janus prints a lot of warnings about the unknown SSRC for incoming RTCP.

[Thu Apr 28 15:56:03 2022] [5913132446596796] Unknown SSRC, dropping RTCP packet (SSRC 3393424866)...
[Thu Apr 28 15:56:04 2022] [5913132446596796] Unknown SSRC, dropping RTCP packet (SSRC 3393424866)...
[Thu Apr 28 15:56:05 2022] [5913132446596796] Unknown SSRC, dropping RTCP packet (SSRC 3393424866)...
[Thu Apr 28 15:56:06 2022] [5913132446596796] Unknown SSRC, dropping RTCP packet (SSRC 3393424866)...

I'm thinking of munging the SDP from janus in order to strip out a=ssrc lines.

boushley commented 2 years ago

Let's start by calling out that this is not an area I'm really familiar with.

I see where pion is starts handling the RTX flow as part of the SDP setup. Pion then passes those details along to the RTPReceiveParameters which end up flowing into the RTPReceiver.Receive method and into startReceive which handles the RTX SSRC. The call to streamsForSSRC results in a call to BindRemoteStream on the interceptor interface. When the ReceiverInterceptor handles that call it adds the stream to the set of streams to send Receiver Reports to.

So that's the flow that's resulting in RTCP receiver reports being sent.

I don't know much about using RTX streams with Pion, but it definitely looks like there's at least support for that setup in the session parsing and setup. I wonder if the comment you linked to is out of date and this is now supported?

It seems to me the actual issue may be that the SDP answer is missing an indication that the RTX stream is supported and setup. Or that RTX support is half baked and we should stop passing the RTX SSRC into BindStream if we're not ready to handle them.

atoppi commented 2 years ago

hello @boushley thanks for the detailed analysis

I don't know much about using RTX streams with Pion, but it definitely looks like there's at least support for that setup in the session parsing and setup. I wonder if the comment you linked to is out of date and this is now supported?

when trying to offer rtx from pion, this is the outcome

pion m-line with rtx ``` m=video 9 UDP/TLS/RTP/SAVPF 100 101 c=IN IP4 0.0.0.0 a=setup:actpass a=mid:1 a=ice-ufrag:AJuScACpyDizGMPr a=ice-pwd:DWQKYwgYAxQxNTrVpypGcEsGXycXoCPC a=rtcp-mux a=rtcp-rsize a=rtpmap:100 VP8/90000 a=rtcp-fb:100 goog-remb a=rtcp-fb:100 ccm fir a=rtcp-fb:100 nack a=rtcp-fb:100 nack pli a=rtcp-fb:100 transport-cc a=rtpmap:101 rtx/90000 a=fmtp:101 apt=100 a=rtcp-fb:101 transport-cc a=extmap:1 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01 a=ssrc:3807425665 cname:jattack-pion a=ssrc:3807425665 msid:jattack-pion video a=ssrc:3807425665 mslabel:jattack-pion a=ssrc:3807425665 label:video a=msid:jattack-pion video a=sendrecv ```

as you can see only one SSRC is being announced and there is no a=ssrc-group:FID, so I guess that rtx is not being negotiated like e.g. does Chrome/FF.

Chrome m-line with rtx ``` m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 127 121 125 107 108 109 124 120 123 119 35 36 41 42 114 115 116 (124 more lines) mid=1 c=IN IP4 0.0.0.0 a=rtcp:9 IN IP4 0.0.0.0 a=ice-ufrag:iahB a=ice-pwd:Z/PSwlvCHxmjSk+XHnWb/zmr a=ice-options:trickle a=fingerprint:sha-256 6F:BC:07:B5:D8:BD:62:C9:30:28:14:80:BF:14:B0:A5:01:E9:D3:47:78:6E:2E:A4:60:49:93:3B:88:67:1D:E2 a=setup:actpass a=mid:1 a=extmap:14 urn:ietf:params:rtp-hdrext:toffset a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time a=extmap:13 urn:3gpp:video-orientation a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01 a=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space a=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid a=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id a=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id a=sendrecv a=msid:YWYk3JQuk8iEjX8z86WrcGq973LFw2xCowvH e2968c38-583b-4702-ad05-b1a381991b25 a=rtcp-mux a=rtcp-rsize a=rtpmap:96 VP8/90000 a=rtcp-fb:96 goog-remb a=rtcp-fb:96 transport-cc a=rtcp-fb:96 ccm fir a=rtcp-fb:96 nack a=rtcp-fb:96 nack pli a=rtpmap:97 rtx/90000 a=fmtp:97 apt=96 a=rtpmap:98 VP9/90000 a=rtcp-fb:98 goog-remb a=rtcp-fb:98 transport-cc a=rtcp-fb:98 ccm fir a=rtcp-fb:98 nack a=rtcp-fb:98 nack pli a=fmtp:98 profile-id=0 a=rtpmap:99 rtx/90000 a=fmtp:99 apt=98 a=rtpmap:100 VP9/90000 a=rtcp-fb:100 goog-remb a=rtcp-fb:100 transport-cc a=rtcp-fb:100 ccm fir a=rtcp-fb:100 nack a=rtcp-fb:100 nack pli a=fmtp:100 profile-id=2 a=rtpmap:101 rtx/90000 a=fmtp:101 apt=100 a=rtpmap:127 H264/90000 a=rtcp-fb:127 goog-remb a=rtcp-fb:127 transport-cc a=rtcp-fb:127 ccm fir a=rtcp-fb:127 nack a=rtcp-fb:127 nack pli a=fmtp:127 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f a=rtpmap:121 rtx/90000 a=fmtp:121 apt=127 a=rtpmap:125 H264/90000 a=rtcp-fb:125 goog-remb a=rtcp-fb:125 transport-cc a=rtcp-fb:125 ccm fir a=rtcp-fb:125 nack a=rtcp-fb:125 nack pli a=fmtp:125 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f a=rtpmap:107 rtx/90000 a=fmtp:107 apt=125 a=rtpmap:108 H264/90000 a=rtcp-fb:108 goog-remb a=rtcp-fb:108 transport-cc a=rtcp-fb:108 ccm fir a=rtcp-fb:108 nack a=rtcp-fb:108 nack pli a=fmtp:108 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f a=rtpmap:109 rtx/90000 a=fmtp:109 apt=108 a=rtpmap:124 H264/90000 a=rtcp-fb:124 goog-remb a=rtcp-fb:124 transport-cc a=rtcp-fb:124 ccm fir a=rtcp-fb:124 nack a=rtcp-fb:124 nack pli a=fmtp:124 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f a=rtpmap:120 rtx/90000 a=fmtp:120 apt=124 a=rtpmap:123 H264/90000 a=rtcp-fb:123 goog-remb a=rtcp-fb:123 transport-cc a=rtcp-fb:123 ccm fir a=rtcp-fb:123 nack a=rtcp-fb:123 nack pli a=fmtp:123 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=4d001f a=rtpmap:119 rtx/90000 a=fmtp:119 apt=123 a=rtpmap:35 H264/90000 a=rtcp-fb:35 goog-remb a=rtcp-fb:35 transport-cc a=rtcp-fb:35 ccm fir a=rtcp-fb:35 nack a=rtcp-fb:35 nack pli a=fmtp:35 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=4d001f a=rtpmap:36 rtx/90000 a=fmtp:36 apt=35 a=rtpmap:41 AV1/90000 a=rtcp-fb:41 goog-remb a=rtcp-fb:41 transport-cc a=rtcp-fb:41 ccm fir a=rtcp-fb:41 nack a=rtcp-fb:41 nack pli a=rtpmap:42 rtx/90000 a=fmtp:42 apt=41 a=rtpmap:114 red/90000 a=rtpmap:115 rtx/90000 a=fmtp:115 apt=114 a=rtpmap:116 ulpfec/90000 a=ssrc-group:FID 1202711560 1827609002 a=ssrc:1202711560 cname:SYt2IjUvX5bWwto8 a=ssrc:1202711560 msid:YWYk3JQuk8iEjX8z86WrcGq973LFw2xCowvH e2968c38-583b-4702-ad05-b1a381991b25 a=ssrc:1202711560 mslabel:YWYk3JQuk8iEjX8z86WrcGq973LFw2xCowvH a=ssrc:1202711560 label:e2968c38-583b-4702-ad05-b1a381991b25 a=ssrc:1827609002 cname:SYt2IjUvX5bWwto8 a=ssrc:1827609002 msid:YWYk3JQuk8iEjX8z86WrcGq973LFw2xCowvH e2968c38-583b-4702-ad05-b1a381991b25 a=ssrc:1827609002 mslabel:YWYk3JQuk8iEjX8z86WrcGq973LFw2xCowvH a=ssrc:1827609002 label:e2968c38-583b-4702-ad05-b1a381991b25 ```

As a consequence, I'm disabling rtx support by not registering the codec but that leads to the original issue.

atoppi commented 2 years ago

fyi I was unable to reproduce on #2231

atoppi commented 1 year ago

Getting hit again by the same issue :-( It looks like the triggering condition is disabling rtx in the pion media engine (e.g. not registering rtx support).

pion/webrtc v3.1.46

The scenario is still the same:

Janus offering rtx in different SSRC ``` m=video 9 UDP/TLS/RTP/SAVPF 96 97 c=IN IP4 192.168.1.21 a=sendonly a=mid:video a=rtcp-mux a=ice-ufrag:q7MJ a=ice-pwd:xb3fA3BKob5bbFlvPk59Be a=ice-options:trickle a=fingerprint:sha-256 7F:56:8F:3B:9A:7F:B5:2D:07:38:0D:D6:F6:02:C8:53:BD:A7:23:58:0F:25:B8:78:D6:EC:6A:38:05:39:4D:C9 a=setup:actpass a=rtpmap:96 VP8/90000 a=rtcp-fb:96 ccm fir a=rtcp-fb:96 nack a=rtcp-fb:96 nack pli a=rtcp-fb:96 goog-remb a=rtcp-fb:96 transport-cc a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time a=rtpmap:97 rtx/90000 a=fmtp:97 apt=96 a=ssrc-group:FID 3334769273 2297016547 a=msid:janus janusv0 a=ssrc:3334769273 cname:janus a=ssrc:3334769273 msid:janus janusv0 a=ssrc:3334769273 mslabel:janus a=ssrc:3334769273 label:janusv0 a=ssrc:2297016547 cname:janus a=ssrc:2297016547 msid:janus janusv0 a=ssrc:2297016547 mslabel:janus a=ssrc:2297016547 label:janusv0 ```
pion not understanding the ssrc-group:FID in the offer -> starts sending RTCP to the rtx SSRC ``` m=video 9 UDP/TLS/RTP/SAVPF 96 c=IN IP4 0.0.0.0 a=setup:active a=mid:video a=ice-ufrag:QVwRmmRySXeSEOcF a=ice-pwd:rcpfyMdDWUJweTkMiandYJdEpRcfxkxu a=rtcp-mux a=rtcp-rsize a=rtpmap:96 VP8/90000 a=rtcp-fb:96 ccm fir a=rtcp-fb:96 nack a=rtcp-fb:96 nack pli a=rtcp-fb:96 goog-remb a=rtcp-fb:96 transport-cc a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid a=recvonly ``` 
[Mon Dec 12 14:52:03 2022] [1726755082988956] Dropping RTCP packet with unknown SSRC (2297016547)
[Mon Dec 12 14:52:04 2022] [1726755082988956] Dropping RTCP packet with unknown SSRC (2297016547)
[Mon Dec 12 14:52:05 2022] [1726755082988956] Dropping RTCP packet with unknown SSRC (2297016547)
[Mon Dec 12 14:52:06 2022] [1726755082988956] Dropping RTCP packet with unknown SSRC (2297016547)
[Mon Dec 12 14:52:07 2022] [1726755082988956] Dropping RTCP packet with unknown SSRC (2297016547)
[Mon Dec 12 14:52:08 2022] [1726755082988956] Dropping RTCP packet with unknown SSRC (2297016547)

With rtx on, the negotiation completes but the feature will not actually work since Janus does not support same-ssrc rtx. On the other hand pion will not send RTCP to the rtx SSRC in this case.

The only solution I found is removing the offending SSRC lines from the remote SDP before setting remote desc in pion.