WebRTC: TCP packet is invalid for session not found when using replicas > 1

Describe the bug Actually, I'm not sure it's a bug or it's caused by wrong srs configuration.

I deployed SRS5 in K8s enviroment. and set replicas as 2. both of 2 SRS pods are sharing same FQDN (ingress) WebRTC: test-k8s-srs-api.dev.net API: test-k8s-srs-webrtc.dev.net

then when playing WebRTC. it's possible that 2 requests doesn't send into the same pod and leads to such error "(RTC TCP packet is invalid for session not found) : process rtc tcp pkt : no session"

I was wondering if there's any way which can resolve WebRTC running on more than 2 pods (k8s env.)

Version SRS5

To Reproduce Steps to reproduce the behavior:

deploy SRS in K8s
set replicas as 2
use built-in SRS web player to play WebRTC
sometimes cannot play stream dut to session not found (not on the same pod)

Expected behavior in K8s, when replicas > 1, still can always play WebRTC successfully

Screenshots

[pod-1]

[2024-01-08 07:24:06.430][INFO][1][8i714808] HTTP #0 10.42.212.2:53934 POST http://test-k8s-srs-api.dev.net/rtc/v1/play/, content-length=6149
[2024-01-08 07:24:06.430][INFO][1][8i714808] RTC play webrtc://test-k8s-srs-webrtc.dev.net/live/test123, api=https://test-k8s-srs-api.dev.net/rtc/v1/play/, tid=73d60c4, clientip=10.42.187.64, app=live, stream=test123, offer=5571B, eip=, codec=, srtp=, dtls=
[2024-01-08 07:24:06.430][INFO][1][8i714808] ignore attribute=, value=
[2024-01-08 07:24:06.430][INFO][1][8i714808] RTC player nack=1, nnc=1
[2024-01-08 07:24:06.431][INFO][1][8i714808] RTC connection player gcc=4
[2024-01-08 07:24:06.431][INFO][1][8i714808] RTC: Init tracks {track: video-53o5qx56, is_active: 0=>1}, ok
[2024-01-08 07:24:06.596][INFO][1][8i714808] RTC: Use candidates 172.17.15.6, 172.17.2.21, protocol=tcp
[2024-01-08 07:24:06.597][INFO][1][8i714808] RTC init session, user=zzc5724a:BI6f, url=/live/test123, encrypt=1/1, DTLS(role=passive, version=auto), timeout=30000ms, nack=1
[2024-01-08 07:24:06.597][INFO][1][8i714808] RTC username=zzc5724a:BI6f, dtls=1, srtp=1, offer=5571B, answer=988B
[2024-01-08 07:24:06.597][INFO][1][8i714808] RTC remote offer: v=0\r\no=- 745905371023040234 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0\r\na=extmap-allow-mixed\r\na=msid-semantic: WMS\r\nm=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 35 36 37 38 102 103 104 105 106 107 108 109 127 125 39 40 41 42 43 44 45 46 47 48 112 113 114 115 116 117 118 49\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:BI6f\r\na=ice-pwd:SZbiQlmBFZSjOf2RFUOJEnUA\r\na=ice-options:trickle\r\na=fingerprint:sha-256 62:99:B1:A3:EB:81:AF:4D:8D:19:4E:A2:FE:17:26:69:78:76:4D:F6:5E:C6:FD:91:23:18:4B:0C:BA:1B:7D:C4\r\na=setup:actpass\r\na=mid:0\r\na=extmap:1 urn:ietf:params:rtp-hdrext:toffset\r\na=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:3 urn:3gpp:video-orientation\r\na=extmap:4 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay\r\na=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type\r\na=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing\r\na=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space\r\na=extmap:9 urn:ietf:params:rtp-hdrext:sdes:mid\r\na=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id\r\na=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id\r\na=recvonly\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:96 VP8/90000\r\na=rtcp-fb:96 goog-remb\r\na=rtcp-fb:96 transport-cc\r\na=rtcp-fb:96 ccm fir\r\na=rtcp-fb:96 nack\r\na=rtcp-fb:96 nack pli\r\na=rtpmap:97 rtx/90000\r\na=fmtp:97 apt=96\r\na=rtpmap:98 VP9/90000\r\na=rtcp-fb:98 goog-remb\r\na=rtcp-fb:98 transport-cc\r\na=rtcp-fb:98 ccm fir\r\na=rtcp-fb:98 nack\r\na=rtcp-fb:98 nack pli\r\na=fmtp:98 profile-id=0\r\na=rtpmap:99 rtx/90000\r\na=fmtp:99 apt=98\r\na=rtpmap:100 VP9/90000\r\na=rtcp-fb:100 goog-remb\r\na=rtcp-fb:100 transport-cc\r\na=rtcp-fb:100 ccm fir\r\na=rtcp-fb:100 nack\r\na=rtcp-fb:100 nack pli\r\na=fmtp:100 profile-id=2\r\na=rtpmap:101 rtx/90000\r\na=fmtp:101 apt=100\r\na=rtpmap:35 VP9/90000\r\na=rtcp-fb:35 goog-remb\r\na=rtcp-fb:35 transport-cc\r\na=rtcp-fb:35 ccm fir\r\na=rtcp-fb:35 nack\r\na=rtcp-fb:35 nack pli\r\na=fmtp:35 profile-id=1\r\na=rtpmap:36 rtx/90000\r\na=fmtp:36 apt=35\r\na=rtpmap:37 VP9/90000\r\na=rtcp-fb:37 goog-remb\r\na=rtcp-fb:37 transport-cc\r\na=rtcp-fb:37 ccm fir\r\na=rtcp-fb:37 nack\r\na=rtcp-fb:37 nack pli\r\na=fmtp:37 profile-id=3\r\na=rtpmap:38 rtx/90000\r\na=fmtp:38 apt=37\r\na=rtpmap:102 H264/90000\r\na=rtcp-fb:102 goog-remb\r\na=rtcp-fb:102 transport-cc\r\na=rtcp-fb:102 ccm fir\r\na=rtcp-fb:102 nack\r\na=rtcp-fb:102 nack pli\r\na=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f\r\na=rtpmap:103 rtx/90000\r\na=fmtp:103 apt=102\r\na=rtpmap:104 H264/90000\r\na=rtcp-fb:104 goog-remb\r\na=rtcp-fb:104 transport-cc\r\na=rtcp-fb:104 ccm fir\r\na=rtcp-fb:104 nack\r\na=rtcp-fb:104 nack pli\r\na=fmtp:104 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f\r\na=rtpmap:105 rtx/90000\r\na=fmtp:105 apt=104\r\na=rtpmap:106 H264/90000\r\na=rtcp-fb:106 goog-remb\r\na=rtcp-fb:106 transport-cc\r\na=rtcp-fb:106 ccm fir\r\na=rtcp-fb:106 nack\r\na=rtcp-fb:106 nack pli\r\na=fmtp:106 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\na=rtpmap:107 rtx/90000\r\na=fmtp:107 apt=106\r\na=rtpmap:108 H264/90000\r\na=rtcp-fb:108 goog-remb\r\na=rtcp-fb:108 transport-cc\r\na=rtcp-fb:108 ccm fir\r\na=rtcp-fb:108 nack\r\na=rtcp-fb:108 nack pli\r\na=fmtp:108 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f\r\na=rtpmap:109 rtx/90000\r\na=fmtp:109 apt=108\r\na=rtpmap:127 H264/90000\r\na=rtcp-fb:127 goog-remb\r\na=rtcp-fb:127 transport-cc\r\na=rtcp-fb:127 ccm fir\r\na=rtcp-fb:127 nack\r\na=rtcp-fb:127 nack pli\r\na=fmtp:127 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=4d001f\r\na=rtpmap:125 rtx/90000\r\na=fmtp:125 apt=127\r\na=rtpmap:39 H264/90000\r\na=rtcp-fb:39 goog-remb\r\na=rtcp-fb:39 transport-cc\r\na=rtcp-fb:39 ccm fir\r\na=rtcp-fb:39 nack\r\na=rtcp-fb:39 nack pli\r\na=fmtp:39 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=4d001f\r\na=rtpmap:40 rtx/90000\r\na=fmtp:40 apt=39\r\na=rtpmap:41 H264/90000\r\na=rtcp-fb:41 goog-remb\r\na=rtcp-fb:41 transport-cc\r\na=rtcp-fb:41 ccm fir\r\na=rtcp-fb:41 nack\r\na=rtcp-fb:41 nack pli\r\na=fmtp:41 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=f4001f\r\na=rtpmap:42 rtx/90000\r\na=fmtp:42 apt=41\r\na=rtpmap:43 H264/90000\r\na=rtcp-fb:43 goog-remb\r\na=rtcp-fb:43 transport-cc\r\na=rtcp-fb:43 ccm fir\r\na=rtcp-fb:43 nack\r\na=rtcp-fb:43 nack pli\r\na=fmtp:43 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=f4001f\r\na=rtpmap:44 rtx/90000\r\na=fmtp:44 apt=43\r\na=rtpmap:45 AV1/90000\r\na=rtcp-fb:45 goog-remb\r\na=rtcp-fb:45 transport-cc\r\na=rtcp-fb:45 ccm fir\r\na=rtcp-fb:45 nack\r\na=rtcp-fb:45 nack pli\r\na=rtpmap:46 rtx/90000\r\na=fmtp:46 apt=45\r\na=rtpmap:47 AV1/90000\r\na=rtcp-fb:47 goog-remb\r\na=rtcp-fb:47 transport-cc\r\na=rtcp-fb:47 ccm fir\r\na=rtcp-fb:47 nack\r\na=rtcp-fb:47 nack pli\r\na=fmtp:47 profile=1\r\na=rtpmap:48 rtx/90000\r\na=fmtp:48 apt=47\r\na=rtpmap:112 H264/90000\r\na=rtcp-fb:112 goog-remb\r\na=rtcp-fb:112 transport-cc\r\na=rtcp-fb:112 ccm fir\r\na=rtcp-fb:112 nack\r\na=rtcp-fb:112 nack pli\r\na=fmtp:112 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=64001f\r\na=rtpmap:113 rtx/90000\r\na=fmtp:113 apt=112\r\na=rtpmap:114 H264/90000\r\na=rtcp-fb:114 goog-remb\r\na=rtcp-fb:114 transport-cc\r\na=rtcp-fb:114 ccm fir\r\na=rtcp-fb:114 nack\r\na=rtcp-fb:114 nack pli\r\na=fmtp:114 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=64001f\r\na=rtpmap:115 rtx/90000\r\na=fmtp:115 apt=114\r\na=rtpmap:116 red/90000\r\na=rtpmap:117 rtx/90000\r\na=fmtp:117 apt=116\r\na=rtpmap:118 ulpfec/90000\r\na=rtpmap:49 flexfec-03/90000\r\na=rtcp-fb:49 goog-remb\r\na=rtcp-fb:49 transport-cc\r\na=fmtp:49 repair-window=10000000\r\n
[2024-01-08 07:24:06.597][INFO][1][8i714808] RTC local answer: v=0\r\no=SRS/5.0.205(Bee) 140292691944016 2 IN IP4 0.0.0.0\r\ns=SRSPlaySession\r\nt=0 0\r\na=ice-lite\r\na=group:BUNDLE 0\r\na=msid-semantic: WMS live/test123\r\nm=video 9 UDP/TLS/RTP/SAVPF 106\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:zzc5724a\r\na=ice-pwd:n305085962671756n5227334ce14o19r\r\na=fingerprint:sha-256 46:96:CA:D4:84:C8:A2:39:2D:F3:F0:6C:B4:03:AF:B5:9B:99:99:3C:98:BB:40:68:91:CC:E4:5D:D9:77:A8:E2\r\na=setup:passive\r\na=mid:0\r\na=extmap:4 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=sendonly\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:106 H264/90000\r\na=rtcp-fb:106 transport-cc\r\na=rtcp-fb:106 nack\r\na=rtcp-fb:106 nack pli\r\na=fmtp:106 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\na=ssrc:10256 cname:f606550579162u0f\r\na=ssrc:10256 label:video-53o5qx56\r\na=candidate:0 1 tcp 2130706431 172.17.15.6 8000 typ host tcptype passive\r\na=candidate:1 1 tcp 2130706431 172.17.2.21 8000 typ host tcptype passive\r\n
[2024-01-08 07:24:06.600][INFO][1][8i714808] TCP: before dispose resource(HttpConn)(0x7f987013a5c0), conns=3, zombies=0, ign=0, inz=0, ind=0
[2024-01-08 07:24:06.600][WARN][1][8i714808][104] client disconnect peer. ret=1007
[2024-01-08 07:24:06.600][INFO][1][8i714808] TCP: disposing #0 resource(HttpConn)(0x7f987013a5c0), conns=3, disposing=1, zombies=0

------------------------------------

[pod-2]

[2024-01-08 07:24:06.632][INFO][1][16501122] TCP: before dispose resource(Tcp)(0x7f2ac80821e0), conns=2, zombies=0, ign=0, inz=0, ind=0
[2024-01-08 07:24:06.632][ERROR][1][16501122][11] serve error code=5034(RtcTcpSession)(RTC TCP packet is invalid for session not found) : process rtc tcp pkt : no session, stun username=zzc5724a:BI6f
thread [1][16501122]: do_cycle() [./src/app/srs_app_rtc_network.cpp:805][errno=11]
thread [1][16501122]: handshake() [./src/app/srs_app_rtc_network.cpp:853][errno=11](Resource temporarily unavailable)
[2024-01-08 07:24:06.632][INFO][1][16501122] TCP: disposing #0 resource(Tcp)(0x7f2ac80821e0), conns=2, disposing=1, zombies=0

Additional context

# main config for srs.
# @see full.conf for detail config.
listen              1935;
max_connections     7500;
srs_log_tank        file;
srs_log_file        ./objs/logs/srs.log;
daemon              off;

inotify_auto_reload  on;
auto_reload_for_docker on;

http_api {
    enabled         on;
    listen          1985;
}

http_server {
    enabled         on;
    listen          8080;
    dir             ./objs/nginx/html;
}

stats {
    network         0;
    disk            sda sdb xvda xvdb;
}

rtc_server {
    enabled on;
    listen 8000;
    tcp {
        enabled on;
        listen 8000;
    }
    protocol  tcp;
    candidate $SRS_RTC_SERVER_CANDIDATE;
}

vhost __defaultVhost__ {
    min_latency on;
    tcp_nodelay on;

    publish {
        mr      off;
    }

    play{
        gop_cache on;
        queue_length 5;
        gop_cache_max_frames 2500;
        mw_latency 100;
    }

    rtc {
        enabled     on;
        rtmp_to_rtc on;
        rtc_to_rtmp on;
    }
}

ossrs / srs

WebRTC: TCP packet is invalid for session not found when using replicas > 1 #3927