Webrtc frame skipping - Githubissues

Ignattious commented 8 months ago

Im seeing some weird behaviour using webrtc where I get frame skipping / stuttering every 2 seconds.

The issue isnt present when using mse whatsoever. Ive tried the default Frigate version of go2rtc and also updated to 1.8.4 with no change.

{
  "producers": [
    {
      "type": "RTSP active producer",
      "url": "rtsp://10.200.128.50:554/cam/realmonitor?channel=1\u0026subtype=0/",
      "remote_addr": "10.200.128.50:554",
      "user_agent": "go2rtc/1.8.4",
      "sdp": "v=0\r\no=- 2251940367 2251940367 IN IP4 0.0.0.0\r\ns=Media Server\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\na=control:*\r\na=packetization-supported:DH\r\na=rtppayload-supported:DH\r\na=range:npt=now-\r\nm=video 0 RTP/AVP 96\r\na=control:trackID=0\r\na=framerate:25.000000\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1;profile-level-id=4D0033;sprop-parameter-sets=Z00AM+kAeACH0IAAAfSAAGHBAgA=,aOqPIAA=\r\na=recvonly\r\nm=audio 0 RTP/AVP 8\r\na=control:trackID=5\r\na=rtpmap:8 PCMA/8000\r\na=sendonly\r\n",
      "medias": [
        "video, recvonly, H.264 Main 5.1",
        "audio, sendonly, PCMA/8000"
      ],
      "receivers": [
        "96 H264, bytes=165715478, senders=3"
      ],
      "recv": 167130230
    }
  ],
  "consumers": [
    {
      "type": "RTSP passive consumer",
      "url": "rtsp://127.0.0.1:8554/front",
      "remote_addr": "127.0.0.1:55208",
      "user_agent": "FFmpeg Frigate/0.12.1-367d724",
      "sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.8.4\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1;profile-level-id=4D0033;sprop-parameter-sets=Z00AM+kAeACH0IAAAfSAAGHBAgA=,aOqPIAA=\r\na=control:trackID=0\r\n",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, MPEG4-GENERIC"
      ],
      "senders": [
        "96 H264, bytes=165715478, receivers=1"
      ],
      "send": 167591030
    },
    {
       "type": "WebRTC/WebSocket async passive consumer",
      "remote_addr": "tcp4 prflx 10.200.120.253:54068 related :0",
      "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36",
      "medias": [
        "video, sendonly, VP8, RTX, VP9, H264, AV1, RED, ULPFEC, FLEXFEC-03",
        "audio, sendonly, OPUS/48000/2, RED/48000/2, G722/8000, PCMU/8000, PCMA/8000, CN/8000, TELEPHONE-EVENT/48000, TELEPHONE-EVENT/8000, L16, PCML"
      ],
      "senders": [
        "102 H264, bytes=113648791, receivers=1"
      ],
      "send": 114843212
    },
    {
      "type": "RTSP passive consumer",
      "url": "rtsp://10.200.128.100:8554/front",
      "remote_addr": "10.200.128.200:57216",
      "user_agent": "Lavf59.27.100",
      "sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.8.4\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1;profile-level-id=4D0033;sprop-parameter-sets=Z00AM+kAeACH0IAAAfSAAGHBAgA=,aOqPIAA=\r\na=control:trackID=0\r\n",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, MPEG4-GENERIC"
      ],
      "senders": [
        "96 H264, bytes=71272938, receivers=1"
      ],
      "send": 72074346
    }
  ]
}

Ignattious commented 8 months ago

Just a quick update there is nothing in the chrome debug stats outside of:

total freezeduration 21.141 freezecount 109

I have ruled out any network or transport related issues. MSE/Websocket is buttery smooth.

No dropped frames are detected

AlexxIT commented 8 months ago

You should try to use go2rtc addon. I can see that WebRTC works for you via TCP. This is strange because for LAN connection it should be UDP. Maybe problem related to port forwarding for Frigate docker container.

Ignattious commented 8 months ago

Its weird using firefox it uses UDP

{
  "producers": [
    {
      "type": "RTSP source",
      "url": "rtsp://10.200.128.50:554/cam/realmonitor?channel=1\u0026subtype=0/",
      "remote_addr": "10.200.128.50:554",
      "user_agent": "go2rtc/1.2.0",
      "medias": [
        "video, sendonly, 96 H264/90000",
        "audio, recvonly, 8 PCMA/8000"
      ],
      "tracks": [
        "96 H264/90000, sinks=3"
      ],
      "recv": 3031867332
    }
  ],
  "consumers": [
    {
      "type": "RTSP client",
      "url": "rtsp://127.0.0.1:8554/front",
      "remote_addr": "127.0.0.1:59940",
      "user_agent": "FFmpeg Frigate/0.12.1-367d724",
      "medias": [
        "video, recvonly, 96 H264/90000",
        "audio, recvonly, 0 MPEG4-GENERIC/0"
      ],
      "tracks": [
        "96 H264/90000, sinks=3"
      ],
      "send": 3031544484
    },
    {
      "type": "RTSP client",
      "url": "rtsp://10.200.128.100:8554/front",
      "remote_addr": "10.200.128.200:36750",
      "user_agent": "Lavf59.27.100",
      "medias": [
        "video, recvonly, 96 H264/90000",
        "audio, recvonly, 0 MPEG4-GENERIC/0"
      ],
      "tracks": [
        "96 H264/90000, sinks=3"
      ],
      "send": 3029767869
    },
    {
      "type": "WebRTC client",      "remote_addr": "udp4 prflx
10.200.128.67:58247 related :0",
      "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64;
rv:120.0) Gecko/20100101 Firefox/120.0",
      "send": 3073996
    }
  ]
}

On Sat, Dec 2, 2023 at 6:50 PM Alex X @.***> wrote:

You should try to use go2rtc addon. I can see that WebRTC works for you via TCP. This is strange because for LAN connection it should be UDP. Maybe problem related to port forwarding for Frigate docker container.

— Reply to this email directly, view it on GitHub https://github.com/AlexxIT/go2rtc/issues/789#issuecomment-1837078466, or unsubscribe https://github.com/notifications/unsubscribe-auth/AU556KKYKXUG6HLMJFVMVPDYHLMV7AVCNFSM6AAAAABACDQVUSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMZXGA3TQNBWGY . You are receiving this because you authored the thread.Message ID: @.***>

AlexxIT commented 8 months ago

And also has problems? What's the bitrate of the stream?

Ignattious commented 8 months ago

When using version v1.8.4 I have UDP mapped yet it uses TCP? Why ?

On the default version it uses UDP no problem with the same docker settings

On Sat, Dec 2, 2023 at 8:05 PM Alex X @.***> wrote:

And also has problems? What's the bitrate of the stream?

— Reply to this email directly, view it on GitHub https://github.com/AlexxIT/go2rtc/issues/789#issuecomment-1837094974, or unsubscribe https://github.com/notifications/unsubscribe-auth/AU556KLP2LM66SRDG5XRSEDYHLVNLAVCNFSM6AAAAABACDQVUSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMZXGA4TIOJXGQ . You are receiving this because you authored the thread.Message ID: @.***>

Ignattious commented 8 months ago

Bitrate is 8192 Kb/Sec @ 3840*2160

On Sat, Dec 2, 2023 at 8:05 PM Alex X @.***> wrote:

And also has problems? What's the bitrate of the stream?

— Reply to this email directly, view it on GitHub https://github.com/AlexxIT/go2rtc/issues/789#issuecomment-1837094974, or unsubscribe https://github.com/notifications/unsubscribe-auth/AU556KLP2LM66SRDG5XRSEDYHLVNLAVCNFSM6AAAAABACDQVUSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMZXGA4TIOJXGQ . You are receiving this because you authored the thread.Message ID: @.***>

AlexxIT commented 8 months ago

This is hi bitrate and can be a problem real time streaming for WebRTC

erenoglu commented 8 months ago

I'm seeing similar behavior but on both MSE and WebRTC. I tried to disable audio transcoding as well but no change. My streams are transcoded to H264 (from H265 of camera) by go2rtc. To avoid any other posssibilities, I'm testing using go2rtc web interface on port 1984 using "stream" link to test.

All my 4K streams are stuttering like this, skipping frames on both MSE & RTC. On the exact same device, ie a Chromebook or a PC, if I launch VLC and point it to the RTSP stream of go2rtc, like rtsp://go2rtc_ip:8554/Cameraname then it plays flawlessly. This confuses me as it happens in Chrome, Edge and even Thorium, while RTSP stream on VLC works perfectly fine.

AlexxIT / go2rtc

Webrtc frame skipping #789