search

AlexxIT / go2rtc

Ultimate camera streaming application with support RTSP, RTMP, HTTP-FLV, WebRTC, MSE, HLS, MP4, MJPEG, HomeKit, FFmpeg, etc.
https://github.com/AlexxIT/Blog
MIT License
4.14k stars 335 forks source link

hanging consumer webrtc connections #1255

Open marcob79 opened 1 month ago

marcob79 commented 1 month ago

I had a go2rtc 1.9.4 instance running as binary on rpi4

one of the stream is configures as

picam_H264_lowrate: exec:libcamera-vid --nopreview --level 4.2 --profile high --mode 2304:1296 --width 1920 --height 1080 --framerate 15 -b 500000 -t 0 --denoise cdn_off -n --inline -o -

after some days, i noticed that there are many consumer sessions running (more than 15)

I don't understand if the setting

killsignal=2#killtimeout=5

can help to mitigate such behavior and it can be applicable also for such scenario

or eventually which kind of logs can be helpful to understand what's happening

stream info JSON output here below

{
  "producers": [
    {
      "id": 2,
      "format_name": "h264",
      "protocol": "pipe",
      "source": "exec:libcamera-vid --nopreview --level 4.2 --profile high --mode 2304:1296 --width 1920 --height 1080 --framerate 15 -b 500000 -t 0 --denoise cdn_off -n --inline -o -",
      "medias": [
        "video, recvonly, H264"
      ],
      "receivers": [
        {
          "id": 3,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video",
            "level": 42,
            "profile": "High"
          },
          "childs": [
            4,
            10,
            20,
            22,
            24,
            26,
            28,
            30,
            32,
            34,
            36,
            45,
            47,
            49,
            51,
            56,
            63
          ],
          "bytes": 16731736006,
          "packets": 4015611
        }
      ],
      "bytes_recv": 16731738112
    }
  ],
  "consumers": [
    {
      "id": 1,
      "format_name": "webrtc",
      "protocol": "ws+udp",
      "remote_addr": "xx.yy.zzz.121:51098 srflx 0.0.0.0:51098",
      "user_agent": "Go-http-client/1.1",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, OPUS/48000/2, PCMU/8000, PCMA/8000, L16, PCML",
        "audio, recvonly, OPUS/48000/2, PCMU/8000, PCMA/8000"
      ],
      "senders": [
        {
          "id": 4,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "parent": 3,
          "bytes": 16731736006,
          "packets": 4015611
        }
      ],
      "bytes_send": 16934746016
    },
    {
      "id": 9,
      "format_name": "webrtc",
      "protocol": "ws+udp",
      "remote_addr": "xx.yy.zzz.121:54775 srflx 0.0.0.0:54775",
      "user_agent": "Go-http-client/1.1",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, OPUS/48000/2, PCMU/8000, PCMA/8000, L16, PCML",
        "audio, recvonly, OPUS/48000/2, PCMU/8000, PCMA/8000"
      ],
      "senders": [
        {
          "id": 10,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "parent": 3,
          "bytes": 16393879623,
          "packets": 3934528
        }
      ],
      "bytes_send": 16592766584
    },
    {
      "id": 19,
      "format_name": "webrtc",
      "protocol": "ws+udp",
      "remote_addr": "xx.yy.zzz.121:52974 srflx 0.0.0.0:52974",
      "user_agent": "Go-http-client/1.1",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, OPUS/48000/2, PCMU/8000, PCMA/8000, L16, PCML",
        "audio, recvonly, OPUS/48000/2, PCMU/8000, PCMA/8000"
      ],
      "senders": [
        {
          "id": 20,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "parent": 3,
          "bytes": 16021875796,
          "packets": 3845262
        }
      ],
      "bytes_send": 16216217957
    },
    {
      "id": 21,
      "format_name": "webrtc",
      "protocol": "ws+udp",
      "remote_addr": "xx.yy.zzz.121:48745 srflx 0.0.0.0:48745",
      "user_agent": "Go-http-client/1.1",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, OPUS/48000/2, PCMU/8000, PCMA/8000, L16, PCML",
        "audio, recvonly, OPUS/48000/2, PCMU/8000, PCMA/8000"
      ],
      "senders": [
        {
          "id": 22,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "parent": 3,
          "bytes": 15960627688,
          "packets": 3830547
        }
      ],
      "bytes_send": 16154222620
    },
    {
      "id": 23,
      "format_name": "webrtc",
      "protocol": "ws+udp",
      "remote_addr": "xx.yy.zzz.121:41298 srflx 0.0.0.0:41298",
      "user_agent": "Go-http-client/1.1",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, OPUS/48000/2, PCMU/8000, PCMA/8000, L16, PCML",
        "audio, recvonly, OPUS/48000/2, PCMU/8000, PCMA/8000"
      ],
      "senders": [
        {
          "id": 24,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "parent": 3,
          "bytes": 15922306335,
          "packets": 3821348
        }
      ],
      "bytes_send": 16115431835
    },
    {
      "id": 25,
      "format_name": "webrtc",
      "protocol": "ws+udp",
      "remote_addr": "xx.yy.zzz.121:42384 srflx 0.0.0.0:42384",
      "user_agent": "Go-http-client/1.1",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, OPUS/48000/2, PCMU/8000, PCMA/8000, L16, PCML",
        "audio, recvonly, OPUS/48000/2, PCMU/8000, PCMA/8000"
      ],
      "senders": [
        {
          "id": 26,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "parent": 3,
          "bytes": 15915999446,
          "packets": 3819834
        }
      ],
      "bytes_send": 16109048139
    },
    {
      "id": 27,
      "format_name": "webrtc",
      "protocol": "ws+udp",
      "remote_addr": "xx.yy.zzz.121:54748 srflx 0.0.0.0:54748",
      "user_agent": "Go-http-client/1.1",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, OPUS/48000/2, PCMU/8000, PCMA/8000, L16, PCML",
        "audio, recvonly, OPUS/48000/2, PCMU/8000, PCMA/8000"
      ],
      "senders": [
        {
          "id": 28,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "parent": 3,
          "bytes": 15073896973,
          "packets": 3617732
        }
      ],
      "bytes_send": 15256675157
    },
    {
      "id": 29,
      "format_name": "webrtc",
      "protocol": "ws+udp",
      "remote_addr": "xx.yy.zzz.121:40850 srflx 0.0.0.0:40850",
      "user_agent": "Go-http-client/1.1",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, OPUS/48000/2, PCMU/8000, PCMA/8000, L16, PCML",
        "audio, recvonly, OPUS/48000/2, PCMU/8000, PCMA/8000"
      ],
      "senders": [
        {
          "id": 30,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "parent": 3,
          "bytes": 15053264117,
          "packets": 3612778
        }
      ],
      "bytes_send": 15235791811
    },
    {
      "id": 31,
      "format_name": "webrtc",
      "protocol": "ws",
      "user_agent": "Go-http-client/1.1",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, OPUS/48000/2, PCMU/8000, PCMA/8000, L16, PCML",
        "audio, recvonly, OPUS/48000/2, PCMU/8000, PCMA/8000"
      ],
      "senders": [
        {
          "id": 32,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "parent": 3,
          "bytes": 17123495,
          "packets": 4096,
          "drops": 3531280
        }
      ]
    },
    {
      "id": 33,
      "format_name": "webrtc",
      "protocol": "ws+udp",
      "remote_addr": "xx.yy.zzz.121:44558 srflx 0.0.0.0:44558",
      "user_agent": "Go-http-client/1.1",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, OPUS/48000/2, PCMU/8000, PCMA/8000, L16, PCML",
        "audio, recvonly, OPUS/48000/2, PCMU/8000, PCMA/8000"
      ],
      "senders": [
        {
          "id": 34,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "parent": 3,
          "bytes": 14505613929,
          "packets": 3481357
        }
      ],
      "bytes_send": 14681486104
    },
    {
      "id": 35,
      "format_name": "webrtc",
      "protocol": "ws+udp",
      "remote_addr": "xx.yy.zzz.121:55413 srflx 0.0.0.0:55413",
      "user_agent": "Go-http-client/1.1",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, OPUS/48000/2, PCMU/8000, PCMA/8000, L16, PCML",
        "audio, recvonly, OPUS/48000/2, PCMU/8000, PCMA/8000"
      ],
      "senders": [
        {
          "id": 36,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "parent": 3,
          "bytes": 11816367977,
          "packets": 2835943
        }
      ],
      "bytes_send": 11959684332
    },
    {
      "id": 44,
      "format_name": "webrtc",
      "protocol": "ws+udp",
      "remote_addr": "xx.yy.zzz.121:45766 srflx 0.0.0.0:45766",
      "user_agent": "Go-http-client/1.1",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, OPUS/48000/2, PCMU/8000, PCMA/8000, L16, PCML",
        "audio, recvonly, OPUS/48000/2, PCMU/8000, PCMA/8000"
      ],
      "senders": [
        {
          "id": 45,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "parent": 3,
          "bytes": 6276767612,
          "packets": 1506422
        }
      ],
      "bytes_send": 6352807386
    },
    {
      "id": 46,
      "format_name": "webrtc",
      "protocol": "ws+udp",
      "remote_addr": "xx.yy.zzz.121:52787 srflx 0.0.0.0:52787",
      "user_agent": "Go-http-client/1.1",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, OPUS/48000/2, PCMU/8000, PCMA/8000, L16, PCML",
        "audio, recvonly, OPUS/48000/2, PCMU/8000, PCMA/8000"
      ],
      "senders": [
        {
          "id": 47,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "parent": 3,
          "bytes": 5663882449,
          "packets": 1359348
        }
      ],
      "bytes_send": 5732448934
    },
    {
      "id": 48,
      "format_name": "webrtc",
      "protocol": "ws+udp",
      "remote_addr": "xx.yy.zzz.121:41250 srflx 0.0.0.0:41250",
      "user_agent": "Go-http-client/1.1",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, OPUS/48000/2, PCMU/8000, PCMA/8000, L16, PCML",
        "audio, recvonly, OPUS/48000/2, PCMU/8000, PCMA/8000"
      ],
      "senders": [
        {
          "id": 49,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "parent": 3,
          "bytes": 4727321210,
          "packets": 1134556
        }
      ],
      "bytes_send": 4784497018
    },
    {
      "id": 50,
      "format_name": "webrtc",
      "protocol": "ws",
      "user_agent": "Go-http-client/1.1",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, OPUS/48000/2, PCMU/8000, PCMA/8000, L16, PCML",
        "audio, recvonly, OPUS/48000/2, PCMU/8000, PCMA/8000"
      ],
      "senders": [
        {
          "id": 51,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "parent": 3,
          "bytes": 17112666,
          "packets": 4096,
          "drops": 96701
        }
      ]
    },
    {
      "id": 55,
      "format_name": "webrtc",
      "protocol": "ws+udp",
      "remote_addr": "xx.yy.zzz.121:49998 srflx 0.0.0.0:49998",
      "user_agent": "Go-http-client/1.1",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, OPUS/48000/2, PCMU/8000, PCMA/8000, L16, PCML",
        "audio, recvonly, OPUS/48000/2, PCMU/8000, PCMA/8000"
      ],
      "senders": [
        {
          "id": 56,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "parent": 3,
          "bytes": 418863967,
          "packets": 100539
        }
      ],
      "bytes_send": 423968399
    },
    {
      "id": 62,
      "format_name": "webrtc",
      "protocol": "ws+udp",
      "remote_addr": "xx.yy.zzz.121:42335 srflx 0.0.0.0:42335",
      "user_agent": "Go-http-client/1.1",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, OPUS/48000/2, PCMU/8000, PCMA/8000, L16, PCML",
        "audio, recvonly, OPUS/48000/2, PCMU/8000, PCMA/8000"
      ],
      "senders": [
        {
          "id": 63,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "parent": 3,
          "bytes": 194845330,
          "packets": 46776
        }
      ],
      "bytes_send": 197224186
    }
  ]
}
AlexxIT commented 1 month ago

What is "Go-http-client/1.1"? I don't think this problem is producer related. I think problem with consumer.

marcob79 commented 1 month ago

Go-http-client/1.1 should be a session of WebRTC installed on my Home Assistant that is referencing the go2rtc instance I was referring to in the ticket, that is a remote go2rtc instance

AlexxIT commented 1 month ago

Show config for your "WebRTC installed on my Home Assistant". You can hide any private data.

marcob79 commented 1 month ago

here it is. usually I use the 1st and I didn't cycle to other possible links

type: custom:webrtc-camera
media: video
ui: true
streams:
  - url: >-
      webrtc:ws://externalGO2RTCurl/api/ws?src=picam_H264_lowrate
    name: LR
  - url: >-
      webrtc:ws://externalGO2RTCurl/api/ws?src=picam_H264_FHD
    name: HR
  - url: wetorrent_link_lowrate
    name: webtorrentLR
  - url: webtorrent_link
    name: webtorrentHR
AlexxIT commented 1 month ago

It's better to use server option - https://github.com/AlexxIT/WebRTC?tab=readme-ov-file#custom-card

But anyway webrtc:ws source has a bug with handling broken connections.

marcob79 commented 1 month ago

interesting.... I'll have a try to server option. On top of that, how I suppose to mange my url together with server option?

it should become like the following?

type: custom:webrtc-camera
media: video
ui: true
server: https://externalGO2RTCurl
streams:
  - url: picam_H264_lowrate
    name: LR
  - url: picam_H264_FHD
    name: HR
  - url: wetorrent_link_lowrate
    name: webtorrentLR
  - url: webtorrent_link
    name: webtorrentHR

thanks

AlexxIT commented 1 month ago

Server will be same for all streams

marcob79 commented 1 month ago

is the "server" option for custom:webrtc-camera also suggested (referring to bad handling of broken connection) to link to the home assistant addon that is running "locally", not on a different network as the above? thanks

AlexxIT commented 1 month ago

I don't understand your question. You can set go2rtc server URL when setup WebRTC integration. But you can overwrite this params via this card option. This is useful when you have multiple servers. If you have only one remote server - it's better to set it on setup step.