AlexxIT / go2rtc

Ultimate camera streaming application with support RTSP, RTMP, HTTP-FLV, WebRTC, MSE, HLS, MP4, MJPEG, HomeKit, FFmpeg, etc.
https://github.com/AlexxIT/Blog
MIT License
3.56k stars 297 forks source link

H265 WEBRTC in Safari #1108

Open gonzalezcalleja opened 1 month ago

gonzalezcalleja commented 1 month ago

Hi,

I'm using go2rtc (v1.8.5) from frigate (dev) with an

image

This is my stream config:

streams:
      entrada3_sub:
      - ffmpeg:rtsp://REDACTED:REDACTED@192.168.6.227:554/Streaming/Channels/102/?transportmode=unicast.sdp
      - ffmpeg:entrada3_sub#audio=opus

This is the info json from go2rtc:

{
  "producers": [
    {
      "type": "RTSP passive producer",
      "url": "rtsp://127.0.0.1:8554/6d41568241d431f388997cb918f33d17",
      "remote_addr": "127.0.0.1:55768",
      "user_agent": "ffmpeg/go2rtc",
      "sdp": "v=0\r\no=- 0 0 IN IP4 127.0.0.1\r\ns=Media Presentation\r\nc=IN IP4 127.0.0.1\r\nt=0 0\r\na=tool:libavformat LIBAVFORMAT_VERSION\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H265/90000\r\na=fmtp:96 sprop-vps=QAEMAf//AWAAAAMAAAMAAAMAAAMAmawJ; sprop-sps=QgEBAWAAAAMAAAMAAAMAAAMAmaAFAgFx8uKrTuiS7/8AAQABbAgBSZcADN/mAEA=; sprop-pps=RAHAcvAbJJI=\r\na=control:streamid=0\r\nm=audio 0 RTP/AVP 97\r\na=rtpmap:97 MPEG4-GENERIC/16000/1\r\na=fmtp:97 profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3; config=1408\r\na=control:streamid=1\r\n",
      "medias": [
        "video, recvonly, H265",
        "audio, recvonly, MPEG4-GENERIC/16000"
      ],
      "receivers": [
        "96 H265, bytes=265854189, senders=2",
        "97 MPEG4-GENERIC/16000, bytes=4968659, senders=2"
      ],
      "recv": 273146732
    },
    {
      "type": "RTSP passive producer",
      "url": "rtsp://127.0.0.1:8554/b6cc377fae1838f4ab946a2de3c87d25",
      "remote_addr": "127.0.0.1:48578",
      "user_agent": "ffmpeg/go2rtc",
      "sdp": "v=0\r\no=- 0 0 IN IP4 127.0.0.1\r\ns=go2rtc/1.8.5\r\nc=IN IP4 127.0.0.1\r\nt=0 0\r\na=tool:libavformat LIBAVFORMAT_VERSION\r\nm=audio 0 RTP/AVP 96\r\nb=AS:64\r\na=rtpmap:96 opus/48000/2\r\na=control:streamid=0\r\n",
      "medias": [
        "audio, recvonly, OPUS/48000/2"
      ],
      "receivers": [
        "96 OPUS/48000/2, bytes=430744, senders=1"
      ],
      "recv": 466552
    }
  ],
  "consumers": [
    {
      "type": "RTSP passive consumer",
      "url": "rtsp://127.0.0.1:8554/entrada3_sub",
      "remote_addr": "127.0.0.1:47930",
      "user_agent": "FFmpeg Frigate/0.14.0-9680f2a",
      "sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.8.5\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H265/90000\r\na=fmtp:96 sprop-vps=QAEMAf//AWAAAAMAAAMAAAMAAAMAmawJ; sprop-sps=QgEBAWAAAAMAAAMAAAMAAAMAmaAFAgFx8uKrTuiS7/8AAQABbAgBSZcADN/mAEA=; sprop-pps=RAHAcvAbJJI=\r\na=control:trackID=0\r\nm=audio 0 RTP/AVP 97\r\na=rtpmap:97 MPEG4-GENERIC/16000\r\na=fmtp:97 profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3; config=1408\r\na=control:trackID=1\r\n",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, MPEG4-GENERIC"
      ],
      "senders": [
        "96 H265, bytes=265854189, receivers=1",
        "97 MPEG4-GENERIC/16000, bytes=4968659, receivers=1"
      ],
      "send": 273903888
    },
    {
      "type": "RTSP passive consumer",
      "url": "rtsp://127.0.0.1:8554/entrada3_sub?audio",
      "remote_addr": "127.0.0.1:48570",
      "user_agent": "go2rtc/ffmpeg",
      "sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.8.5\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=audio 0 RTP/AVP 96\r\na=rtpmap:96 MPEG4-GENERIC/16000\r\na=fmtp:96 profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3; config=1408\r\na=control:trackID=0\r\n",
      "medias": [
        "audio, sendonly, ANY"
      ],
      "senders": [
        "96 MPEG4-GENERIC/16000, bytes=247305, receivers=1"
      ],
      "send": 250345
    },
    {
      "type": "WebRTC/WebSocket async passive consumer",
      "remote_addr": "tcp4 prflx 192.168.1.162:57831 related :0",
      "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4.1 Safari/605.1.15",
      "medias": [
        "video, sendonly, H264, RTX, H265, VP8, VP9, AV1, RED, ULPFEC, FLEXFEC-03",
        "audio, sendonly, OPUS/48000/2, RED/48000/2, G722/8000, PCMU/8000, PCMA/8000, CN/8000, TELEPHONE-EVENT/48000, TELEPHONE-EVENT/8000, L16, PCML"
      ],
      "senders": [
        "104 H265, bytes=6084614, receivers=1",
        "111 OPUS/48000/2, bytes=198494, receivers=1"
      ],
      "send": 6360433
    }
  ]
}

Using:

image

WEBRTC is always "loading..." and it doesn't show anything (mse works well).

image

I know H265 support in Safari is experimental, but I understand this should work. My problem comes because the camera does not allow more than H265 and I would like to save the cost and CPU time of transcoding to H264, thanks for your support!

AlexxIT commented 1 month ago

I confirm that something is broken in the latest versions of Safari

gonzalezcalleja commented 1 month ago

Tested with iOS 17.4.1 (with H265 codec for safari enabled) with same failed result:

JSON INFO output:

{
  "producers": [
    {
      "type": "RTSP passive producer",
      "url": "rtsp://127.0.0.1:8554/6d41568241d431f388997cb918f33d17",
      "remote_addr": "127.0.0.1:38158",
      "user_agent": "ffmpeg/go2rtc",
      "sdp": "v=0\r\no=- 0 0 IN IP4 127.0.0.1\r\ns=Media Presentation\r\nc=IN IP4 127.0.0.1\r\nt=0 0\r\na=tool:libavformat LIBAVFORMAT_VERSION\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H265/90000\r\na=fmtp:96 sprop-vps=QAEMAf//AWAAAAMAAAMAAAMAAAMAmawJ; sprop-sps=QgEBAWAAAAMAAAMAAAMAAAMAmaAFAgFx8uKrTuiS7/8AAQABbAgCJVEADN/mAEA=; sprop-pps=RAHAcvAbJJI=\r\na=control:streamid=0\r\nm=audio 0 RTP/AVP 97\r\na=rtpmap:97 MPEG4-GENERIC/16000/1\r\na=fmtp:97 profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3; config=1408\r\na=control:streamid=1\r\n",
      "medias": [
        "video, recvonly, H265",
        "audio, recvonly, MPEG4-GENERIC/16000"
      ],
      "receivers": [
        "96 H265, bytes=2786124510, senders=2",
        "97 MPEG4-GENERIC/16000, bytes=125864422, senders=2"
      ],
      "recv": 2938887308
    },
    {
      "type": "RTSP passive producer",
      "url": "rtsp://127.0.0.1:8554/b6cc377fae1838f4ab946a2de3c87d25",
      "remote_addr": "127.0.0.1:48040",
      "user_agent": "ffmpeg/go2rtc",
      "sdp": "v=0\r\no=- 0 0 IN IP4 127.0.0.1\r\ns=go2rtc/1.8.5\r\nc=IN IP4 127.0.0.1\r\nt=0 0\r\na=tool:libavformat LIBAVFORMAT_VERSION\r\nm=audio 0 RTP/AVP 96\r\nb=AS:64\r\na=rtpmap:96 opus/48000/2\r\na=control:streamid=0\r\n",
      "medias": [
        "audio, recvonly, OPUS/48000/2"
      ],
      "receivers": [
        "96 OPUS/48000/2, bytes=70345, senders=1"
      ],
      "recv": 75933
    }
  ],
  "consumers": [
    {
      "type": "RTSP passive consumer",
      "url": "rtsp://127.0.0.1:8554/entrada3_sub",
      "remote_addr": "127.0.0.1:44078",
      "user_agent": "FFmpeg Frigate/0.14.0-9680f2a",
      "sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.8.5\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H265/90000\r\na=fmtp:96 sprop-vps=QAEMAf//AWAAAAMAAAMAAAMAAAMAmawJ; sprop-sps=QgEBAWAAAAMAAAMAAAMAAAMAmaAFAgFx8uKrTuiS7/8AAQABbAgCJVEADN/mAEA=; sprop-pps=RAHAcvAbJJI=\r\na=control:trackID=0\r\nm=audio 0 RTP/AVP 97\r\na=rtpmap:97 MPEG4-GENERIC/16000\r\na=fmtp:97 profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3; config=1408\r\na=control:trackID=1\r\n",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, MPEG4-GENERIC"
      ],
      "senders": [
        "96 H265, bytes=2786124510, receivers=1",
        "97 MPEG4-GENERIC/16000, bytes=125864422, receivers=1"
      ],
      "send": 2947373028
    },
    {
      "type": "RTSP passive consumer",
      "url": "rtsp://127.0.0.1:8554/entrada3_sub?audio",
      "remote_addr": "127.0.0.1:48028",
      "user_agent": "go2rtc/ffmpeg",
      "sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.8.5\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=audio 0 RTP/AVP 96\r\na=rtpmap:96 MPEG4-GENERIC/16000\r\na=fmtp:96 profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3; config=1408\r\na=control:trackID=0\r\n",
      "medias": [
        "audio, sendonly, ANY"
      ],
      "senders": [
        "96 MPEG4-GENERIC/16000, bytes=35523, receivers=1"
      ],
      "send": 35971
    },
    {
      "type": "WebRTC/WebSocket async passive consumer",
      "remote_addr": "udp4 prflx 192.168.1.133:53347 related :0",
      "user_agent": "Mozilla/5.0 (iPhone; CPU iPhone OS 17_4_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4.1 Mobile/15E148 Safari/604.1",
      "medias": [
        "video, sendonly, H264, RTX, H265, VP8, VP9, RED, ULPFEC, FLEXFEC-03",
        "audio, sendonly, OPUS/48000/2, RED/48000/2, G722/8000, PCMU/8000, PCMA/8000, CN/8000, TELEPHONE-EVENT/48000, TELEPHONE-EVENT/8000, L16, PCML"
      ],
      "senders": [
        "104 H265, bytes=997707, receivers=1",
        "111 OPUS/48000/2, bytes=70345, receivers=1"
      ],
      "send": 1069900
    }
  ]
}
hertzg commented 9 hours ago

To add a bit more info (and maybe confirm what @AlexxIT said about safari being broken), I can confirm that the traffic even though the video is stuck at loading.