search

AlexxIT / go2rtc

Ultimate camera streaming application with support RTSP, RTMP, HTTP-FLV, WebRTC, MSE, HLS, MP4, MJPEG, HomeKit, FFmpeg, etc.
https://github.com/AlexxIT/Blog
MIT License
4.66k stars 378 forks source link

2-way audio is not working on Amcrest AD410 doorbell #1156

Closed towerhand closed 3 months ago

towerhand commented 4 months ago

I'm using Frigate as an NVR, and I have been struggling to get the 2-way audio on the AD410 for a while since it is recommended to use the #backchannel=0 for the button to work correctly, but I have been reading that having a separate stream with &unicast=true&proto=Onvif will allow the 2-way audio to work too. However, it doesn't work with these settings:

go2rtc:
  streams:
    doorbell:
      - rtsp://admin:{FRIGATE_RTSP_PASSWORD}@192.168.40.2:554/cam/realmonitor?channel=1&subtype=0#backchannel=0
      - rtsp://admin:{FRIGATE_RTSP_PASSWORD}@192.168.40.2:554/cam/realmonitor?channel=1&subtype=1&unicast=true&proto=Onvif
      - ffmpeg:doorbell#audio=aac
go2rtc:
  streams:
    doorbell:
      - rtsp://admin:{FRIGATE_RTSP_PASSWORD}@192.168.40.2:554/cam/realmonitor?channel=1&subtype=0#backchannel=0
      - rtsp://admin:{FRIGATE_RTSP_PASSWORD}@192.168.40.2:554/cam/realmonitor?channel=1&subtype=1&unicast=true&proto=Onvif#backchannel=1
      - ffmpeg:doorbell#audio=aac
go2rtc:
  streams:
    doorbell:
      - rtsp://admin:{FRIGATE_RTSP_PASSWORD}@192.168.40.2:554/cam/realmonitor?channel=1&subtype=0#backchannel=0
      - rtsp://admin:{FRIGATE_RTSP_PASSWORD}@192.168.40.2:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif#backchannel=1
      - ffmpeg:doorbell#audio=aac

Go2rtc probe info:

{
  "producers": [
    {
      "type": "RTSP active producer",
      "url": "rtsp://192.168.40.2:554/cam/realmonitor?channel=1\u0026subtype=0/",
      "remote_addr": "192.168.40.2:554",
      "user_agent": "go2rtc/1.9.2",
      "sdp": "v=0\r\no=- 2251952516 2251952516 IN IP4 0.0.0.0\r\ns=Media Server\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\na=control:*\r\na=packetization-supported:DH\r\na=rtppayload-supported:DH\r\na=range:npt=now-\r\na=x-packetization-supported:IV\r\na=x-rtppayload-supported:IV\r\nm=video 0 RTP/AVP 96\r\na=control:trackID=0\r\na=framerate:15.000000\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1;profile-level-id=4D0032;sprop-parameter-sets=Z00AMukAUAHjQgAAB9IAAOqcCAA=,aOqPIAA=\r\na=recvonly\r\nm=audio 0 RTP/AVP 8\r\na=control:trackID=1\r\na=rtpmap:8 PCMA/8000\r\na=recvonly\r\n",
      "medias": [
        "video, recvonly, H.264 Main 5.0",
        "audio, recvonly, PCMA/8000"
      ],
      "receivers": [
        "96 H264, bytes=77299338378, senders=3",
        "8 PCMA/8000, bytes=589560960, senders=3"
      ],
      "recv": 78564077166
    },
    {
      "url": "rtsp://admin:pass@192.168.40.2:554/cam/realmonitor?channel=1\u0026subtype=0\u0026unicast=true\u0026proto=Onvif"
    },
    {
      "type": "RTSP passive producer",
      "url": "rtsp://127.0.0.1:8554/5f402a2ba0414b1503dd4a4436f299bc",
      "remote_addr": "127.0.0.1:50082",
      "user_agent": "ffmpeg/go2rtc",
      "sdp": "v=0\r\no=- 0 0 IN IP4 127.0.0.1\r\ns=go2rtc/1.9.2\r\nc=IN IP4 127.0.0.1\r\nt=0 0\r\na=tool:libavformat LIBAVFORMAT_VERSION\r\nm=audio 0 RTP/AVP 96\r\nb=AS:48\r\na=rtpmap:96 MPEG4-GENERIC/8000/1\r\na=fmtp:96 profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3; config=158856E500\r\na=control:streamid=0\r\n",
      "medias": [
        "audio, recvonly, MPEG4-GENERIC/8000"
      ],
      "receivers": [
        "96 MPEG4-GENERIC/8000, bytes=5827954, senders=2"
      ],
      "recv": 5902334
    }
  ],
  "consumers": [
    {
      "type": "RTSP passive consumer",
      "url": "rtsp://192.168.55.14:8554/doorbell?audio",
      "remote_addr": "192.168.55.14:56824",
      "user_agent": "FFmpeg Frigate/0.14.0-ec88752",
      "sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.9.2\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=audio 0 RTP/AVP 96\r\na=rtpmap:96 PCMA/8000\r\na=control:trackID=0\r\n",
      "medias": [
        "audio, sendonly, ANY"
      ],
      "senders": [
        "96 PCMA/8000, bytes=589560960, receivers=1"
      ],
      "send": 619039008
    },
    {
      "type": "RTSP passive consumer",
      "url": "rtsp://192.168.55.14:8554/doorbell",
      "remote_addr": "192.168.55.14:51856",
      "user_agent": "FFmpeg Frigate/0.14.0-ec88752",
      "sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.9.2\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1;profile-level-id=4D0032;sprop-parameter-sets=Z00AMukAUAHjQgAAB9IAAOqcCAA=,aOqPIAA=\r\na=control:trackID=0\r\nm=audio 0 RTP/AVP 97\r\na=rtpmap:97 MPEG4-GENERIC/8000\r\na=fmtp:97 profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3; config=158856E500\r\na=control:trackID=1\r\n",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, MPEG4-GENERIC"
      ],
      "senders": [
        "96 H264, bytes=50452459903, receivers=1",
        "97 MPEG4-GENERIC/8000, bytes=171604900, receivers=1"
      ],
      "send": 51193385036
    },
    {
      "type": "WebRTC/WebSocket async passive consumer",
      "user_agent": "Mozilla/5.0 (iPhone; CPU iPhone OS 17_5_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.5 Mobile/15E148 Safari/604.1",
      "medias": [
        "video, sendonly, H264, VP8, VP9, AV1",
        "audio, sendonly, OPUS/48000/2, G722/8000, PCMU/8000, PCMA/8000, L16, PCML"
      ],
      "senders": [
        "96 H264, bytes=0, receivers=1, overflow=28703699",
        "8 PCMA/8000, bytes=0, receivers=1, overflow=974582"
      ]
    },
    {
      "type": "probe",
      "remote_addr": "192.168.10.3, 192.168.55.11:60768",
      "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.5 Safari/605.1.15",
      "medias": [
        "video, sendonly, ALL",
        "audio, sendonly, ALL",
        "audio, recvonly, ANY"
      ],
      "senders": [
        "96 H264, bytes=0, receivers=1",
        "8 PCMA/8000, bytes=0, receivers=1",
        "96 MPEG4-GENERIC/8000, bytes=0, receivers=1"
      ]
    }
  ]
}

Probe with the 2-way working, but the backchannel=0 removed:

{
  "producers": [
    {
      "type": "RTSP active producer",
      "url": "rtsp://192.168.40.2:554/cam/realmonitor?channel=1\u0026subtype=0",
      "remote_addr": "192.168.40.2:554",
      "user_agent": "go2rtc/1.9.2",
      "sdp": "v=0\r\no=- 2252026594 2252026594 IN IP4 0.0.0.0\r\ns=Media Server\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\na=control:*\r\na=packetization-supported:DH\r\na=rtppayload-supported:DH\r\na=range:npt=now-\r\na=x-packetization-supported:IV\r\na=x-rtppayload-supported:IV\r\nm=video 0 RTP/AVP 96\r\na=control:trackID=0\r\na=framerate:15.000000\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1;profile-level-id=4D0032;sprop-parameter-sets=Z00AMukAUAHjQgAAB9IAAOqcCAA=,aOqPIAA=\r\na=recvonly\r\nm=audio 0 RTP/AVP 8\r\na=control:trackID=1\r\na=rtpmap:8 PCMA/8000\r\na=recvonly\r\nm=audio 0 RTP/AVP 8\r\na=control:trackID=5\r\na=rtpmap:8 PCMA/8000\r\na=sendonly\r\n",
      "medias": [
        "video, recvonly, H.264 Main 5.0",
        "audio, recvonly, PCMA/8000",
        "audio, sendonly, PCMA/8000"
      ],
      "receivers": [
        "96 H264, bytes=70882572, senders=2",
        "8 PCMA/8000, bytes=538560, senders=2"
      ],
      "recv": 72040308
    },
    {
      "url": "rtsp://admin:pass@192.168.40.2:554/cam/realmonitor?channel=1\u0026subtype=0\u0026unicast=true\u0026proto=Onvif"
    },
    {
      "type": "RTSP passive producer",
      "url": "rtsp://127.0.0.1:8554/5f402a2ba0414b1503dd4a4436f299bc",
      "remote_addr": "127.0.0.1:51924",
      "user_agent": "ffmpeg/go2rtc",
      "sdp": "v=0\r\no=- 0 0 IN IP4 127.0.0.1\r\ns=go2rtc/1.9.2\r\nc=IN IP4 127.0.0.1\r\nt=0 0\r\na=tool:libavformat LIBAVFORMAT_VERSION\r\nm=audio 0 RTP/AVP 96\r\nb=AS:48\r\na=rtpmap:96 MPEG4-GENERIC/8000/1\r\na=fmtp:96 profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3; config=158856E500\r\na=control:streamid=0\r\n",
      "medias": [
        "audio, recvonly, MPEG4-GENERIC/8000"
      ],
      "receivers": [
        "96 MPEG4-GENERIC/8000, bytes=101090, senders=2"
      ],
      "recv": 102254
    }
  ],
  "consumers": [
    {
      "type": "RTSP passive consumer",
      "url": "rtsp://192.168.55.14:8554/doorbell?audio",
      "remote_addr": "192.168.55.14:46928",
      "user_agent": "FFmpeg Frigate/0.14.0-ec88752",
      "sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.9.2\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=audio 0 RTP/AVP 96\r\na=rtpmap:96 PCMA/8000\r\na=control:trackID=0\r\n",
      "medias": [
        "audio, sendonly, ANY"
      ],
      "senders": [
        "96 PCMA/8000, bytes=707200, receivers=1"
      ],
      "send": 742560
    },
    {
      "type": "RTSP passive consumer",
      "url": "rtsp://192.168.55.14:8554/doorbell",
      "remote_addr": "192.168.55.14:48792",
      "user_agent": "FFmpeg Frigate/0.14.0-ec88752",
      "sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.9.2\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1;profile-level-id=4D0032;sprop-parameter-sets=Z00AMukAUAHjQgAAB9IAAOqcCAA=,aOqPIAA=\r\na=control:trackID=0\r\nm=audio 0 RTP/AVP 97\r\na=rtpmap:97 MPEG4-GENERIC/8000\r\na=fmtp:97 profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3; config=158856E500\r\na=control:trackID=1\r\n",
      "medias": [
        "video, sendonly, H264, H265",
        "audio, sendonly, MPEG4-GENERIC"
      ],
      "senders": [
        "96 H264, bytes=31072052, receivers=1",
        "97 MPEG4-GENERIC/8000, bytes=101090, receivers=1"
      ],
      "send": 31347359
    },
    {
      "type": "probe",
      "remote_addr": "192.168.10.3, 192.168.55.11:59800",
      "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.5 Safari/605.1.15",
      "medias": [
        "video, sendonly, ALL",
        "audio, sendonly, ALL",
        "audio, recvonly, ANY"
      ],
      "receivers": [
        "0 ANY, bytes=0, senders=0"
      ],
      "senders": [
        "96 H264, bytes=0, receivers=1",
        "8 PCMA/8000, bytes=0, receivers=1",
        "96 MPEG4-GENERIC/8000, bytes=0, receivers=1"
      ]
    }
  ]
}

I got some assistance from Nick (Frigate contributor), who suggested I create an issue here since it appears to be a go2rtc problem. I would appreciate some help in getting it fully working.

towerhand commented 4 months ago

And obviously, the two way audio doesn't work from the go2rtc dashboard either.

felipecrs commented 4 months ago

Why not simply:

streams:
  doorbell:
    - rtsp://admin:{FRIGATE_RTSP_PASSWORD}@192.168.40.2:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif

?

felipecrs commented 4 months ago

Also, make sure Frigate is not consuming this stream during your test. I suggest you try with go2rtc standalone and later migrate your findings to Frigate.

felipecrs commented 4 months ago

If you really want to use the 2-way audio capable stream in Frigate as well, make sure to set ?backchannel=0 when consuming it in the Frigate detect or record roles, as realized in https://github.com/AlexxIT/go2rtc/issues/1134#issuecomment-2129496238.

towerhand commented 4 months ago

Thanks, I'll try that this weekend. How are you quickly and easily accessing the separate stream for the two-way audio only when needed?

felipecrs commented 4 months ago

In the following link you can see my frigate.yaml, and then you can see that I have one exclusive stream only for 2-way audio communication. This stream isn't used anywhere, except in my Home Assistant Frigate Card where I want the 2-way audio to happen.

https://github.com/felipecrs/dahua-vto-on-home-assistant/blob/164d142b7ee5e473e657e663848c2a140a8cd981/frigate/frigate.yaml

I know Amcrest is Dahua under the hoods, but I don't know to how extent the same things apply. For example, I don't think you need to tinker with audio codecs (fix_vto_codecs.sh) like I do.

AlexxIT commented 4 months ago

Probe can discover two way audio. It works fine. Are you sure you using HTTPS in your browser?

towerhand commented 4 months ago

Thanks for your feedback. It looks like I read too much trying to set this up and overcomplicated things. It turned out the doorbell stream was giving wrong user/pass due to the "proto=Onvif" setting, but I removed it and now it seems to work. I'll do more testing tomorrow.

towerhand commented 3 months ago

Once again, thanks for the assistance. For future reference since I found a lot of conflicting information. The two-way audio works with these settings for the Amcrest AD410, go2rtc version 1.9.3, and Frigate version 0.14.0.

  streams:
    doorbell:
      - rtsp://admin:pass@ip:554/cam/realmonitor?channel=1&subtype=0#backchannel=0
      - rtsp://admin:pass@ip:554/cam/realmonitor?channel=1&subtype=0
      - ffmpeg:doorbell#audio=aac
    doorbell_sub:
      - rtsp://admin:pass@ip:554/cam/realmonitor?channel=1&subtype=1
      - ffmpeg:doorbell_sub#audio=aac
felipecrs commented 3 months ago

Are you basically saying that your doorbell doesn't support proto=Onvif?

towerhand commented 3 months ago

Are you basically saying that your doorbell doesn't support proto=Onvif?

II couldn't it to work with the "proto=Onvif" because it kept giving me wrong user/pass, even with VLC. So, it looks like the AD410 doesn't support it unless I'm missing something, but I'm not sure what else to do to confirm.

felipecrs commented 3 months ago

I think there's no secret. If:

rtsp://admin:pass@ip:554/cam/realmonitor?channel=1&subtype=0

Is working in VLC but:

rtsp://admin:pass@ip:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif

Is not, then I guess AD410 doesn't support it.

Fortunately, go2rtc's documentation doesn't suggest using unicast=true&proto=Onvif anywhere for Amcrest, so I don't think any documentation change is due.

image