search

AlexxIT / go2rtc

Ultimate camera streaming application with support RTSP, RTMP, HTTP-FLV, WebRTC, MSE, HLS, MP4, MJPEG, HomeKit, FFmpeg, etc.
https://github.com/AlexxIT/Blog
MIT License
7.17k stars 514 forks source link

Doorbird camera discussion #1060

Open AlexxIT opened 7 months ago

AlexxIT commented 7 months ago
oeiber commented 6 months ago

Hi, how can I help to integrate backchannel audio for doorbird devices?

AlexxIT commented 6 months ago

Developing two way without physical access to device is very hard task. There is no feedback. Theoretically only possible if there is some other camera nearby that I can watch and hear through.

oeiber commented 6 months ago

I'll give you access to my device. Here's the link to the api documentation: Doorbird API You can find related information on page 18.

AlexxIT commented 6 months ago

I also need yet another camera for feedback.

fergalom commented 6 months ago

I'll give you access to my device. Here's the link to the api documentation: Doorbird API You can find related information on page 18.

@oeiber I can help also as requested

oeiber commented 6 months ago

@oeiber I can help also as requested

I configured an public endpoint for Alex using nginx. Now he should be able access Doorbird's api. But Alex requested a second camera in the front of Doorbird device to test 2way audio. Unfortunately I don't have a second device. @fergalom May I ask you if you have a second camera, which you can place in front of your Doorbird?

Gis70 commented 2 months ago

Hello, what do you need exactly, i own a doorbird too.

fransiouz commented 1 month ago

I also have a doorbird if I can help. On my side both video and audio receive work well. But not microphone.

fransiouz commented 3 weeks ago

There is a plugin in scrypted.app for doobird which seems to support audio transmit. Can it help ? https://github.com/koush/scrypted/tree/main/plugins/doorbird

luke17287 commented 1 week ago

I also have a doorbird and would love this feature.

fransiouz commented 1 week ago

I switch to something else to do the trick. Doorbird has SIP capability that is working very well : https://www.doorbird.com/fr/sip. I installed asterisk add-on on HA + sip hass card + asterisk integration : https://tech7fox.github.io/sip-hass-docs/ Finally, I use frigate/go2rtc for the video flow and SIP for bidirectionnal audio. Everything is perfect !!

luke17287 commented 1 week ago

Thank you - sounds great. I also tried a bit with SIP but im a noob and was not able to get it running;) .Ill try with your guide.

fransiouz commented 1 week ago

Yes, same for me, but I succeed. Forum/discord + chatgpt help... I can share you some config if needed

luke17287 commented 1 week ago

@fransiouz: I get crazy with asterisk. Iam not able to get my doorbird registered after several hours. I would appreciate if you could share your asterisk custom config files (extensions.conf / pjsip.conf / sip.conf)?

fransiouz commented 1 week ago

Here is my config. The main point is that it's not the doorbird that initiate the call, but your HA user from your phone or tablet. I created a HA user for my tablet where i have HA installed where I can initiate the call with the SIP CARD. In asterisk add-on, I set up a AMI password + auto add extensions. It will create an extension for the tablet user (in my case extension 102). Then you need to copy some conf files : /homeassistant/asterisk/custom/extensions.conf

[default]
exten => _X!,1,Dial(${PJSIP_DIAL_CONTACTS(${EXTEN})})

/homeassistant/asterisk/custom/pjsip_custom.conf

[transport-udp]
type=transport
protocol=udp
bind=0.0.0.0:5060

[transport-wss]
type=transport
protocol=wss
bind=0.0.0.0

;choose an extension number for the doorbird
[6001]
type=endpoint
transport=transport-udp
context=default
disallow=all
allow=ulaw
auth=6001
aors=6001

[6001]
type=auth
auth_type=userpass
password=PASSWORD_TO_SET_IN_DOORBIRD
username=6001

[6001]
type=aor
max_contacts=1
contact=sip:6001@IP_DOORBIRD:5060

/homeassistant/asterisk/custom/confbridge.conf Not sure if necessary, but it's there...

[general]
; Don't delete, required by asterisk

; Privileged HA users that respond to calls
[admin_user]
type=user
marked=yes
admin=yes
music_on_hold_when_empty=false
quiet=yes
hear_own_join_sound=no
send_events=yes

; Unprivileged doorbell user that initiates calls
[default_user]
type=user
send_events=yes
music_on_hold_when_empty=yes
quiet=yes
hear_own_join_sound=no
wait_marked=yes
end_marked=yes
timeout=300 ;5 minute max call time, set to 0 to make it unlimited

; Set up a conference bridge for each doorbell like this:
[doorbird]
type=bridge
max_members=10
enable_events=yes

Then, in doorbird apps, you need to configure and authorize SIP calls: Administration /Favorites/Calls SIP ==> you need to add the HA users that are able to call the doorbird. For my tablet, i have added 102@IP_HOMEASSISTANT

Administration/SIP Parameters. Activate SIP with IP where asterisk is installed (normally HA IP) SIP users = 6001 Password SIP = same pass than in pjsip_custom.conf

Check Authorize calls IN and add your tablet (102@HA_IP) in Activated SIP users. When you save this config in doorbird APPS, you should see in asterisk addon logs sip:6001@IPDOORBIRD is added.

Finally, in HA fronted, I added the sip card to be able to call the doorbird :

- type: custom:sipjs-card
    server: ASTERISK_IP
    port: "443" <== don't remember the default port
    video: false
    button_size: "62"
    prefix: ""
    custom:
      - name: Doorbell
        number: "6001"
        icon: mdi:doorbell
        camera: camera.doorbird
    extensions:
      - person: person.tablet
        name: Tablet
        extension: "102"
        secret: AUTO_ADD_PASSWORD

My whole setup is in https. You may have issues without https to be able to activate mic within the apps. But check the logs in asterisk add-on to see if the call is initiate or not.

Hope it help ! Keep me in touch

luke17287 commented 1 week ago

Thanks for your great support. I finally got my doorbird connected without any issues. But im forcing the SSL error. I think this could connected to your "whole setup in https". How do you generate your SSL Certifiacte for HAAS? I tried to connect to https://HA-IP:8089/ws to temporarily fix it, but still the same issue.

Acutally I use "Generate SSL Certificate" in the addon config.

Asterisk Addon Logs: [Nov 20 20:47:08] ERROR[2383]: iostream.c:663 ast_iostream_start_tls: Problem setting up ssl connection: error:00000001:lib(0)::reason(1), Internal SSL error [Nov 20 20:47:08] ERROR[2383]: tcptls.c:179 handle_tcptls_connection: Unable to set up ssl connection with peer '192.168.178.83:57840' [Nov 20 20:47:08] ERROR[2383]: iostream.c:563 ast_iostream_close: SSL_shutdown() failed: error:00000001:lib(0)::reason(1), Internal SSL error

Further i got an frontend issue. Not sure if this is connected to the ssl issue or a seperat task: image

type: custom:sipjs-card server: port: "8089" video: false button_size: "62" prefix: "" custom: name: Doorbell number: "6001" icon: mdi:doorbell camera: camera.doorbird extensions: '- person: person.me name: me extension: "100" secret: "Auto Add Secret from addon" '- person: person.wife name: wife extension: "101" secret: "Auto Add Secret from addon"

oeiber commented 1 week ago

After investing some time with my colleague, here is our current status regarding backchannel audio with doorbird devices: ffmpeg does not send "Content-Lenght: 99999" header, which is needed to keep the stream open. The following go code streams pipeline inputs to the doorbird device:

package main

import (
    "bufio"
    "flag"
    "fmt"
    "net"
    "os"
    "time"
)

type DoorbirdClient struct {
    host      string
    port      string
    httpUser  string
    httpPass  string
    maxRateKB int
    conn      net.Conn
}

func NewDoorbirdClient(host, port, httpUser, httpPass string, maxRateKB int) *DoorbirdClient {
    return &DoorbirdClient{
        host:      host,
        port:      port,
        httpUser:  httpUser,
        httpPass:  httpPass,
        maxRateKB: maxRateKB,
    }
}

func (d *DoorbirdClient) doConnect() {
    var err error
    d.conn, err = net.Dial("tcp", d.host+":"+d.port)
    if err != nil {
        fmt.Println("Error connecting to target server:", err)
        return
    }
    defer d.conn.Close()

    fmt.Printf("Connected to %s:%s\n", d.host, d.port)

    postHeader := fmt.Sprintf("POST /bha-api/audio-transmit.cgi?http-user=%s&http-password=%s HTTP/1.0\r\n", d.httpUser, d.httpPass)
    headers := []string{
        postHeader,
        "Content-Type: audio/basic",
        "Connection: Keep-Alive",
        "Cache-Control: no-cache",
        "Content-Length: 999999",
        "",
    }
    for _, header := range headers {
        if _, err := d.conn.Write([]byte(header + "\r\n")); err != nil {
            fmt.Println("Error sending header:", err)
            return
        }
    }

    d.receiveAndSendData()
}

func (d *DoorbirdClient) receiveAndSendData() {
    bufferSize := 1024
    maxBytesPerSecond := d.maxRateKB * 1024
    reader := bufio.NewReader(os.Stdin)

    for {
        data := make([]byte, bufferSize)
        n, err := reader.Read(data)
        if err != nil {
            if err.Error() == "EOF" {
                break
            }
            fmt.Println("Error reading data:", err)
            return
        }

        if n == 0 {
            break
        }

        if _, err := d.conn.Write(data[:n]); err != nil {
            fmt.Println("Error sending data:", err)
            return
        }

        timeToWait := float64(n) / float64(maxBytesPerSecond)
        time.Sleep(time.Duration(timeToWait * float64(time.Second)))
    }

    fmt.Println("Finished sending all received data.")
}

func main() {
    host := flag.String("host", "10.10.10.10", "Target host")
    port := flag.String("port", "80", "Target port")
    httpUser := flag.String("user", "user", "HTTP user")
    httpPass := flag.String("pass", "pass", "HTTP password")
    maxRateKB := flag.Int("rate", 8, "Maximum rate in KB/s")

    flag.Parse()

    client := NewDoorbirdClient(*host, *port, *httpUser, *httpPass, *maxRateKB)
    client.doConnect()
}

example: ffmpeg -fflags nobuffer -f alaw -ar 8000 -i test.wav -ac 1 -ar 8000 -f mulaw - | go run main.go -host 123.123.123.123 -port 80 -user user -pass password

As far as i know, tapo devices are working very similar, don't they?

@AlexxIT Maybe this code can be modified and added directly to go2rtc?

AlexxIT commented 1 week ago

This is hard to do because there is no way I can test this code at all

oeiber commented 1 week ago

Ok. I'll give you access to my dorbird device and will bring a notebook with microphone and speaker in front of it.

AlexxIT commented 1 week ago

I think this should work

oeiber commented 1 week ago

perfect! when would it be convenient for you?

AlexxIT commented 1 week ago

It's hard to say. We could try this weekend.

oeiber commented 1 week ago

sounds good! what do you need to access the windows notebook?

AlexxIT commented 1 week ago

Any messenger from my contacts. Just voice call.

AlexxIT commented 5 days ago

Thanks to @oeiber new source added to master version. Will be in next release:

streams:
  video-audio:
    - rtsp://user:pass@123.123.123.123:554/mpeg/media.amp
    - doorbird://user:pass@123.123.123.123?media=audio
  two-way:
    - rtsp://user:pass@123.123.123.123:554/mpeg/media.amp
    - doorbird://user:pass@123.123.123.123?media=audio
    - doorbird://user:pass@123.123.123.123
  mjpeg-video:
    - doorbird://user:pass@123.123.123.123?media=video
oeiber commented 5 days ago

Unfortunately backchannel audio is not working for me, yet:

Configuration:

streams:
  spa01:
    - rtsp://user:password@192.168.178.13:8557/mpeg/720p/media.amp
    - doorbird://user:password@192.168.178.13?media=audio
    - doorbird://user:password@192.168.178.13

Stream info:

{
  "producers": [
    {
      "id": 7998,
      "format_name": "rtsp",
      "protocol": "rtsp+tcp",
      "remote_addr": "192.168.178.13:8557",
      "url": "rtsp://user:password@192.168.178.13:8557/mpeg/720p/media.amp",
      "sdp": "v=0\r\no=- 1731313804359174 1 IN IP4 192.168.178.13\r\ns=RTSP/RTP stream from DoorBird\r\ni=mpeg/720p/media.amp\r\nt=0 0\r\na=tool:LIVE555 Streaming Media v2024.05.15\r\na=type:broadcast\r\na=control:*\r\na=range:npt=now-\r\na=x-qt-text-nam:RTSP/RTP stream from DoorBird\r\na=x-qt-text-inf:mpeg/720p/media.amp\r\nm=video 0 RTP/AVP 96\r\nc=IN IP4 0.0.0.0\r\nb=AS:2\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1;profile-level-id=4D0028;sprop-parameter-sets=Z00AKNoBQBbpUgAAAwDwAAA4QMCAAehIAAiVRe98LwiEag==,aO48gA==\r\na=control:track1\r\n",
      "user_agent": "go2rtc/1.9.7",
      "medias": [
        "video, recvonly, H264"
      ],
      "receivers": [
        {
          "id": 7999,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video",
            "level": 40,
            "profile": "Main"
          },
          "childs": [
            8000
          ],
          "bytes": 7104992,
          "packets": 5426
        }
      ],
      "bytes_recv": 7171280
    },
    {
      "id": 8001,
      "format_name": "pcm",
      "protocol": "http",
      "remote_addr": "192.168.178.13",
      "url": "http://user:password@192.168.178.13/bha-api/audio-receive.cgi?media=audio",
      "medias": [
        "audio, recvonly, PCMU/8000"
      ],
      "receivers": [
        {
          "id": 8005,
          "codec": {
            "codec_name": "pcm_mulaw",
            "codec_type": "audio",
            "sample_rate": 8000
          },
          "childs": [
            8006
          ],
          "bytes": 758784,
          "packets": 741
        }
      ],
      "bytes_recv": 758784
    },
    {
      "id": 25465,
      "format_name": "doorbird",
      "protocol": "http",
      "url": "http://192.168.178.13/bha-api/audio-transmit.cgi?http-user=user\u0026http-password=password",
      "medias": [
        "audio, sendonly, PCMU/8000"
      ],
      "senders": [
        {
          "id": 25466,
          "codec": {
            "codec_name": "pcm_mulaw",
            "codec_type": "audio",
            "sample_rate": 8000
          },
          "parent": 8003
        }
      ]
    }
  ],
  "consumers": [
    {
      "id": 7997,
      "format_name": "webrtc",
      "protocol": "ws+udp",
      "remote_addr": "192.168.178.105:61586 host",
      "user_agent": "Mozilla/5.0 (iPhone; CPU iPhone OS 18_0_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Home Assistant/2024.11 (io.robbie.HomeAssistant; build:2024.971; iOS 18.0.1) Mobile/HomeAssistant, like Safari",
      "medias": [
        "video, sendonly, H264, H265, VP8, VP9",
        "audio, recvonly, OPUS/48000/2, G722/8000, PCMU/8000, PCMA/8000",
        "audio, sendonly, OPUS/48000/2, G722/8000, PCMU/8000, PCMA/8000, L16, PCML"
      ],
      "receivers": [
        {
          "id": 8003,
          "codec": {
            "codec_name": "pcm_mulaw",
            "codec_type": "audio",
            "sample_rate": 8000
          },
          "childs": [
            25466
          ],
          "bytes": 375520,
          "packets": 2347
        }
      ],
      "senders": [
        {
          "id": 8000,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "parent": 7999,
          "bytes": 7104992,
          "packets": 5426
        },
        {
          "id": 8006,
          "codec": {
            "codec_name": "pcm_mulaw",
            "codec_type": "audio",
            "sample_rate": 8000
          },
          "parent": 8005,
          "bytes": 758784,
          "packets": 741
        }
      ],
      "bytes_recv": 424420,
      "bytes_send": 7952220
    }
  ]
}
AlexxIT commented 5 days ago

Maybe you have some errors in logs? The device supports only one connection. I've experienced a number of failures in testing.

oeiber commented 5 days ago

I can see a lot of retries in the logs. My college did also some testing with transfering audio to doorbird devices. He finally ended up with using a tcp socket instead of http. He also meantioned, it would be neccacary to implement some rate control, because of doorbird's weak tcp buffering behaviour:

package main

import (
  "bufio"
  "flag"
  "fmt"
  "net"
  "os"
  "time"
)

type DoorbirdClient struct {
  host      string
  port      string
  httpUser  string
  httpPass  string
  maxRateKB int
  conn      net.Conn
}

func NewDoorbirdClient(host, port, httpUser, httpPass string, maxRateKB int) *DoorbirdClient {
  return &DoorbirdClient{
      host:      host,
      port:      port,
      httpUser:  httpUser,
      httpPass:  httpPass,
      maxRateKB: maxRateKB,
  }
}

func (d *DoorbirdClient) doConnect() {
  var err error
  d.conn, err = net.Dial("tcp", d.host+":"+d.port)
  if err != nil {
      fmt.Println("Error connecting to target server:", err)
      return
  }
  defer d.conn.Close()

  fmt.Printf("Connected to %s:%s\n", d.host, d.port)

  postHeader := fmt.Sprintf("POST /bha-api/audio-transmit.cgi?http-user=%s&http-password=%s HTTP/1.0\r\n", d.httpUser, d.httpPass)
  headers := []string{
      postHeader,
      "Content-Type: audio/basic",
      "Connection: Keep-Alive",
      "Cache-Control: no-cache",
      "Content-Length: 999999",
      "",
  }
  for _, header := range headers {
      if _, err := d.conn.Write([]byte(header + "\r\n")); err != nil {
          fmt.Println("Error sending header:", err)
          return
      }
  }

  d.receiveAndSendData()
}

func (d *DoorbirdClient) receiveAndSendData() {
  bufferSize := 1024
  maxBytesPerSecond := d.maxRateKB * 1024
  reader := bufio.NewReader(os.Stdin)

  for {
      data := make([]byte, bufferSize)
      n, err := reader.Read(data)
      if err != nil {
          if err.Error() == "EOF" {
              break
          }
          fmt.Println("Error reading data:", err)
          return
      }

      if n == 0 {
          break
      }

      if _, err := d.conn.Write(data[:n]); err != nil {
          fmt.Println("Error sending data:", err)
          return
      }

      timeToWait := float64(n) / float64(maxBytesPerSecond)
      time.Sleep(time.Duration(timeToWait * float64(time.Second)))
  }

  fmt.Println("Finished sending all received data.")
}

func main() {
  host := flag.String("host", "10.10.10.10", "Target host")
  port := flag.String("port", "80", "Target port")
  httpUser := flag.String("user", "user", "HTTP user")
  httpPass := flag.String("pass", "pass", "HTTP password")
  maxRateKB := flag.Int("rate", 8, "Maximum rate in KB/s")

  flag.Parse()

  client := NewDoorbirdClient(*host, *port, *httpUser, *httpPass, *maxRateKB)
  client.doConnect()
}

What do you think?

AlexxIT commented 5 days ago
kevp89 commented 4 days ago

Hi Alex. Although i am not firm in go it seems your send method is using the native http go client, which uses HTTP/1.1 instead of HTTP/1.0 as Protocol. Which in Turn does not allow to set a custom content-length size. That's why Oliver and me tried this implementation with a native socket instead of the go http client.

i captured the post request to the doorbird which reflects these findings: Content-Length is set to 0 and i can see a new TCP Connection to the camera for each Chunk that is being sent. This starves the Network Stack of the camera and it soon starts running out of TCP Ports for new connections.

POST /bha-api/audio-transmit.cgi?http-user=USER&http-password=PASSWORD HTTP/1.1 Host: 10.200.13.114 User-Agent: Go-http-client/1.1 Content-Length: 0 Cache-Control: no-cache Connection: Keep-Alive Content-Type: audio/basic

HTTP/1.0 503 Service Not Available Content-Type: text/html Content-Length: 365 Connection: keep-alive Date: Mon, 25 Nov 2024 08:58:59 GMT Server: lighttpd

<?xml version="1.0" encoding="iso-8859-1"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

503 Service Not Available

503 Service Not Available

Best Regards, Kevin

AlexxIT commented 4 days ago

@kevp89 Thanks. I don't use standard http client, but I do use standard http request. Maybe that's where the problem is. I'll look into it.

https://github.com/AlexxIT/go2rtc/blob/d7cdc8b3b07f6fbff7daae2736377de98444b962/pkg/doorbird/backchannel.go#L30-L53