Possibility to talk with doorbell

[moshiko2312]

Hi its possible talking when someone passes on doorbell, with home assistant?

[luzik]

Please take look at https://github.com/rroller/dahua/issues/55

[calisro]

Would be nice to talk with doorbell. I can now send clips though which is nice through the shell_command. IF the gstream would work with hassio, it would be far easier.

shell_command:
  doorbelltts: >-
    /config/ttsplay "{{ states('input_text.tts_door_text') }}"

#!/bin/bash
TEXT="$1"
LOG=ttslog
LOG=/config/ttslog
echo "TEXT: $TEXT" > $LOG

FILE=`curl -v -X POST \
 -H "Authorization: Bearer xxxxxxx" \
 -H "Content-Type: application/json" \
 -d "{\"message\": \"$TEXT\", \"platform\": \"google_cloud\"}" \
 https://ha.xxxxxxx.org:xxxxx/api/tts_get_url | awk -F/ '{print $6}' |awk -F\" '{print $1}'` >> $LOG 2>&1
echo CURLRC=$? >> $LOG

echo
echo FILE:$FILE >> $LOG
echo
ffmpeg -y -i "/config/tts/$FILE" -c:a pcm_alaw -ac 1 -ar 8000 -sample_fmt s16 /config/audio_test.al >> $LOG 2>&1
echo FFMPEGRC=$? >> $LOG
echo

#  --digest
curl -vvv --user 'admin:xxxxxxxx' \
  --limit-rate 8K \
  --expect100-timeout 3 \
  --connect-timeout 5 \ 
  --no-keepalive \
  --max-time 7 \ 
  -F "file=@/config/audio_test.al;type=Audio/G.711A" \
  -H "Content-Type: Audio/G.711A" \
  -H "content-length: 9999999" \
  "http://10.100.x.xx/cgi-bin/audio.cgi?action=postAudio&httptype=singlepart&channel=1" >> $LOG 2>&1
echo DOORRC=$? >> $LOG
exit 0

[NickM-27]

@calisro Nice! This is working well for me in the meantime until a media player is added

[rroller]

Hey all. Sorry I've been super busy. I'll try to get to this soon. I just need to set aside some time :-)

[cabrower]

Been playing with this myself and cannot get the web request to end without an error. Have you been able to get it to exit error free? It seems the "singlepart" type is meant for a continuous stream. I tried to get the "multipart" working but i cannot get the request formed properly.

[NickM-27]

Yeah it works cleanly for me

[calisro]

Been playing with this myself and cannot get the web request to end without an error. Have you been able to get it to exit error free? It seems the "singlepart" type is meant for a continuous stream. I tried to get the "multipart" working but i cannot get the request formed properly.

Since none of this is published in their API, we really don't know how it is expected to work. Remember, this is supposed to be a 'call' to the front door. This is as error free as it gets:

* Server auth using Basic with user 'admin'
> POST /cgi-bin/audio.cgi?action=postAudio&httptype=singlepart&channel=1 HTTP/1.1
> Host: 10.100.1.143
> Authorization: Basic xxxxxxxxxx
> User-Agent: curl/7.79.1
> Accept: */*
> content-length: 9999999
> Content-Type: Audio/G.711A; boundary=------------------------f6342357c58795f8
> 
} [16636 bytes data]
* We are completely uploaded and fine
100 16636    0     0  100 16636      0   1510  0:00:11  0:00:11 --:--:--     0* Operation timed out after 12001 milliseconds with 0 bytes received
100 16636    0     0  100 16636      0   1386  0:00:12  0:00:12 --:--:--     0
* Closing connection 0
curl: (28) Operation timed out after 12001 milliseconds with 0 bytes received
DOORRC=28

Without the timeouts the microphone stays 'open'. You can tell in the app when you try to turn it on again. It says 'error' and then clicking it again causes it to 'work' again.... What we're really missing is the way to 'end the call'. but the timeouts work okay for me.

EDIT: Btw, it works exactly the same in my code with singlepart or multipart.

[mattkerrison]

Ooooohh this is really promising, It's been my biggest gripe with the doorbell, I just want to be able to respond in some way

[strarsis]

So no easy tap-to-talk in Home Assistant yet?

A Dahua doorbell should be at least as usuable by Home Assistant as by using its own vendor app.

[NickM-27]

So no easy tap-to-talk in Home Assistant yet?

A Dahua doorbell should be at least as usuable by Home Assistant as by using its own vendor app.

It's a commonly requested feature in the forums, has not been implemented yet (best is that some integrations support media player)

[strarsis]

@NickM-27: Are there doorbells which two-way-audio just works with HA without any workarounds?

[NickM-27]

@NickM-27: Are there doorbells which two-way-audio just works with HA without any workarounds?

HA does not have any notion of two way talk. The iOS app doesn't even request the microphone permission.

It will need to be built from the ground up (to be like nest, Arlo, ring, etc currently implement it)

[conorlap]

@strarsis @NickM-27 There's being work done to see if this can be implemented on the WebRTC card by @AlexxIT https://github.com/AlexxIT/WebRTC/issues/243

Sounds like even if this is possible, the HA iOS app will need the feature added for microphone permission.

[strarsis]

@NickM-27, @conorlap: Are there apps that are non-vendor and can be used for two-way-audio with Dahua doorbells? Well, one could also use the app in the meantime. The reason for my questions are that I want a doorbell that is not somehow locked for being used from non-vendor apps.

[conorlap]

I can get local two way audio with my Amcrest AD410 on tinycam app on Android with below settings:

It only works locally however. I found this out via Reddit: https://www.reddit.com/r/BlueIris/comments/lpkt8z/achieved_local_doorbell_with_2_way_audio/

[strarsis]

@conorlap: Locally would be fine for me as I use a VPN (like WireGuard) anyway if I want local access from the outside.

That Amcrest AD410 seems to be a nice doorbell. But I need one with PoE/Ethernet.

[mattkerrison]

Even being able to do the TTS like you can with Sonos would be awesome.

But on the HA app not supporting Mic I agree, my use case / idea is using a spare Android phone using Fully Kiosk to host the web version of HA (basically the same as the app) which I believe supports Mic input.

[cabrower]

Even being able to do the TTS like you can with Sonos would be awesome.

But on the HA app not supporting Mic I agree, my use case / idea is using a spare Android phone using Fully Kiosk to host the web version of HA (basically the same as the app) which I believe supports Mic input.

I currently have TTS working with it via NodeRed. Here is my flow:

[{"id":"f44654e214d92aa7","type":"ttsultimate","z":"3d4fa65bb2fbbf54","name":"","voice":"en-US","ssml":false,"sonosipaddress":"192.168.1.158","sonosvolume":"75","sonoshailing":"0","config":"e05dc2be35faa69b","property":"payload","propertyType":{},"rules":[],"playertype":"noplayer","speakingrate":"1","speakingpitch":"0","unmuteIfMuted":true,"x":290,"y":80,"wires":[["16cd8dd95f997800"],[]]},{"id":"35e8516f197e0774","type":"inject","z":"3d4fa65bb2fbbf54","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"\"Please wait while we come to the door.\"","payloadType":"str","x":90,"y":60,"wires":[["f44654e214d92aa7"]]},{"id":"40a2b1600946dee8","type":"function","z":"3d4fa65bb2fbbf54","name":"Parse Filename","func":"msg.filename = String(msg.filesArray[0].file);\nconst ffmpegIn = String(msg.filename);\nconst ffmpegOut = ffmpegIn.split('.').slice(0, -1).join('.');\nconst ffmpeg = \"-y -i \" + ffmpegIn +\" -c:a pcm_alaw -ac 1 -ar 8000 -sample_fmt s16 \" + ffmpegOut + \".wav\";\nmsg.ffmpegIn = ffmpegIn;\nmsg.ffmpegOut = ffmpegOut;\nmsg.ffmpeg = ffmpeg;\nmsg.filename = msg.ffmpegOut+\".wav\";\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":760,"y":80,"wires":[["d8f5be96b6a75f20"]]},{"id":"d8f5be96b6a75f20","type":"exec","z":"3d4fa65bb2fbbf54","command":"/usr/bin/ffmpeg","addpay":"ffmpeg","append":"","useSpawn":"false","timer":"15","winHide":false,"oldrc":false,"name":"Convert Audio","x":960,"y":80,"wires":[["5289ad38ed65ad56"],[],[]]},{"id":"66fac65b9f255c2b","type":"http request","z":"3d4fa65bb2fbbf54","name":"HTTP Request to Doorbell","method":"PUT","ret":"txt","paytoqs":"ignore","url":"http://192.168.1.35/cgi-bin/audio.cgi?action=postAudio&httptype=singlepart&channel=1","tls":"","persist":false,"proxy":"","authType":"basic","senderr":false,"x":840,"y":160,"wires":[[]]},{"id":"bde46b4bf680e208","type":"function","z":"3d4fa65bb2fbbf54","name":"Build http payload","func":"msg.headers = {};\nmsg.headers['Content-Length'] = \"9999999\"\nmsg.headers['Content-Type'] = \"Audio/G.711A\"\nmsg.headers['User-Agent'] = \"NodeRed\"\nmsg.requestTimeout = 5000\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":510,"y":160,"wires":[["66fac65b9f255c2b"]]},{"id":"5289ad38ed65ad56","type":"file in","z":"3d4fa65bb2fbbf54","name":"Read Converted File","filename":"","format":"","chunk":false,"sendError":false,"allProps":false,"x":260,"y":160,"wires":[["bde46b4bf680e208"]]},{"id":"16cd8dd95f997800","type":"switch","z":"3d4fa65bb2fbbf54","name":"Finished Processing","property":"payload","propertyType":"msg","rules":[{"t":"true"}],"checkall":"true","repair":false,"outputs":1,"x":540,"y":80,"wires":[["40a2b1600946dee8"]]},{"id":"e05dc2be35faa69b","type":"ttsultimate-config","name":"TTS Service","noderedipaddress":"192.168.1.14","noderedport":"1980","purgediratrestart":"purge","ttsservice":"googletranslate","TTSRootFolderPath":""}]

[mattkerrison]

Even being able to do the TTS like you can with Sonos would be awesome. But on the HA app not supporting Mic I agree, my use case / idea is using a spare Android phone using Fully Kiosk to host the web version of HA (basically the same as the app) which I believe supports Mic input.

I currently have TTS working with it via NodeRed. Here is my flow:

[{"id":"f44654e214d92aa7","type":"ttsultimate","z":"3d4fa65bb2fbbf54","name":"","voice":"en-US","ssml":false,"sonosipaddress":"192.168.1.158","sonosvolume":"75","sonoshailing":"0","config":"e05dc2be35faa69b","property":"payload","propertyType":{},"rules":[],"playertype":"noplayer","speakingrate":"1","speakingpitch":"0","unmuteIfMuted":true,"x":290,"y":80,"wires":[["16cd8dd95f997800"],[]]},{"id":"35e8516f197e0774","type":"inject","z":"3d4fa65bb2fbbf54","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"\"Please wait while we come to the door.\"","payloadType":"str","x":90,"y":60,"wires":[["f44654e214d92aa7"]]},{"id":"40a2b1600946dee8","type":"function","z":"3d4fa65bb2fbbf54","name":"Parse Filename","func":"msg.filename = String(msg.filesArray[0].file);\nconst ffmpegIn = String(msg.filename);\nconst ffmpegOut = ffmpegIn.split('.').slice(0, -1).join('.');\nconst ffmpeg = \"-y -i \" + ffmpegIn +\" -c:a pcm_alaw -ac 1 -ar 8000 -sample_fmt s16 \" + ffmpegOut + \".wav\";\nmsg.ffmpegIn = ffmpegIn;\nmsg.ffmpegOut = ffmpegOut;\nmsg.ffmpeg = ffmpeg;\nmsg.filename = msg.ffmpegOut+\".wav\";\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":760,"y":80,"wires":[["d8f5be96b6a75f20"]]},{"id":"d8f5be96b6a75f20","type":"exec","z":"3d4fa65bb2fbbf54","command":"/usr/bin/ffmpeg","addpay":"ffmpeg","append":"","useSpawn":"false","timer":"15","winHide":false,"oldrc":false,"name":"Convert Audio","x":960,"y":80,"wires":[["5289ad38ed65ad56"],[],[]]},{"id":"66fac65b9f255c2b","type":"http request","z":"3d4fa65bb2fbbf54","name":"HTTP Request to Doorbell","method":"PUT","ret":"txt","paytoqs":"ignore","url":"http://192.168.1.35/cgi-bin/audio.cgi?action=postAudio&httptype=singlepart&channel=1","tls":"","persist":false,"proxy":"","authType":"basic","senderr":false,"x":840,"y":160,"wires":[[]]},{"id":"bde46b4bf680e208","type":"function","z":"3d4fa65bb2fbbf54","name":"Build http payload","func":"msg.headers = {};\nmsg.headers['Content-Length'] = \"9999999\"\nmsg.headers['Content-Type'] = \"Audio/G.711A\"\nmsg.headers['User-Agent'] = \"NodeRed\"\nmsg.requestTimeout = 5000\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":510,"y":160,"wires":[["66fac65b9f255c2b"]]},{"id":"5289ad38ed65ad56","type":"file in","z":"3d4fa65bb2fbbf54","name":"Read Converted File","filename":"","format":"","chunk":false,"sendError":false,"allProps":false,"x":260,"y":160,"wires":[["bde46b4bf680e208"]]},{"id":"16cd8dd95f997800","type":"switch","z":"3d4fa65bb2fbbf54","name":"Finished Processing","property":"payload","propertyType":"msg","rules":[{"t":"true"}],"checkall":"true","repair":false,"outputs":1,"x":540,"y":80,"wires":[["40a2b1600946dee8"]]},{"id":"e05dc2be35faa69b","type":"ttsultimate-config","name":"TTS Service","noderedipaddress":"192.168.1.14","noderedport":"1980","purgediratrestart":"purge","ttsservice":"googletranslate","TTSRootFolderPath":""}]

I don't often use Node Red so can you help me out as I'm getting an error using your flow:

I'm getting an error message:

"Error: ENOENT: no such file or directory, open '/config/node-red/sonospollyttsstorage/ttsfiles/6181fe2f1bc4977fae86a610bac4b330.wav'"

When I check that directory I see it's an mp3 file not wav:

/config/node-red/sonospollyttsstorage/ttsfiles/6181fe2f1bc4977fae86a610bac4b330.mp3

If I change the two references of .wav in the Parse Filename step to .mp3 I get a burst of white noise come out of the doorbell but not the default "Please wait while we come to the door.", any obvious things I'm missing?

[cabrower]

Do you have ffmpeg installed on your node-red instance?

[mattkerrison]

Do you have ffmpeg installed on your node-red instance?

Haha yep that was it, thanks mate.

Works well, I don't even think I'd use 2 way audio if I could get the ring led to change colour when the reply was sent to just give a visual que as well, but looking at the API it's just on or off

[cabrower]

Thats awesome! Glad it was something simple! I have it setup with an iOS alert that gives me 4 options from my phone plus a live camera feed on my notifications screen:

[strarsis]

Oh, so many cool things can be done with open source automation.

[mattkerrison]

Thats awesome! Glad it was something simple! I have it setup with an iOS alert that gives me 4 options from my phone plus a live camera feed on my notifications screen:

That's exactly what I want to achieve, Nice!

[elkarto]

Do any of you have problems with message length?

I have managed to get it working on an Amcrest AD110.

If the message is very short, everything works fine. But if the message is longer, only the last few seconds (2-3s) are played.

[calisro]

I have complete 2-way now working with the AD410 using the go2rtc project. Might want to look at it.

https://github.com/AlexxIT/go2rtc

[luzik]

I can confirm that 2-way audio is working using go2rtc. https://github.com/AlexxIT/go2rtc/issues/49 The only issue I had to figure out is to switch audio codec, which wasn't possible via dahua web ui. I did use this tool https://github.com/blakeblackshear/frigate/discussions/2572 to switch it.

Maybe there is a way to make audio/video codec/resolution/bitrate/etc as home assistant select entity? @rroller what do You think ?

Now we are waiting for homeassistant card supporting 2-way audio with mute microphone button.

[NickM-27]

@luzik I have it setup but I'm not sure where to use the feature, is it in the go2rtc UI?

[strarsis]

That's great news! I plan to add a doorbell with camera and 2-way-audio to my HA setup, and it would be great if the full hardware can be used.

[luzik]

@luzik I have it setup but I'm not sure where to use the feature, is it in the go2rtc UI?

yes, for now its working only in go2rtc ui

[calisro]

You can use a iframe though for now referencing the api page. THis is needed to actually 'talk' because there are no lovelace cards yet that activate the microphone and manage it.

[NickM-27]

Sorry I should have been more specific, where in the UI is it?

[calisro]

Sorry I should have been more specific, where in the UI is it?

It isn't. You can use existing lovelace cards for the cameras but you won't get talk. That's why I mentioned a iframe. the go2trc is beta still but its in HACS

https://github.com/AlexxIT/go2rtc/

[NickM-27]

@calisro Your suggestion to use an iframe makes me think it should be in the go2rtc UI but I do not see the option. I have already setup go2rtc and I have also added it to my NPM setup so I can access it with https (which I understand is required for microphone access).

[calisro]

Im not sure what you're asking. :) Im sure a lovelace card will eventually be added.

[NickM-27]

The docs say:

PS. Default Home Assistant lovelace cards don't support 2-way audio. You can use 2-way audio from Add-on Web UI. But you need use HTTPS to access the microphone. This is a browser restriction and cannot be avoided.

But it is unclear to me where in the go2rtc UI this is

I'm a developer looking to implement support for this outside of HASS lovelace, just trying to figure out how to interface with go2rtc first.

I will probably just make an issue with go2rtc directly as it is off topic for this.

[calisro]

I would actually try to use it before making an issue. Its pretty simple. The API is just hosted on port 1984.

I will probably just make an issue with go2rtc directly as it is off topic for this.

Yes. :)

[felipecrs]

https://user-images.githubusercontent.com/29582865/191561486-a181827f-44db-4a8a-84de-4c8811c17f09.mp4

@NickM-27 microphone gets automatically enabled once you open the stream from the go2rtc Web UI, and it automatically plays your browser microphone audio in the doorbell.

[NickM-27]

@felipecrs Thank you that is what I was looking for. It asked for microphone permission and I can see the browser is using microphone. Unfortunately it was not playing on the doorbell but that might be due to the configuration of my reverse proxy.

[felipecrs]

Are you using a VTO? If so, check this comment onwards:

• https://github.com/AlexxIT/go2rtc/issues/52#issuecomment-1253651152

[theTaikun]

I'm trying to set this up manually, as I'm using HA Unsupervised and don't have access to addons. These are the issues as I understand it:

• To interact with the microphone, the stream has to be https
• go2rtc doesn't provide https access
• I added my certificate to HA, and can access it via https
• An iframe in a secure home assistant, can't serve the unsecured go2rtc

So is the only solution to run a separate proxy (nginx, apache2, etc...) on the home assistant server, so that https://localhost:port will proxy to http://localhost:port?

[felipecrs]

The ideal would be to use some 2-way audio capable Home Assistant card. I'm sure someone will build one soon.

[morpheus8888]

solved

[morpheus8888]

Thats awesome! Glad it was something simple! I have it setup with an iOS alert that gives me 4 options from my phone plus a live camera feed on my notifications screen:

@mattkerrison

awesome, can you share your flow? thx

[THorst92]

Even being able to do the TTS like you can with Sonos would be awesome. But on the HA app not supporting Mic I agree, my use case / idea is using a spare Android phone using Fully Kiosk to host the web version of HA (basically the same as the app) which I believe supports Mic input.

I currently have TTS working with it via NodeRed. Here is my flow:

[{"id":"f44654e214d92aa7","type":"ttsultimate","z":"3d4fa65bb2fbbf54","name":"","voice":"en-US","ssml":false,"sonosipaddress":"192.168.1.158","sonosvolume":"75","sonoshailing":"0","config":"e05dc2be35faa69b","property":"payload","propertyType":{},"rules":[],"playertype":"noplayer","speakingrate":"1","speakingpitch":"0","unmuteIfMuted":true,"x":290,"y":80,"wires":[["16cd8dd95f997800"],[]]},{"id":"35e8516f197e0774","type":"inject","z":"3d4fa65bb2fbbf54","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"\"Please wait while we come to the door.\"","payloadType":"str","x":90,"y":60,"wires":[["f44654e214d92aa7"]]},{"id":"40a2b1600946dee8","type":"function","z":"3d4fa65bb2fbbf54","name":"Parse Filename","func":"msg.filename = String(msg.filesArray[0].file);\nconst ffmpegIn = String(msg.filename);\nconst ffmpegOut = ffmpegIn.split('.').slice(0, -1).join('.');\nconst ffmpeg = \"-y -i \" + ffmpegIn +\" -c:a pcm_alaw -ac 1 -ar 8000 -sample_fmt s16 \" + ffmpegOut + \".wav\";\nmsg.ffmpegIn = ffmpegIn;\nmsg.ffmpegOut = ffmpegOut;\nmsg.ffmpeg = ffmpeg;\nmsg.filename = msg.ffmpegOut+\".wav\";\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":760,"y":80,"wires":[["d8f5be96b6a75f20"]]},{"id":"d8f5be96b6a75f20","type":"exec","z":"3d4fa65bb2fbbf54","command":"/usr/bin/ffmpeg","addpay":"ffmpeg","append":"","useSpawn":"false","timer":"15","winHide":false,"oldrc":false,"name":"Convert Audio","x":960,"y":80,"wires":[["5289ad38ed65ad56"],[],[]]},{"id":"66fac65b9f255c2b","type":"http request","z":"3d4fa65bb2fbbf54","name":"HTTP Request to Doorbell","method":"PUT","ret":"txt","paytoqs":"ignore","url":"http://192.168.1.35/cgi-bin/audio.cgi?action=postAudio&httptype=singlepart&channel=1","tls":"","persist":false,"proxy":"","authType":"basic","senderr":false,"x":840,"y":160,"wires":[[]]},{"id":"bde46b4bf680e208","type":"function","z":"3d4fa65bb2fbbf54","name":"Build http payload","func":"msg.headers = {};\nmsg.headers['Content-Length'] = \"9999999\"\nmsg.headers['Content-Type'] = \"Audio/G.711A\"\nmsg.headers['User-Agent'] = \"NodeRed\"\nmsg.requestTimeout = 5000\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":510,"y":160,"wires":[["66fac65b9f255c2b"]]},{"id":"5289ad38ed65ad56","type":"file in","z":"3d4fa65bb2fbbf54","name":"Read Converted File","filename":"","format":"","chunk":false,"sendError":false,"allProps":false,"x":260,"y":160,"wires":[["bde46b4bf680e208"]]},{"id":"16cd8dd95f997800","type":"switch","z":"3d4fa65bb2fbbf54","name":"Finished Processing","property":"payload","propertyType":"msg","rules":[{"t":"true"}],"checkall":"true","repair":false,"outputs":1,"x":540,"y":80,"wires":[["40a2b1600946dee8"]]},{"id":"e05dc2be35faa69b","type":"ttsultimate-config","name":"TTS Service","noderedipaddress":"192.168.1.14","noderedport":"1980","purgediratrestart":"purge","ttsservice":"googletranslate","TTSRootFolderPath":""}]

Hi @cabrower,

I have a VTO4202f-p-s2 and copied your flow. I added the credentials to the HTTP-Request node and changed the IP-Adress ( http://xx.xx.xx.xx/cgi-bin/audio.cgi?action=postAudio&httptype=singlepart&channel=1 ). Everything is working fine until this node. I get the following error message: 6 Jan 21:14:28 - [error] [http request:HTTP Request to Doorbell] no response from server

Did you hade similar issues during your development?

Best regards.