search

JurajNyiri / HomeAssistant-Tapo-Control

Control for Tapo cameras as a Home Assistant component
Apache License 2.0
967 stars 81 forks source link

Feature Request: Tapo D230S1 integration #397

Open ah92uk opened 10 months ago

ah92uk commented 10 months ago

Is your feature request related to a problem? Please describe.

Currently there is limited native support for the D230S1, I can configure settings and options on the device but there's no video stream and some options such as day/ night mode switching have glitches (such as switching to night mode enabling the spotlight).

Describe the solution you'd like

Integration of the D230S1 would be a HA gamechanger I think and enable a high-featured doorbell to be LAN only and fully integrated with HA.

If you could please consider this, that would be great!

Describe alternatives you've considered

The D230S1 is already visible via the Hub but lacking video stream features etc.

Additional context

None.

JurajNyiri commented 10 months ago

Does this doorbell work via a hub? How does it ring the doorbell? Is it battery powered? Can you explain as much as possible about it? Could you test and show us exactly how each feature works that is already supported by HA and in case it doesnt work properly how it doesnt work (can you test if the same happens via the app?) Could you screenshot everything from the app to show us how it is different from the other cameras? What exactly is lacking that is possible via the app? (List everything)

ah92uk commented 10 months ago

The doorbell works via the Hub - it's one of TP Link's "sub-G" devices - https://uk.store.tapo.com/products/tapo-d230s1-smart-battery-video-doorbell - it's battery powered and seems to just be a doorbell version of the other cameras in this line up (e.g. C420 etc.).

The doorbell has a pushbutton on the front which, when pushed, rings a sounder in the doorbell itself (adjustable on/ off, sound level), the Tapo Hub (adjustable on/ off, sound type, sound level) and integrates with other devices (e.g. Alexa via the Tapo app) - however I can't find any entity that reflects this in the Home Assistant integration whether that be a switch, sensor or otherwise.

Screenshots of the Tapo app and Home Assistant integration can be found here: https://imgur.com/a/C1uHNev

I've included a table below which compares the Tapo app functions (with descriptions) and the Home Assistant integrations and vice versa, please note in the entities below, "front_door" is the name of my doorbell, so I assume this takes the name of the Tapo D230S1 device and converts it into part of the entity name.

Hope this helps - and thanks for looking into this! Please let me know if there's anything you want more information of or testing/ trials of.

<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns="http://www.w3.org/TR/REC-html40">

Tapo App | Function Description | Options Available | Home Assistant Integration | Remarks -- | -- | -- | -- | -- AI Detection | Allows the user to configure on-board AI detection settings | Person detection, pet detection, vehicle detection, package detection | select.front_door_pet_detection select.front_door_person_detection select.front_door_vehicle_detection No function for package detection. |   Anti-Theft Alarm | Allows the user to enable or disable the doorbell anti-theft alarm, if enabled, the doorbell will sound an alarm if removed from the mounting bracket. | On/ Off | No function |   Battery Status | Allows the user to view the battery status | View battery state (percentage), enable power saving mode | sensor.front_door_battery |   Clip Settings | Allows the user to configure the length and trigger configuration for the doorbell video recording | Retrigger time - 0 to 60 seconds, Record Buffer - 0 to 10 seconds, Maximum Clip Length 20 to 120 seconds | No function |   Connected Hub Name | Displays the name of the Hub the doorbell is connected to. | "Name" | No function |   Connected Hub Signal Quality | Displays the signal strength between the doorbell and the Hub. | Great, good, bad | No function |   Doorbell Push | Rings a sounder in the doorbell itself (adjustable on/ off, sound level), the Tapo Hub (adjustable on/ off, sound type, sound level) and integrates with other devices (e.g. Alexa via the Tapo app) | None | No function |   Doorbell Ring | Allows the user to configure if the doorbell itself rings when the doorbell button is pushed, at what volume and if the Tapo Hub rings when the doorbell is pressed | On/ Off, Sound Settings - Low to High, Ring Connected Hub On/ Off | No function |   Doorbell Video Call | Allows the user to enable or disable the doorbell video call notification pop-up (if enabled, a doorbell press will appear as an incoming call does on the user's device, if disabled, it will just be a notification). | On/ Off | No function | I think this only works with a Tapo Cloud subscription. Firmware Update | Allows the user to check current camera firmware and upgrade if an updated firmware is available |   | update.front_door_update |   Light Frequency | Allows the user to select the light frequency (anti-flicker) | Auto, 50Hz, 60Hz | select.front_door_light_frequency |   MicroSD Recording | Allows the user to turn on/off SD card recording to the Tapo Hub | On/ Off | No function |   Motion Detection | Allows the user to turn on/ off motion detection and adjust configuration | On/ Off, sensitivity adjustable 1 - 10, activity zones/ masking to ignore motion in certain parts of the image | select.front_door_motion_detection - high, normal, low, off number.front_door_motion_detection - sensitivity 0 to 100 |   Night Mode Selection | Allows the user to select day/ night mode switch settings | Auto - uses light sensor to switch between day/ night; On - night mode on; Off- day mode on | select.front_door_night_vision |   Night Mode Settings | Allows the user to select various night mode options which are enabled when the doorbell is switched to night mode automatically (via light sensor) or via user selection | Infrared - black and white image, IR illumination on; Full Colour Mode - colour image, white light illumination (spotlight) on; Doorbell Mode, IR illumination on except when doorbell is pressed when the doorbell will switch to a colour image with white light illumination (spotlight) on; IR Power selector (Near, Medium, Far) | No function. | select.front_door_night_vision only permits on/ off/ auto and reverts to "full colour mode" if this is used, no matter what the setting in the Tapo app (infrared, full colour, doorbell) Playback & Memory | Allows the user to view recordings from the doorbell stored on the Hub | Timeline of recordings, play recordings, save to device, filter by event (e.g. doorbell push, motion detection, AI detection) | No function (video synchronisation does not work it appears) |   Privacy Mode | Allows the user to use privacy mode (software disable of the doorbell) | On-screen pushbutton to enable/ disable privacy mode | switch.front_door_privacy |   Privacy Zones | Allows the user to set privacy masking on the doorbell view | User interface to permit the user to create and shape privacy masks/ zones on the camera view | No function |   Quick Response | Allows the user to select and play a text-to-speech message. | Selection of various text-to-speech messages or field for custom message, tap to play | No function |   Recording Schedule | Allows the user to configure a recording schedule for the doorbell video recording | Monday - Sunday, 00.00 to 00.00 - Detection Recording On/ Off | No function |   Spotlight Brightness | Allows the user to adjust the spotlight brightness | 1 to 5 | select.front_door_spotlight_intensity |   Spotlight On/ Off | Allows the user to turn the spotlight on or off | On/ Off | light.front_door_floodlight_timed |   Status LED | Allows the user to enable the status LED on the front of the doorbell | The status LED can be turned on or off via an on-screen slider selector | switch.front_door_indicator_led |   Talk | Allows the user to use the two-way communication  function (push to talk) | The speaker is always on, microphone is enabled by on-screen push to talk pushbutton | No function |   Tapo Care | Allows the user to configure the Tapo Cloud storage/ notifications | Opens Tapo Care/ Cloud configuration | No function |   Video Stream | Allows the user to view live video from the doorbell | Tap to view video | No function - live video is not available via the Home Assistant integration |   Video Quality | Allows the user to configure the video quality of the doorbell video. | 5MP, 960p, Auto | No function |   Voice Call | Allows the user to use the two-way communication function (microphone and speaker on) | The speaker can be turned on or off, the microphone can be turned on or off | No function |   Wake Up Settings | Allows the user to configure when and how the device wakes up from a quiescent state | Wake up when event detected (via PIR, with sensitivity adjustable 1 - 10), Wake up when doorbell pressed | No function but may be linked to select.front_door_motion_detection also |   No function | Allows the user to trigger an alarm function which puts spotlight in strobe mode and sounds a siren (UNTESTED) | Off, Light, Sound, Both | select.front_door_automatic alarm | This appears to be a manually triggerable anti-theft alarm No function | Allows the user to configure the period of time that elapses after activating the spotlight, which the spotlight then turns off. | 5 min, 10 min, 15 min, 30 min | select.front_door_spotlight_on_off_for | Appears to be an automatic shut-off function for the spotlight No function | Unknown | Sensor | sensor.front_door_recordings_synchronisation | Shows as "idle" and always shows as "idle". No function | Allows the user to trigger an alarm function which sounds a siren (UNTESTED) | On/ Off | siren.front_door_siren | This appears to be a manually triggerable siren No function | Allows the user to flip the image | On/ Off | switch.front_door_flip |   No function | Unknown | On/ Off | switch.front_door_lens_distortion_correction | This appears to be fisheye distortion correction but I can't see a difference to the image whether it's on or off.

daRKhomeSK commented 10 months ago

I second this problem. I bought it yesterday and I am disappointed that there is no support for DS230 yet. I hope someone will solve it soon. Thank you.

JurajNyiri commented 10 months ago

Thank you for excellent documentation @ah92uk .

I do not own this doorbell so it is not possible to find the missing function calls in your table. We will need someone to find the proper calls how to get / set the information and then we can implement full support for it.

trevkillick88 commented 10 months ago

@JurajNyiri

I've got this doorbell, is there any information/guidance you could give so those of us with the camera can get the full details for you to add support for this in?

Thanks

JurajNyiri commented 10 months ago

You will need to inspect traffic between the doorbell and your phone - see the tcpip calls and payloads when executing actions within the app. There are many ways how to do this and it's not trivial. If you want to try you can Google things like Burp suite, mitm inspect traffic and Frida(for android) there are quite a lot of tutorials on the web. Thank you in advance!

trevkillick88 commented 10 months ago

@JurajNyiri

Thanks appreciate the reply, will have a read up and see if I can get any additional information.

👍

phil-schneider commented 9 months ago

I have the same cam/door bell. I would like to help to get this working?

How can I help? What Information is needed? What steps to take?

chooseYourUserName commented 9 months ago

@ah92uk i'm also looking to get this doorbell too, was about to buy it when i saw your post. you mentioned no video signal.

can i check if you have tried to see if you can view the video stream from a PC. from what i've looked up the camera should support RTSP. https://www.tp-link.com/us/support/faq/2680/ link from TP link.
you can use VLC to view the steam. with the video stream up, at least it works like a camera.

phil-schneider commented 9 months ago

@chooseYourUserName The door bell is battery based and connected to a hub. The bell, itself, has no IP Adress, only the hub has a visible ip from the local Network.

Christiaan-sd commented 6 months ago

I found that the hub software is actually open source, but was not able to find any meaningful starting points.

Anyway, I own a Doorbell as well and happy to help where possible. I found that Chromecast also support a video feed from the doorbell, is it an idea to sniff that in the network?

smholsen commented 6 months ago

Hi! @Christiaan-sd Do you have a link to that open source code?

I also have a doorbell of this type, and I tested a bit with intercepting the traffic, but I am not really able to make sense of it. When opening the stream theres a few endpoints that are being hit:

But for some reason my proxy is unable to decipher these, complains about ssl. I haven't really dug deep into this (just using an app directly on my phone, not going via burpsuite server on pc). So I don't really know the utility of these calls.

However, I can see a stream of calls going to the IP of the hub, which I can decipher. But the contents are not straight-forward. The requests uses transfer-encoding: chunked, and are formatted as follows:

URL: https://<ip>/stok/<some-uuid>/ds
Method: POST
Headers:
 - tapo_tag: long string
 - seq: <sequence number>
 - content-type: application/json
 - .... more, but not special

The responses are jsons that include a base64 encoded string, which I assume are the video stream chunked into smaller data parts. It sends and receives about ~10 of these per second. I tried decoding the base64 string to try to see if I could re-build the files, but I was unable to. I assume there might be additional encryption here. @JurajNyiri perhaps you know something about this from your other work with tapo cameras?

Edit: Looks like someone has been able to view stream via the hub here: https://github.com/JurajNyiri/pytapo/issues/57#issue-1737685488

Edit2: Ahh, so looks like theres an initial handshake I missed last round. First, theres a POST request to the IP only, with the following body:

{
  "method": "login",
  "params": {
    "cnonce": "<nonce>",
    "username": "admin",
    "encrypt_type": "<integer>"
  }
}

Response:

{
  "error_code": -<int>,
  "result": {
    "data": {
      "code": -<int>,
      "encrypt_type": [
        "<int>"
      ],
      "key": "<key>",
      "nonce": "<nonce>",
      "device_confirm": "<string>"
    }
  }
}

It looks like this response is an error of some sort, which leads to a new login request (which uses a cnonce which is part of the device_confirm from the above response, and also includes a digest_passwd

{
  "method": "login",
  "params": {
    "username": "admin",
    "digest_passwd": "<string>",
    "cnonce": "<substring of device_confirm>",
    "encrypt_type": "<int>"
  }
}

Landing on a successful response:

{
  "error_code": 0,
  "result": {
    "stok": "<string>",
    "user_group": "root",
    "start_seq": <int>
  }
}

So this does the login, but I also assume this sets up some sort of encryption of the sessions payloads. The stok var is what is used in the subsequent chunked video requests (The encrypt type), which may explain why I was unable to get an actual file from the other payload.

Christiaan-sd commented 6 months ago

@smholsen
Great work! I hope I find some time in the future to dive into it again as well.

You can just download it from the support page: https://www.tapo.com/en/product/smart-hub/tapo-h200/#tapo-product-download

JurajNyiri commented 6 months ago

@smholsen

This is a common encryption Tapo cameras and devices use nowadays, it was first introduced in around October 2023 and was added into pytapo in 3.3.0 (see PR) and this integration in 5.4.0. There has been various fixes since then introduced to both pytapo and this integration, but that PR gives you the main information.

JurajNyiri commented 6 months ago

For ssl issue in burp - the app uses certificate pinning. You will need to use Frida on rooted android phone to pin a burp certificate and then the app will be able to communicate and burp will be able to decrypt all.

JurajNyiri commented 6 months ago

As for the videos, pytapo supports that as well (although for recordings) but it is the same for live stream as well. The URL just needs to be changed. We will need to find the URL directly on hub though as I do not want this integration to communicate with cloud.

smholsen commented 6 months ago

As for the videos, pytapo supports that as well (although for recordings) but it is the same for live stream as well. The URL just needs to be changed. We will need to find the URL directly on hub though as I do not want this integration to communicate with cloud.

Thats pretty awesome. Which URL are you referrencing here? This one https://<ip>/stok/<some-uuid>/ds?

Also I am curious what you think the possibilities are for having this stream only run when actually requested from a Home Assistant dashboard or stream view? Since this is a battery powered device, it would be unfortunate for the integration to request constant streaming of the device. (e.g. start a stream and never stop it). But from what I can see it looks to me like as long as you just stop sending the chunked requests it will stop recording.

bifi87 commented 3 months ago

Tapo D230S1, I can enter the IP of the hub200 via the tapo integration of hacs and confirm with a password. After i can see two devices and 25 entitys in the tapo integration. A message appears saying that the camera stream needs to be restarted. I have tried this several times without success... restart HA, Hub and doorbell... can someone help me please?

bifi87 commented 3 months ago

I forgot to say that my google nest hub 2 enables a live view of the tapo doorbell d230s1 via google assistant

edith101 commented 3 months ago

Similar issue for D130. I couldn't find doorbell press binary in integration.

jvcsw commented 2 months ago

I'm confused by the comments, this doorbell can't show the live video in HA then?

JurajNyiri commented 2 months ago

No

Alexius30 commented 1 month ago

Hallo, Is there any update for this problem? I tried a few hours to get the stream into my HA, but I didn't find any solution :/

JurajNyiri commented 1 month ago

No there isn't. I do not own this so we will have to wait for somebody who does to implement a solution.

See https://github.com/JurajNyiri/HomeAssistant-Tapo-Control/issues/397#issuecomment-1890419970