search

dresden-elektronik / deconz-serial-protocol

deCONZ Serial Protocol
7 stars 2 forks source link

Native serial UART protocol support for RaspBee and ConBee in Home Assistant without deCONZ software #2

Closed Gamester17 closed 4 years ago

Gamester17 commented 6 years ago

Dresden-Elektronik should consider adding native serial protocol (UART) support for ConBee and RaspBee hardware to open source home automation software such as Home Assistant and its Hass.io (Home Assistant appliance OS for RAspberry Pi).

That is, make open source home automation software such as Home Assistant support using ConBee and RaspBee adapters directly via serial potocol over UART and CLI using existing open source Zigbee libraries without the need to the deCONZ gateway software.

Commercial motivation for Dresden-Elektronik; open source home automation software such as Home Assistant and Hass.io (Home Assistant embedded Linux OS for Raspberry Pi) are the very popular today and as they are already available and used by perhaps millions of people right now, Dresden-Elektronik would most likely sell many more devices if its RaspBee and ConBee hardware had native support them inside open source home automation software such as Home Assistant and Hass.io

Please checkout: https://home-assistant.io

as well as

https://home-assistant.io/hassio/

Home Assistant recently gained native serial protocol (UART) support for ZigBee Home Automation via the ZigPy (Zigbee for Python) open source library that currently can only communicate with XBee and EmberZNet based devices (competing USB adapter by Silicon Labs) via other Python modules which in turn communicates with different radios native serial protocol (UART) for zigpy, for more information see:

https://github.com/zigpy/zigpy

https://github.com/zigpy/bellows https://github.com/zigpy/zigpy-xbee https://github.com/doudz/zigpy-zigate

Update! @damarco and @Equidamoid have both independently started working on zigpy radio libraries a native serial UART protocol for Conbee/Raspbee here:

https://github.com/zigpy/zigpy-deconz

https://github.com/Equidamoid/pyconz/

See also:

https://home-assistant.io/components/zha/

https://github.com/home-assistant/home-assistant/tree/dev/homeassistant/components/zha

https://github.com/home-assistant/home-assistant/pull/6263

https://github.com/home-assistant/home-assistant/pull/12187

https://github.com/home-assistant/home-assistant/pull/19664

https://github.com/home-assistant/home-assistant-polymer/pull/2389

https://github.com/home-assistant/home-assistant-polymer/pull/2421

shayanhaider commented 6 years ago

@manup, can you invite me too for the documentation repository, thanks

Gamester17 commented 6 years ago

@manup maybe time to make the deCONZ Serial Protocol (UART) documentation repository public?

infra-rat commented 6 years ago

@manup could you please add me as well? Thank you :)

neuro159 commented 6 years ago

Could you kindly send me an invitation? Thank you)

marthoc commented 6 years ago

@manup Can you advise where I might be able to get the RaspBee / ConBee hardware in Canada? It seems like there are no distributors actively carrying the products here, which makes it hard to test and develop ;). My primary focus will be developing a deConz plugin for Hass.io but also a stable and easy to maintain Docker build of deConz generally to run alongside Home Assistant and the new integration that @Kane610 is currently developing. Also, can you please add me to the docs repo?

manup commented 6 years ago

Hi marthoc, not sure if our guys already replied to you but short note here: we'll provide you a ConBee and RaspBee for your plugin development. There are already some integration affords for Hass.io ongoing on their forum, might be worth to check that out too :)

Tomorrow is holiday here so response from us may come a day or two later.

marthoc commented 6 years ago

@manup - yes I have heard from Herr Borsdorf and thank you very much for the hardware. I look forward to contributing! I am in touch with the developer of the current Home Assistant integration and will certainly be working with him on testing of both his component and my Docker efforts moving forward. Thanks again and have a good holiday!

Kane610 commented 6 years ago

@manup would it be possible for me to get an extra Conbee? As soon as I get the deCONZ component integrated for release I will be moving my Conbee over to my production environment. It would be great if I could have an extra for active support of the component and possible new features/devices in newer releases of deCONZ from you guys. I'm also considering as a next step to add native support for Conbee hardware to HASS, after I've had a short break from HASS development.

manup commented 6 years ago

I'll see what I can do :)

Kane610 commented 6 years ago

Hey @manup how did it go?

amagnolo commented 6 years ago

I can't help with development at the moment, but I would definitely buy a RaspBee if there was native support in Home Assistant. I appreciate Dresden Elektroniks support, I think it's in their best interest to promote the community efforts.

manup commented 6 years ago

Hey @manup how did it go?

Hi @Kane610 sorry for the late response was a bit busy the last days. Can you please send another request for a second ConBee, to our email and reference me here.

Kane610 commented 6 years ago

@amagnolo as soon as the PR has been approved :)

Kane610 commented 6 years ago

@manup just sent an email, updated this comment for time reference

manup commented 6 years ago

@Kane610 have you send it to support@dresden-elektronik.de?

Kane610 commented 6 years ago

I sent it to info@... Should I send a new one to support?

manup commented 6 years ago

If you haven't received a response yet, yes please send a copy-paste to support mail as well.

Kane610 commented 6 years ago

@manup I got the conbee today, but I also got a raspbee, an unexpected gift. Thank you guys!

manup commented 6 years ago

Yes we thought it might be useful :)

frenck commented 6 years ago

Hi there,

Just responding to this topic. I hear a lot of folks having troubles with integrating your products with Home Assistant. There is already a lot going on and saw some PRs on our main branches... Awesome! That helps.

Home Assistant nowadays also ships an OS called Hass.io. This is a Docker container-based OS allowing people to one-click install other stuff, like Homebridge, Tor, Pi-hole, FTP servers, ... . I'm one of the developers that build and pushes (allot of those) add-ons. (I'm a organization member as well).

I was thinking, there should be an add-on for Hass.io as well, which allows the user to single-click install and run the daemon necessary.

I'm not sure if someone is already on this if not, I'm willing to help with this or, in case it is still open, to develop that part.

Let me know.

Kane610 commented 6 years ago

@frenck @marthoc is working on a docker image but he has run into possible issues with how deconz communicate with the hardware. I think he appreciates all the help he can get. He is talking to @manup about this since it might be necessary to alter the deConz source code

frenck commented 6 years ago

@Kane610, thanks for the response. Good to hear there is already all kind of effort going on.

@marthoc If you need an extra pair of eyes, let me know. Are you on the Discord channels?

@manup I could jump in on this as well. If Dresden is willing to support me on this by lending me some hardware for testing, that would be awesome.

Hedda commented 6 years ago

@frenck that is a good start for Home Assistant but please understand that what you suggest is off-topic for this specific discussion about "native serial UART protocol support" as this discussion here is only about how-to implement RaspBee and ConBee support in Home Assistant without the deCONZ software/daemon, by talking directly to physical Zigbee adapters, which is a solution that could later possibly be extended to become a unified solution for all Zigbee adapters from all manufacturers, (just like how it already is for Z-Wave adapters in Home Assistant), which IMHO would be a better solution for Home Assistant in the long term.

Can I please ask that all keep this specific issue on-topic for the"native serial UART protocol support" dicussion, (however your request here for ConBee/RaspBee adapters is still OK here IMO as that alone is not off-topic). Suggest that you start a new issue if you want to request or suggest other solutions.

frenck commented 6 years ago

@Hedda I do agree, nevertheless, I'm under the impressions Dresden is not willing to opensource that part (nevertheless, reverse engineering serial devices is not that hard). (they are letting some people in, but it still is not out in the open)

Gamester17 commented 6 years ago

@frenck that is not true actually and there is more information about this if you read this whole discussion above. Dresden have already 'opened sourced' their seriel UART API documentation, and anyone can request access to their private repository that hold that information from @manup

https://github.com/dresden-elektronik/deconz-serial-protocol

They have just not made that private repository public because they don't want to spend man-hours resources on support for it for just any amature developers out there. That private repository has 'as-is' documentation without support, meaning anyone skilled enough to figure it all out on their own could with that documentaion put together a stand-alone native solution for serial UART protocol support using ConBee or RaspBee adapters.

After you request the docummentation @manup will then simply be given access to their private repository that so far only contains a link to the documentation serial UART protocol for ConBee and RaspBee adapters (“deCONZ Serial Protocol”).

Note The protocol requires advanced understanding of the ZigBee standard, further it is fairly complicated for historical reasons to be compatible with legacy devices.

Again, this in turn only contain a link to a PDF-documentation on www.dresden-elektronik.de website containing the written documentation for their serial UART protocol. Reason why this documentation is only provided upon request and not posted in public is that they don’t want to have to offer support to non serious developers.

frenck commented 6 years ago

Ok, thanks @Gamester17 for the clarification there. That makes we wonder why @marthoc is working on add-ons...

I'll wait and see.

Kane610 commented 6 years ago

@frenck the same lessons are useful for deconz on docker

marthoc commented 6 years ago

@frenck - the better issue for our discussion is here: https://github.com/dresden-elektronik/deconz-rest-plugin/issues/283. You can read that thread to get an idea of the issues I and others have been running into, and I would be very grateful for any help. I can send you my base (nonworking) Dockerfile if you’d like to take a look.

In short - you can install the “deconz” software and let HASS talk to that; this is what @Kane610 is working on getting PR’d into HASS. I have been working on a Docker image of deconz to make installing alongside HASS easier. I understand it’s also the first step to making a HASSIO addon, so it’s moving in that direction.

Or, you could add support for the underlying Conbee and RaspBee zigbee hardware to the zha component in HASS, and avoid using deconz altogether. That’s what this thread is about.

Gamester17 commented 6 years ago

@Kane610 @marthoc addons are then deCONZ depent addons for HASS.io and not addons that utilize the native serial UART protocol. Both concept ideas would be very nice for Home Assistant as seperate addons, but this specific issue dicusson here is again only about the concept of a addon that uses the available native serial UART protocol, without being dependent on the deCONZ software at all. It is like the difference between directly using Zigbee serial/USB adapter to talk direct to Zigbee lightbulbs instead of talking to the API of a Philips Hue Hub which in turn talks to its Zigbee lightbulbs. deCONZ software is a hub/gateway software which abstacts away direct communication.

marthoc commented 6 years ago

@Gamester17 I agree, I was just explaining the difference to @frenck and for future users coming across this thread, which is why I pointed @frenck to the relevant topic. I won’t be discussing Docker/deconz on this thread any further, sorry for the interruption.

thundergreen commented 6 years ago

https://snillevilla.se/styr-ikea-tradfri-lampor-i-home-assistant-med-conbee/ Dunno if that helps but it is running well like this

Gamester17 commented 6 years ago

@thundergreen That article describe just a way to use deCONZ software, not native serial UART protocol.

geespinoza commented 6 years ago

@manup: Please send me an invitation to the documentation for the serial UART protocol for ConBee and RaspBee adapters (“deCONZ Serial Protocol”). Thank you.

Gamester17 commented 6 years ago

@manup any chance you reconsider Dresden-Elektronik would publish deCONZ Serial Protocol as public?

https://github.com/dresden-elektronik/deconz-serial-protocol

You can simply add a disclaimer stating that this documentation is available 'as-is' without support.

manup commented 6 years ago

I'll put it on discussion when I'm back from traveling, it may take a while since next priorities will be documentation, fixing bugs and existing device support requests.

orktes commented 6 years ago

@manup Could I also get access to the UART doc repo?

corelgott commented 6 years ago

I'd love to have an invite to the serial protocol as well!

Gamester17 commented 6 years ago

@manup I might soon have time to help with development, would it be possible for me to get a ConBee?

jurriaan commented 6 years ago

@manup I'd love to have an invite to the serial protocol as well!

buzzdev commented 6 years ago

@manup Hi Manu, i've just bought RaspBee and ConBee. Could you please invite me to serial protocol documentation? Thanxs

Gamester17 commented 6 years ago

FYI; @rcloran have now also created "zigpy", a library implementing a ZigBee stack in Python:

https://github.com/zigpy/zigpy

It works with separate libraries, like bellows, which interface with radios.

rcloran commented 6 years ago

https://github.com/home-assistant/home-assistant/pull/12187 will bring the zigpy/bellows into hass, too. Once that is in, I have code just about ready to work with Xbee s2c devices which I’ll pull into hass, and will demonstrate how to use the split.

Does anyone have working open source code for the deconz serial protocol? I’d be happy to bring it into the zigpy organization on github.

Framspott commented 6 years ago

@manup Hi, I would be very grateful if I also could have the documentation about the serial interface of the raspbee module. I want to use java to communicate with it, and a not (yet) supported sensor (Schwaiger multi sensor ZHS05). Is there a chance to use the serial functionality of Pi4J for communicating with Raspbee?

All the best - thanks in advance

Framspott

manup commented 6 years ago

Hi @Framspott you've been invited, not sure if the UART-API is what you need — I think the Schwaiger sensor support can be easier added to the REST-API.

I'm not familiar with Pi4J but there are other Java projects which interact with deCONZ REST-API

https://github.com/EsotericSoftware/deconz

https://github.com/fatihboy/jRaspBee

Framspott commented 6 years ago

Hi @manup thanks for the hints, I am already in contact with those projects, too. ;) I will definitely give the serial interface a try.

Can I help you for integrating the multi sensor into the deconz REST API? Or are there any plans to integrate it?

All the best

Framspott

Framspott commented 6 years ago

And of course thanks for the invitation! ;)

aptitude commented 6 years ago

@manup Hello, I've bought a ConBee and I am also very curious about the ConBee serial protocol. An invite would be very nice.

topfs2 commented 6 years ago

@rcloran Thank you for your hard work with bellows and especially with adding support for multiple radios, would make many (me included) incredibly happy if ConBee and RaspBee gets supported.

This is the only implementation I've found, which I think is used by OpenHAB 2 and is using the UART protocol? https://github.com/zsmartsystems/com.zsmartsystems.zigbee/tree/master/com.zsmartsystems.zigbee.dongle.conbee

Framspott commented 6 years ago

I am currently trying to use the RaspBee with another library. Could anyone provide me the following specifications for the RaspBee and a Raspberry Pi 3?

I would be very grateful if anyone could help me ;)

All the best

Framspott

Gamester17 commented 6 years ago

@Framspott not sure if your question is on-topic however pretty sure that basic information is all available in the RaspBee and ConBee user manuals respectively

https://github.com/dresden-elektronik/deconz-rest-plugin#software-requirements

https://www.dresden-elektronik.de/fileadmin/Downloads/Dokumente/Produkte/ZLL/RaspBee-BHB-en.pdf

https://www.dresden-elektronik.de/fileadmin/Downloads/Produkte/12-ConBee/ConBee_User_Manual.pdf

You of course also need to enable and configure UART specifically for Raspberry Pi (3) as follows:

https://elinux.org/RPi_Serial_Connection

That is, the serial port must be configured to allow communication with the RaspBee or ConBee.