search

bwssytems / ha-bridge

Home automation bridge that emulates a Philips Hue light system and can control other systems such as a Vera, Harmony Hub, Nest, MiLight bulbs or any other system that has an http/https/tcp/udp interface. This is a compact impl to run on small format computers. This is impl started from this project https://github.com/armzilla/amazon-echo-ha-bridge.
Apache License 2.0
1.45k stars 198 forks source link

Version 5.3.0 Release Candidate Testing #1100

Closed bwssytems closed 5 years ago

bwssytems commented 5 years ago

The next version is here!

https://github.com/bwssytems/ha-bridge/releases/download/v5.3.0RC13/ha-bridge-5.3.0RC13.jar This version is compiled with jdk 1.8.0_212 on a Raspberry Pi.

A lot of exciting new functionality and bug fixes.

The main screen now allows for locking device ids directly and startup action editing. There are mouse over for instructions.

Please post all testing issues here, thanks!

670 Execute select Device when bridge loads [Enhancement Request] enhancement

877 Color in Hue/Sat enhancement

917 lock device ID #s enhancement

1032 Support for HomeGenie enhancement

1033 Backup & Restore enhancement

1060 Voice command colour change bug question

1077 several somfy action in a same Row ID doesn't work bug question

1084 Add Mozilla IoT gateway Helper enhancement

1092 ha-bridge incompatible with Amazon Echo Dot Gen. 3? enhancement question

1093 Alexa Issue : ID conflict between HA Bridge an Hue Bridge enhancement question

1094 issue with 5.2.2 and Harmony enhancement question

1095 issue with 5.2.2 and fhem enhancement question

1098 Home Assistant has new authentication methods and Legacy API Passwords will be removed enhancement question

1103 Have single scroll bar for Bridge Devices page. enhancement

1108 Add mDNS Discovery to ha-bridge

A couple of changes not in the issues: * Changed the SSDP Response and Notify messages to have the property hue-bridgeid in capital letters - HUE-BRIDGEID

gohamstergo commented 5 years ago

Thanks for continuing to update!

I was getting a "...compiled with newer java version..." error when trying to start ha-bridge, so I updated/upgraded and am now getting:

ha-bridge.service - HA Bridge
   Loaded: loaded (/etc/systemd/system/ha-bridge.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Wed 2019-06-12 17:33:03 CDT; 1min 48s ago
  Process: 517 ExecStart=/usr/bin/java -jar -Dconfig.file=/home/pi/ha-bridge/data/habridge.config /home/pi/ha-bridge/ha-bridge-5.3.0RC1.jar (code=exited, status=1/FAILURE)
 Main PID: 517 (code=exited, status=1/FAILURE)

Jun 12 17:33:03 domoticzpi java[517]:         at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
Jun 12 17:33:03 domoticzpi java[517]:         at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
Jun 12 17:33:03 domoticzpi java[517]:         at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
Jun 12 17:33:03 domoticzpi java[517]:         at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
Jun 12 17:33:03 domoticzpi java[517]:         at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
Jun 12 17:33:03 domoticzpi java[517]:         at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
Jun 12 17:33:03 domoticzpi java[517]:         at java.security.AccessController.doPrivileged(Native Method)
Jun 12 17:33:03 domoticzpi systemd[1]: ha-bridge.service: Main process exited, code=exited, status=1/FAILURE
Jun 12 17:33:03 domoticzpi systemd[1]: ha-bridge.service: Unit entered failed state.
Jun 12 17:33:03 domoticzpi systemd[1]: ha-bridge.service: Failed with result 'exit-code'.

Which version of java is required for 5.3.0?

bwssytems commented 5 years ago

This is compiled with jdk 1.11

gohamstergo commented 5 years ago

Looks like Java 11 is not yet available by default for Raspbian Stretch. Looking into how to upgrade manually. Will report back.

bwssytems commented 5 years ago

The best bet is to download the source and build with Maven with your current jdk

gohamstergo commented 5 years ago

Im not that knowledgeable, unfortunately. I'll just stick with 5.2.2 until Raspbian Buster is released later this year. :) No worries!

bwssytems commented 5 years ago

I'll compile a jdk 8 version tomorrow and put it up.

gohamstergo commented 5 years ago

If you want, that'd be great. Sorry for being stuck on such an old version :( Debian/Raspbian take forever to update things from what I read researching this Java stuff. Really do appreciate your work on all of this.

bwssytems commented 5 years ago

Java executables updated for Java 11 and Java 8

bwssytems commented 5 years ago

Updated to only have the executable for Java 8

gohamstergo commented 5 years ago

Thanks! Working great so far. Have an enhancement request I just thought of. Will create new ticket.

audiofreak9 commented 5 years ago

Working well for me too, nothing to report

ammawel commented 5 years ago

Working well, but I still have error #1044: When I turn off devices, I get the message "Device doesn't support requested value" in the Alexa app (Android).

audiofreak9 commented 5 years ago

I still have error #1044: When I turn off devices, I get the message "Device doesn't support requested value" in the Alexa app (Android).

@bwssytems - My assumption (since do not own a Phillips Hue Bridge) is that the Hue does not go to zero when off, possibly only as low as one? Which leads to the "Device doesn't support requested value" in the Alexa app. Just my 2 cents.

bwssytems commented 5 years ago

I have not done anything in this version to address #1044 yet, so it would still have the issue.

Cardy165 commented 5 years ago

Hi All I have been trying to get my ha-bridge working with little success, I came across something that looks strange whilst trying the last 2 release candidate versions (2 & 3)

I recently split my network into 2 segments (WIFI and LAN) and have been diagnosing the problem with the discovery across them.

I have a proxy for SSDP and I can use SSDP on both a phone and laptop on the WIFI network to discover devices in the LAN network. Whilst diagnosing and ensuring my firewall between the 2 networks was passing the SSDP traffic correctly I was using tcpdump to dump the packet information.

10.1.1.254 - This is my firewall running avahi as a proxy. 10.1.1.245 - This is my host which runs habridge (Currently 5.3 RC3) 10.1.1.140 - This is a host running docker services 10.1.1.6 - This is a machine running a media streaming service plex.

My echo devices are on a different subnet when I request them to do discovery I see the SSDP traffic coming from the firewall. In all I have 3 echo devices 2 Echo plus and an Echo dot all 2nd generation. I also have a physical Philips Hue box. 

    10.1.1.254.58480 > 239.255.255.250.1900: [udp sum ok] UDP, length 417
    10.1.1.254.58480 > 239.255.255.250.1900: [udp sum ok] UDP, length 477
    10.1.1.254.58480 > 239.255.255.250.1900: [udp sum ok] UDP, length 417
    10.1.1.254.58480 > 239.255.255.250.1900: [udp sum ok] UDP, length 481
    10.1.1.254.58480 > 239.255.255.250.1900: [udp sum ok] UDP, length 409
    10.1.1.243.58404 > 239.255.255.250.1900: [udp sum ok] UDP, length 334
    10.1.1.243.58404 > 239.255.255.250.1900: [udp sum ok] UDP, length 264
    10.1.1.243.58404 > 239.255.255.250.1900: [udp sum ok] UDP, length 273
    10.1.1.6.33121 > 239.255.255.250.1900: [udp sum ok] UDP, length 101
    10.1.1.245.1900 > 10.1.1.6.60198: [bad udp cksum 0x7b95 -> 0x2496!] UDP, length 302
    10.1.1.245.1900 > 10.1.1.6.60198: [bad udp cksum 0x7b89 -> 0xa6c7!] UDP, length 290
    10.1.1.140.36575 > 239.255.255.250.1900: [bad udp cksum 0xada7 -> 0x1158!] UDP, length 94
    10.1.1.140.36575 > 239.255.255.250.1900: [bad udp cksum 0xadae -> 0xcb88!] UDP, length 101
    10.1.1.245.1900 > 10.1.1.6.60198: [bad udp cksum 0x7b92 -> 0xf320!] UDP, length 299
    10.1.1.245.1900 > 10.1.1.6.60198: [bad udp cksum 0x7b9b -> 0x354b!] UDP, length 308
    10.1.1.245.1900 > 10.1.1.6.60198: [bad udp cksum 0x7b95 -> 0x2496!] UDP, length 302
    10.1.1.245.1900 > 239.255.255.250.1900: [bad udp cksum 0xaef3 -> 0x2660!] UDP, length 321

The thing I noticed is anything being sent by the ha bridge is being shown as having a bad UDP checksum by tcpdump. I see that .140 also has the same problem so this could just be a quirk but thought it was worth querying.

My problem is no matter what I try ha bridge never responds to searches from the 10.1.1.254 address. This is confirmed by the UPNP Trace logs (see below) which show responses for the other IPs but never for the one ending in 254. The trace of traffic above using tcpdump was run on the host which runs ha bridge so I know the packets from 254 are reaching that machine. I have also checked and ha bridge is listening on 0.0.0.0 on port 1900. 

06-19-2019 00:09:45.540 | INFO | Traceupnp: request of description.xml from: 10.1.1.6:80 filled in with address: 10.1.1.245:80 | com.bwssystems.HABridge.upnp.UpnpSettingsResource
-- | -- | -- | --
06-19-2019 00:09:45.527 | INFO | Traceupnp: send upnp discovery template 1 with response address: 10.1.1.245:80 to address: /10.1.1.6:33121 | com.bwssystems.HABridge.upnp.UpnpListener
06-19-2019 00:09:44.876 | INFO | Traceupnp: send upnp discovery template Original with response address: 10.1.1.245:80 to address: /10.1.1.6:33121 | com.bwssystems.HABridge.upnp.UpnpListener
06-19-2019 00:09:44.225 | INFO | Traceupnp: send upnp discovery template 3 with response address: 10.1.1.245:80 to address: /10.1.1.140:52512 | com.bwssystems.HABridge.upnp.UpnpListener
06-19-2019 00:09:44.225 | INFO | Traceupnp: SSDP M-SEARCH packet from 10.1.1.6:33121 | com.bwssystems.HABridge.upnp.UpnpListener
06-19-2019 00:09:43.574 | INFO | Traceupnp: send upnp discovery template 2 with response address: 10.1.1.245:80 to address: /10.1.1.140:52512 | com.bwssystems.HABridge.upnp.UpnpListener
06-19-2019 00:09:42.924 | INFO | Traceupnp: send upnp discovery template 1 with response address: 10.1.1.245:80 to address: /10.1.1.140:52512 | com.bwssystems.HABridge.upnp.UpnpListener
06-19-2019 00:09:42.273 | INFO | Traceupnp: send upnp discovery template Original with response address: 10.1.1.245:80 to address: /10.1.1.140:52512 | com.bwssystems.HABridge.upnp.UpnpListener
06-19-2019 00:09:41.623 | INFO | Traceupnp: SSDP M-SEARCH packet from 10.1.1.140:52512 | com.bwssystems.HABridge.upnp.UpnpListener

I have been trying to get this working for days but without any luck, as far as I can see the discovery across the LANs is configured correctly and I can certainly see other devices just not the ha bridge.

If anyone has any ideas they would be appreciated because I have tried pretty much everything I can think of.

bwssytems commented 5 years ago

If you do not get a message on Traceupnp like "Traceupnp: SSDP M-SEARCH packet from 10.1.1.6:33121 | com.bwssystems.HABridge.upnp.UpnpListener" that has the IP address of the host you are looking for, it never reached the ha-bridge host process period. What machine did you run the tcpdump from? Is there anything in the log for networking on the ha-bridge host?

<Edit> Also, is there a firewall on the ha-bridge host?

Cardy165 commented 5 years ago

The trace above shows an entry from 10.1.1.6, those are being seen fine and appear in the trace output above as "06-19-2019 00:09:44.225 | INFO | Traceupnp: SSDP M-SEARCH packet from 10.1.1.6:33121 | com.bwssystems.HABridge.upnp.UpnpListener". The trace I don't see but i expect to based on the packets coming in is from the 10.1.1.254 address which send to the multicast address on port 1900.

There is no firewall running on the host running ha-bridge, and the ha-bridge host is the machine I ran the tcpdump from. As can be seen request comes in from 10.1.1.254 to the multicast IP on port 1900 UDP but that never shows up in the trace on ha-bridge.

Cardy165 commented 5 years ago

I can't see any issues in the rest of the system logs. The tracing shows other machines like .6 and .140 doing searches but not from .254. I think I will need to dump the traffic and see if .254 is sending search requests

PeterKoch12 commented 5 years ago

Hi guys, sorry to bug you but I started from the scratch with a brand new Raspi image (Java 8). Looked great at first glance but whenever I enter some smart devices (e.g. my Fibaro or my Harmony) and hit Save. It's not saved bridge restart and log pretends No .. configured.

Unless that just warnings for missing device.db and group.db which is fine in that state. Can you confirm this is due to RC3 or I'm just to dumb. Ah btw. used Chrome and Firefox and put it on port 81

[Unit] Description=HA Bridge Wants=network.target After=network.target

[Service] Type=simple

ExecStart=/usr/bin/java -jar -Dserver.port=81 -Dconfig.file=/home/pi/habridge/data/habridge.config /home/pi/habridge/ha-bridge.jar

[Install] WantedBy=multi-user.target

bwssytems commented 5 years ago

@PeterKoch12 Are there any errors in your log?

PeterKoch12 commented 5 years ago

No errors sorry but I tried with 5.2.2 and it behaves the same

audiofreak9 commented 5 years ago

Are new devices not saving or the Bridge Control settings for Harmony and Fibaro not saving?

If it's just devices, have you tried updating your Device DB Path and File input? I changed mine to /home/pi/habridge/data/device.db and it works fine.

PeterKoch12 commented 5 years ago

It's all about bridge settings but single devices

bwssytems commented 5 years ago

@PeterKoch12 When you add your fibaro and harmony, did you click the add button all the way to the right on those lines before you saved? That has to be done to get it into the config before saving.

Cardy165 commented 5 years ago

Hi I have done some searching with regards to SSDP discovery on multiple LANs. Could you confirm the following I think I know why it does not work but need to check.

The habridge only responds to M-SEARCH requests ? It doesn't use NOTIFY requests to detect new devices ?

I think what is happening and after reading some of the specs is the avahi daemon I have running on my firewall sees the local M-SEARCH request but will not propagate it onto another LAN (as per the specs).

It does however broadcast NOTIFY messages to 239.255.255.250 on 1900 UDP to inform devices on the LAN of new services which are available. After dumping the traffic using tcpdump and wireshark this is what I am seeing coming from the firewall on the LAN network where ha-bridge is running.

I suspect that ha-bridge may not use the notify messages so does not learn about the new services which are available via these notifications and it only responds when it sees a M-SEARCH request coming in.

Could you let me know if my understanding for the M-SEARCH & NOTIFY messages is correct within ha-bridge ?

Thanks

PeterKoch12 commented 5 years ago

@PeterKoch12 When you add your fibaro and harmony, did you click the add button all the way to the right on those lines before you saved? That has to be done to get it into the config before saving.

great hint, yes right you have to click Add till you save. Devices are here now and I'm going to run a test for Fibaro and come back if necessary. Many thanks for your help!!!!!!!

bwssytems commented 5 years ago

@Cardy165 since you have some an unusual network setup, I would encourage you to look at the upnp spec itself. The HA-bridge sends NOTIFY messages for it's advertisement, but it is up to the client to counter on that message. Also, review the code in the ha-bridge upnp handling. It is modeled after the real hue bridge.

Cardy165 commented 5 years ago

@bwssytems I did have a look through the spec and I have done some more investigations. I apologise if I am getting things wrong here I've not explored the SSDP protocol before so am learning as I go. Sadly I don't know java so looking at the code probably wouldn't enlighten me very much.

I did have a look at the NOTIFY packets and your correct that ha-bridge does sent them out,

I have done a trace of the same from my philips hue bridge and it seems to send out the same packet as ha-bridge sends but it also sends other notify packets with different contents specifically rootdevice and schemas.

I have attached a sample of these outputs.

philips-hue-habridge-ssdp-to-multicast.txt

I'm wondering if these could be why I can discovery my philips hue bridge but I can't see the ha-bridge.

bwssytems commented 5 years ago

I can add those notifies in to the notification process

Cardy165 commented 5 years ago

I'm happy to test if you can, it would be great to getting this working across multiple networks too.

bwssytems commented 5 years ago

@Cardy165 Test with the new RC5 release

tuicemen commented 5 years ago

Sorry I've been slow to testing this, I haven't been able to play much this time of year. the Homegenie helper has a issue when clicking the add button to set your HomeGenie info. it adds the info without the admin (name)setting so you have to retype it after clicking add. Also it isn't seeing all x10 devices or their names and adds more devices (not just X10 devices )then I actually have configured. Tested this on 3 separate RasPis and all show same results the x10 devices that don't get seen is the same (unit code 3 house code A & units 1,2,3 house code M). I only use 2 X10 house codes with HG so this may expand to other house codes which I'll attempt to test. Device names do populate correctly for the few Wemo, and Zwave devices I have configured.

bwssytems commented 5 years ago

@tuicemen I will need you to run this command in your browser to see what HomeGenie gives you for output.

http://<IP Adress of HomeGenie>/api/HomeAutomation.HomeGenie/Config/Modules.List

tuicemen commented 5 years ago

OK, did that and it does report seeing A3 M1,M2. & M3and reports the correct names For Addresses I don't use it reports them but shows "" beside the Name value.

bwssytems commented 5 years ago

Could you post the dump for me?

tuicemen commented 5 years ago

I'd like to but it contains some personal info I'll need to edit first. For devices I don't use like M5 the dump for it looks like this.

{
  "Name": "",
  "Description": "X10 Module",
  "DeviceType": "Switch",
  "Domain": "HomeAutomation.X10",
  "Address": "M5",
  "Properties": [],
  "RoutingNode": ""
},
tuicemen commented 5 years ago

Here is what is reported for M2 which I use but is not picked up by HA-Bridge neither name nor device Note: all the options for properties and these could be more.

{
  "Name": "Back Deck Floods",
  "Description": "X10 Module",
  "DeviceType": "Light",
  "Domain": "HomeAutomation.X10",
  "Address": "M2",
  "Properties": [
    {
      "Name": "EnergyManagement.MonitorStatus",
      "Value": "",
      "Description": "",
      "FieldType": "",
      "UpdateTime": "2018-09-06T11:29:41.479663Z",
      "NeedsUpdate": false
    },
    {
      "Name": "EnergyManagement.MonitorWatts",
      "Value": "",
      "Description": "",
      "FieldType": "",
      "UpdateTime": "2018-09-06T11:29:41.521466Z",
      "NeedsUpdate": false
    },
    {
      "Name": "HomeGenie.SecurityAlarm",
      "Value": "",
      "Description": "",
      "FieldType": "",
      "UpdateTime": "2018-09-06T11:29:41.542567Z",
      "NeedsUpdate": false
    },
    {
      "Name": "HomeGenie.ZoneSensors.1",
      "Value": "",
      "Description": "",
      "FieldType": "",
      "UpdateTime": "2018-09-06T11:29:41.570589Z",
      "NeedsUpdate": false
    },
    {
      "Name": "HomeGenie.ZoneSensors.2",
      "Value": "",
      "Description": "",
      "FieldType": "",
      "UpdateTime": "2018-09-06T11:29:41.590838Z",
      "NeedsUpdate": false
    },
    {
      "Name": "HomeGenie.ZoneSensors.3",
      "Value": "",
      "Description": "",
      "FieldType": "",
      "UpdateTime": "2018-09-06T11:29:41.616162Z",
      "NeedsUpdate": false
    },
    {
      "Name": "HomeGenie.ZoneSensors.4",
      "Value": "",
      "Description": "",
      "FieldType": "",
      "UpdateTime": "2018-09-06T11:29:41.618913Z",
      "NeedsUpdate": false
    },
    {
      "Name": "HomeGenie.ZoneSensors.5",
      "Value": "",
      "Description": "",
      "FieldType": "",
      "UpdateTime": "2018-09-06T11:29:41.639402Z",
      "NeedsUpdate": false
    },
    {
      "Name": "HomeGenie.ZoneSensors.Enable",
      "Value": "",
      "Description": "",
      "FieldType": "",
      "UpdateTime": "2018-09-06T11:29:41.564251Z",
      "NeedsUpdate": false
    },
    {
      "Name": "Module.GUID",
      "Value": "bXXXXXXXXXXXXXX",
      "Description": "",
      "FieldType": "",
      "UpdateTime": "2019-06-25T12:48:55.635457Z",
      "NeedsUpdate": false
    },
    {
      "Name": "Status.Level",
      "Value": "0",
      "Description": "",
      "FieldType": "",
      "UpdateTime": "2019-06-25T12:48:55.64147Z",
      "NeedsUpdate": false
    },
    {
      "Name": "VirtualMeter.Watts",
      "Value": "0",
      "Description": "",
      "FieldType": "",
      "UpdateTime": "2019-06-25T12:48:55.648515Z",
      "NeedsUpdate": false
    }
  ],
  "RoutingNode": ""
},
bwssytems commented 5 years ago

@tuicemen So, I am filtering on types such as "Switch", "Dimmer" but not "Light". How are you setting those device types?

tuicemen commented 5 years ago

OK, that seems to be the issue for no names appearing however if the device isn't configured Homegenie is reporting it a switch by default thus all non configured devices even those not X10 are seen.

tuicemen commented 5 years ago

@bwssytems, it may be worth adding lights to the filtering, but also (or) add only if devices have a name assigned in HomeGenie.

gohamstergo commented 5 years ago

Small thing: not sure what version this started with, but looks like the /data/ folder got changed to /habridge/ instead of /ha-bridge/.

A few versions ago, the dash was added, but looks like it got removed again more recently.

edit: habridge.config looks to be the only file that has had the dash removed from its path. the last update i see to my habridge.config in the dashed folder is 7/14/2018. device.db is still in the dashed folder.

bwssytems commented 5 years ago

There is no path except the relative one for the data files in ha-bridge. What ever folder it is started in the data folder is put in that directory. If the command line args for the data files are used, it uses those. If you are using Docker you will need to check what files it uses as I did not create that. Also, if you typed in the path to the data file on the bridge control page, it will use that.

bwssytems commented 5 years ago

@tuicemen Try the new version with HomeGenie

tuicemen commented 5 years ago

@bwssytems thanks, it may be a few days before I can play again but will let you know once installed.

Cardy165 commented 5 years ago

I tested the new changes for the notifications but they have not made the ha-bridge discover-able when on a different LAN to the habridge software.

My physical philips hue device remains discover-able.

I have a limited amount of time for the next 2 weeks but I will try to investigate more what is being sent by habridge vs philips hue to try to identify what allows the hue box to be discover-able.

audiofreak9 commented 5 years ago

Since RC7 my Smartthings hub can not see HA-Bridge, not sure if RC6 had the same issue - I updated but didn't check smartthings with RC6.

bwssytems commented 5 years ago

@audiofreak9 Traceupnp is our friend....

audiofreak9 commented 5 years ago

Can't update log levels? I choose TRACE (or any level) from dropdown for upnp (or any component) click Update Log Levels, but setting doesn't change? What am I missing?

audiofreak9 commented 5 years ago

Keep getting "Updated 0 loggers for log level" feedback

bwssytems commented 5 years ago

That is weird as I set them and they are fine. Refresh your browser, maybe there is a cache issue.