Rsyslog refactor

[xdissent]

Everything stopped working for me when I updated firmware recently: monit got confused and wouldn't start local-control server, log rotation made tailing flakey, and I got inconsistent queries from the apron dbs. I refactored so rsyslog takes care of the app lifecycle and aprontest is used for all queries (seems much more reliable). I also made it npm packable with bundled dependencies, so you can install directly from tarball without worrying about missing node_modules. There's query debouncing and race prevention built in, and setting a value from mqtt doesn't immediately publish the value - it waits for the hub to acknowledge the change. Note that I did get rid of the group topics - not sure if that was being used. Let me know what you think!

[xdissent]

This also fixes #14 for me. Memory is sitting around 70% (wink will reboot at 75%).

[Syco54645]

@xdissent so in the current master branch I am having an issue where all lights come on at full brightness. this may be when the hub is rebooting or it could be at another time, I am not sure when this is happening. Hoping that it is related to #14 though.

[Syco54645]

So testing your code it is working. The lights coming on in my case was in fact caused by the wink hub rebooting, so I need to find a way to maintain state there. Anyway your code appears to be working but it takes about 5-6 seconds for the state to update in HASS. The lights turn off instantly however. I assume there is no fix for this because of the rsyslogd changes.

[xdissent]

I noticed my hub started rebooting a few times a day, sometimes more frequently than others. It turns out the local-control server (another node process) can spike and easily push mem usage over the 75% threshold, causing a reboot. I monitored it for a while and in reality the 70% I quoted was only when everything was absolutely idle. The second anything actually happens, both wink-mqtt and local-control are in danger of triggering a reboot. I tried a few optimizations but was never able to get wink-mqtt memory usage below ~20MB (at idle), which is huge given the limited resources available. And due to node's GC, there's always unpredictable spikes. There are some v8 options for low-mem environments but they're not available until node 4+ (wink is on 0.10).

So with all that in mind, I refactored the whole thing again, this time as a single monolithic shell script, using bish-bosh. If you'd like to try it out, remove this PR:

[root@flex-dvt ~]# monit stop rsyslogd
[root@flex-dvt ~]# rm -rf /opt/wink-mqtt

(don't worry about the rsyslog configs, they'll be overwritten) And then install the bish-bosh branch:

[root@flex-dvt ~]# curl -ksL https://github.com/xdissent/wink-mqtt/releases/download/release_2017.0602.1748-1/wink-mqtt_2017.0602.1748-1_all > /opt/wink-mqtt
[root@flex-dvt ~]# chmod +x /opt/wink-mqtt
[root@flex-dvt ~]# /opt/wink-mqtt install

It should ask you for your mqtt server address and restart rsyslogd. Note that the path isn't important, so you don't really have to remove /opt/wink-mqtt if you wanna keep this PR around too.

With the bish-bosh refactor, memory is around 8MB and total memory usage is around 55%. Way more headroom for local-control spikes. I haven't had a memory-related reboot since I've been running it (about 24 hours). Anyway, let me know how it goes if you give it a shot.

[Syco54645]

I actually just removed the last device from my wink this evening. I was tired of being woke up to all of my lights coming on in the middle of the night when it would crash. However as I go through my house planning where to put smart switches I am finding that I do not have neutrals in all the locations so I will need to use some Lutron Caseta switches. I will get the wink back up tomorrow maybe and see what I find with this new build. Sounds like you really tackled the memory issue. Is it running stable without reboots for the past 3 days?

[Syco54645]

Apparently my entire message was deleted by github having issues. Upon attempting to install I ran into issues

First off the directory /etc/rsyslog.d does not exist.

After creating it I now get the issue /opt/wink-mqtt: line 1: can't create /etc/rsyslog.d/wink-mqtt.conf: nonexistent directory

Which checking it is in fact pointing to a directory that does not exist [root@flex-dvt ~]# ls -al /etc/rsyslog.d/wink-mqtt.conf lrwxrwxrwx 1 root root 29 Jun 2 16:13 /etc/rsyslog.d/wink-mqtt.conf -> /opt/wink-mqtt/wink-mqtt.conf`

It is never creating the config file either. I have not looked at the source to see what is going on exactly. You will probably get it fixed before I find time to do so.