gulliverrr
commented
4 years ago Wow.... That is very well explained! About your last 3 points...
- If you pass me the code (I believe a PR would be fine) I can handle the creation of the new image. Could you have a look on the script that clears anything that needs to be cleared before the image is created and add maybe more new entries needed?
- I will amend the upgrade script. Right now it only copies the files off GitHub without stopping it and reboots. I'm not sure if I can have a script keep running after OH is killed. I'll find a workaround.
- I can make the load checking as advanced as you describe it, no problem. Need to make it work for people running it for testing/development on a Pi 3 or anything faster than a Zero (which practically means any other Pi)
rkoshak
I kill MQTT Explorer quickly otherwise laptop goes unresponsive pretty bad.
I think I've made it to the point now where I can check in my second rewrite of the rules and other configs. At a high level here is what I've done:
Moved all the Things from .things files to JSONDB.
Created some new Exec Things to replace executeCommandLine calls in the Rules. Startup is a little faster when we let the Exec binding handle these calls instead of Rules.
Moved all the Items from .items files except for the five with GPIO binding configs. This improves startup by greatly reducing how many lines of .items files need to be read and parsed. This also allows users to see and more easily change/customize Items (e.g. translating to their native languages) through PaperUI instead of needing to ssh and modify files.
Moved all the Rules to JSONDB written in JavaScript. I chose JavaScript over Python as I've noticed on my own machines that Python, like Rules DSL, takes a good deal of time to load and parse. JavaScript loads very fast but there is a slight delay (one or two seconds) the first time a Rule executes. Because the JSONDB rules work differently in many ways, there were some changes to the structure of the Rules. But I also was able to add some additional error checking (e.g. if you change the FanMode to OFF while Heating or Cooling is ON it will prevent it and restore the FanMode to Auto, it will not accept SystemType or HumidityType or TempUnit values that are not valid, restoring them to their previous value, etc.).
Removed the Language Server Provider. It's not used, never going to be used and it takes up both memory and CPU during boot time.
Changed the kiosk script to wait for the system load to drop to 2.0 instead of 0.5. I found it was spending multiple minutes to drop to 0.5 and sometimes it never got that low because all the rules will have loaded and started running which wasn't the case before.
There is still a default.rules file which gets copied to the rules folder when the load drops. This is done for two reasons. 1. the JSONDB rules do not yet support a System started trigger and 2. we don't want to run the initialization until the LCD is ready to receive the commands generated by initialization so the LCD matches the restored states.
Lots more logging adding that lets us see what is going on.
As a result of the above I have managed to whittle down the boot time from the 25-30 minutes to almost exactly 10 from the white screen on the LCD until it's fully functional. I've made that 500 second countdown on the loading screen true. ;-D and sometimes even beat it.
Advantages:
Users can see all the relevant configs in PaperUI. I know that PaperUI is not great, but for users who are not Linux experienced it's a godsend. This will let all levels of users see and modify all the relevant settings from the browser. For example, they can change the labels for Items to their native language.
They can add or change which topic is subscribed to for the sensor readings (change source of sensor data to a remote sensor) from the UI.
They can see the Rules and see them running (the status icon next to each Rule turns green when a Rule is running).
They can disable/enable/modify rules within the browser. For example, if they want to change the Heating behavior, they can disable the default Rule and create their own without changing the default one. If something goes wrong, they can just reenable the original one.
The JSONDB Rules have three clauses, one of which is the "but only if..." or conditional clause. This lets you define what conditions need to be true before a Rule can execute. I've added lots of additional error checking to these clauses.
Doing things like letting users configure email and create alerts through PaperUI will be possible.
Disadvantages:
PaperUI Sucks! I admit it. It's clunky, hard to use, and requires lots of repetitive actions. I actually didn't use it much at all doing the above and instead create and/or modified the raw JSON and used the REST API instead. Hopefully most of the users will not be making massive changes so this won't be that big of a deal. But it's certainly not fun to use if you have to do a lot of creating.
Your job is going to be harder as the JSONDB is not very human friendly to read and review. If needed we might need to make a script to extract the Rules and make them easier to read. The problem is the actual JavaScript code ends up with all the new lines replaced with URL encoding. As a result the code all runs together. But the good news is no block of code is more than a dozen or so lines of code, most are under 10. Separating the error checking from the logic makes the code very short.
To help you review and help users understand the Rules so they can modify them, I've been working on some diagrams showing how the main Rules interact with each other. Here's an example:
Here are the things I don't know how to best handle as they have to deal with the install image:
The JSONDB and other configs configs are stored in /var/lib/openhab2. I can and will check those in to this repo under var/lib/openhab2 but don't know everything that needs to change to make sure they get copied to the right place during an upgrade and the initial install when you create the SD image.
I think this is OK as is, but you cannot upgrade these files while OH is running. I think the restore script already reboots, does the copying of files around and then reboots again. But on the first reboot, before copying the files, the script needs to kill OH before copying the files.
If we can delay the checking of the CPU load in the kiosk script a bit, I think we can shave another 10-30 seconds off of the boot time. But I don't know the implications of that. What I really want is the while loop that checks for the system load before switching the LCD UI to the working UI to look for a load of 2.5. But I can't use anything over 2.0 because while OH is first starting the load doesn't jump above 2.5 fast enough. So if we can delay that checking until after it first goes above 2.5 we don't have to wait as long for the system load to drop before switching over. Even at around a system load of 3 OH is functional and ready to go and in fact the rules that process the sensor readings will already be running.
Before I dump a whole lotta changes on you I wanted to open up a discussion and find out how you want me to proceed.