Stability Connection // OTA Update

[PipeDeveloper]

Please read the end of the page for all the summary and solution

This happen when i added all my zones, total: 31 zones.

Im using a NodeMCU board v3, but they are all the same, just those cheap from aliexpress.

Other observation is when HTTP web server is enabled with the default configuration, it shows 11 zones correctly. When i do with 31 zones after log on web page the device reboots itself. But if i dont touch anything, i works stable on Home Assistant.

Here's the issue: After 2 minutes online, the device just disconnect and doesnt connect anymore to my wifi. (also for wifi issues, it doesnt enter on AP mode) During those two minutes it shows correctly all zones, sensors. This happen when i disable the HTTP web. It doesnt let me update via OTA, so i unplugged the NodeMCU and flash it manually using ESPHome-Flasher-1.3.0-Windows-x64 (already i used it before, no problem). I made a coppy just in case, so i turned back to the "last" config with HTTP web enabled but the issue persist.

The image shows the exact moment when i did the change, before that point i was connected pretty stable.

I leave the whole config, obviusly some parameters has been changed such passwords and codes.

#for documentation see project at https://github.com/Dilbert66/esphome-dsckeybus
substitutions:
  systemName: "alarma"
  accessCode: "1234" #Only comes into effect if a password prompt occurs when arming eg. night mode
  cmdWaitTime: "0" # milliseconds.  set how long to delay before accepting a new 05 cmd as valid to filter out intermittent short duration bogus commands that some panels send. default = 0

esphome:
  name: $systemName
  platform: ESP8266
  board: nodemcuv2

  includes:
    - dscKeybusInterface/

wifi:
  ssid: "MY_SSID"
  password: "my_password"

  manual_ip:
    static_ip: 192.168.1.90
    gateway: 192.168.1.1
    subnet: 255.255.255.0

  ap:
    ssid: "$systemName Fallback"
    password: "another_password"

#web_server:
#  port: 80
#  css_url: "http://192.168.1.51:8123/local/esphome-web/webserver-v1.min.css"
#  js_url: "http://192.168.1.51:8123/local/esphome-web/webserver-v1.min.js"
#  auth:
#    username: username
#    password: "another_password"

logger:
  baud_rate: 0
  level: info

api:
   password: 1234

ota:
   safe_mode: True
   password: 1234

status_led:
  pin:
    number: D4
    inverted: yes

custom_component:
- lambda: |-
    auto DSCkeybus = new DSCkeybushome();

    DSCkeybus->accessCode="$accessCode";
    DSCkeybus->cmdWaitTime=$cmdWaitTime;
    DSCkeybus->debug=2; // 0 = off, 1 = status change data, 2 = + 05/27 packets, 3 = + all packets received

    DSCkeybus->onSystemStatusChange([&](std::string statusCode) {
       id(system_status).publish_state(statusCode);
    });
    DSCkeybus->onPartitionStatusChange([&](uint8_t partition,std::string statusCode) {
       switch(partition) {
         case 1: id(p1).publish_state(statusCode); break;
         case 2: id(p2).publish_state(statusCode); break;
      }
    });
    DSCkeybus->onPartitionMsgChange([&](uint8_t partition,std::string msg) {
       switch(partition) {
         case 1: id(m1).publish_state(msg); break;
         case 2: id(m2).publish_state(msg); break;
      }
    });
    DSCkeybus->onTroubleStatusChange([&](troubleStatus ts,bool open) {
        switch(ts) {
            case trStatus: id(tr).publish_state(open);break;
            case batStatus: id(bat).publish_state(open);break;
            case acStatus: id(ac).publish_state(open);break;
            case panicStatus: id(panic).publish_state(open);break;
            default: break;
        }
    });
    DSCkeybus->onFireStatusChange([&](uint8_t partition, bool open) {
      switch (partition) {
          case 1: id(f1).publish_state(open); break;
      }
    });
    DSCkeybus->onZoneStatusChange([&](uint8_t zone, bool open) {
      switch (zone) {
        case 1: id(z1).publish_state(open); break;
        case 2: id(z2).publish_state(open); break;
        case 3: id(z3).publish_state(open); break;
        case 4: id(z4).publish_state(open); break;
        case 5: id(z5).publish_state(open); break;
        case 6: id(z6).publish_state(open); break;
        case 7: id(z7).publish_state(open); break;
        case 8: id(z8).publish_state(open); break;
        case 9: id(z9).publish_state(open); break;
        case 10: id(z10).publish_state(open); break;
        case 11: id(z11).publish_state(open); break;
        case 12: id(z12).publish_state(open); break;
        case 13: id(z13).publish_state(open); break;
        case 14: id(z14).publish_state(open); break;
        case 15: id(z15).publish_state(open); break;
        case 16: id(z16).publish_state(open); break;
        case 17: id(z17).publish_state(open); break;
        case 18: id(z18).publish_state(open); break;
        case 19: id(z19).publish_state(open); break;
        case 20: id(z20).publish_state(open); break;
        case 21: id(z21).publish_state(open); break;
        case 22: id(z22).publish_state(open); break;
        case 23: id(z23).publish_state(open); break;
        case 24: id(z24).publish_state(open); break;
        case 25: id(z25).publish_state(open); break;
        case 26: id(z26).publish_state(open); break;
        case 27: id(z27).publish_state(open); break;
        case 28: id(z28).publish_state(open); break;
        case 29: id(z29).publish_state(open); break;
        case 30: id(z30).publish_state(open); break;
        case 31: id(z31).publish_state(open); break;
      }
    });
    DSCkeybus->onZoneAlarmChange([&](uint8_t zone, bool open) {
      switch (zone) {
        case 1: id(za1).publish_state(open); break;
        case 2: id(za2).publish_state(open); break;
        case 3: id(za3).publish_state(open); break;
        case 4: id(za4).publish_state(open); break;
        case 5: id(za5).publish_state(open); break;
        case 6: id(za6).publish_state(open); break;
        case 7: id(za7).publish_state(open); break;
        case 8: id(za8).publish_state(open); break;
        case 9: id(za9).publish_state(open); break;
        case 10: id(za10).publish_state(open); break;
        case 11: id(za11).publish_state(open); break;
        case 12: id(za12).publish_state(open); break;
        case 13: id(za13).publish_state(open); break;
        case 14: id(za14).publish_state(open); break;
        case 15: id(za15).publish_state(open); break;
        case 16: id(za16).publish_state(open); break;
        case 17: id(za17).publish_state(open); break;
        case 18: id(za18).publish_state(open); break;
        case 19: id(za19).publish_state(open); break;
        case 20: id(za20).publish_state(open); break;
        case 21: id(za21).publish_state(open); break;
        case 22: id(za22).publish_state(open); break;
        case 23: id(za23).publish_state(open); break;
        case 24: id(za24).publish_state(open); break;
        case 25: id(za25).publish_state(open); break;
        case 26: id(za26).publish_state(open); break;
        case 27: id(za27).publish_state(open); break;
        case 28: id(za28).publish_state(open); break;
        case 29: id(za29).publish_state(open); break;
        case 30: id(za30).publish_state(open); break;
        case 31: id(za31).publish_state(open); break;
      }
    });
    return {DSCkeybus};

binary_sensor:
    #zone status
  - platform: template
    id: z1
    name: "z1 Puerta Principal"
    device_class: door
  - platform: template
    id: z2
    name: "z2 Puerta Cocina"
    device_class: door
  - platform: template
    id: z3
    name: "z3 Cocina 1"
    device_class: window
  - platform: template
    id: z4
    name: "z4 Cocina 2"
    device_class: window
  - platform: template
    id: z5
    name: "z5 Comedor 1"
    device_class: window
  - platform: template
    id: z6
    name: "z6 Comedor 2"
    device_class: window
  - platform: template
    id: z7
    name: "z7 Living 1"
    device_class: window

    #z8 is the tamper
  - platform: template
    id: z8
    name: "z8 Caja Sirena"
    device_class: lock
  - platform: template
    id: z9
    name: "z9 Living 2"
    device_class: window
  - platform: template
    id: z10
    name: "z10 Living 3"
    device_class: window
  - platform: template
    id: z11
    name: "z11 Living 4"
    device_class: window
  - platform: template
    id: z12
    name: "z12 Sala Star 1"
    device_class: window
  - platform: template
    id: z13
    name: "z13 Sala Star 2"
    device_class: window
  - platform: template
    id: z14
    name: "z14 Dormitorio Final 1"
    device_class: window
  - platform: template
    id: z15
    name: "z15 Dormitorio Final 2"
    device_class: window
  - platform: template
    id: z16
    name: "z16 Dormitorio Final 3"
    device_class: window
  - platform: template
    id: z17
    name: "z17 Dormitorio Felipe"
    device_class: window
  - platform: template
    id: z18
    name: "z18 Dormitorio Andres"
    device_class: window
  - platform: template
    id: z19
    name: "z19 Baño Felipe"
    device_class: window
  - platform: template
    id: z20
    name: "z20 Baño Pieza Fondo"
    device_class: window
  - platform: template
    id: z21
    name: "z21 Pieza Cocina"
    device_class: window
  - platform: template
    id: z22
    name: "z22 Baño Cocina"
    device_class: window
  - platform: template
    id: z23
    name: "z23 Rayo 1 Living"
    device_class: motion
  - platform: template
    id: z24
    name: "z24 Rayo 2 Cocina"
    device_class: motion
  - platform: template
    id: z25
    name: "z25 Rayo 3 Sala Star"
    device_class: motion
  - platform: template
    id: z26
    name: "z26 Dorm Matrimonial 1"
    device_class: window
  - platform: template
    id: z27
    name: "z27 Dorm Matrimonial 2"
    device_class: window
  - platform: template
    id: z28
    name: "z28 Dorm Matrimonial 3"
    device_class: window
  - platform: template
    id: z29
    name: "z29 Dorm Matrimonial 4"
    device_class: window
  - platform: template
    id: z30
    name: "z30 Closet Matrimonial"
    device_class: window
  - platform: template
    id: z31
    name: "z31 Baño Matrimonial"
    device_class: window

    #zone alarm status (I just disabled the entities za1 to za31, i dont feel it is usefull when the alarm is not armed)
  - platform: template
    id: za1
    name: "za1 Puerta Principal"
    #device_class: safety
  - platform: template
    id: za2
    name: "za2 Puerta Cocina"
    #device_class: safety
  - platform: template
    id: za3
    name: "za3 Cocina 1"
    #device_class: safety
  - platform: template
    id: za4
    name: "za4 Cocina 2"
    #device_class: safety
  - platform: template
    id: za5
    name: "za5 Comedor 1"
    #device_class: safety
  - platform: template
    id: za6
    name: "za6 Comedor 2"
    #device_class: safety
  - platform: template
    id: za7
    name: "za7 Living 1"
    #device_class: safety
  - platform: template
    id: za8
    name: "za8 Caja Sirena"
    #device_class: safety
  - platform: template
    id: za9
    name: "za9 Living 2"
    #device_class: safety
  - platform: template
    id: za10
    name: "za10 Living 3"
    #device_class: safety
  - platform: template
    id: za11
    name: "za11 Living 4"
    #device_class: safety
  - platform: template
    id: za12
    name: "za12 Sala Star 1"
    #device_class: safety
  - platform: template
    id: za13
    name: "za13 Sala Star 2"
    #device_class: safety
  - platform: template
    id: za14
    name: "za14 Dormitorio Final 1"
    #device_class: safety
  - platform: template
    id: za15
    name: "za15 Dormitorio Final 2"
    #device_class: safety
  - platform: template
    id: za16
    name: "za16 Dormitorio Final 3"
    #device_class: safety
  - platform: template
    id: za17
    name: "za17 Dormitorio Felipe"
    #device_class: safety
  - platform: template
    id: za18
    name: "za18 Dormitorio Andres"
    #device_class: safety
  - platform: template
    id: za19
    name: "za19 Baño Felipe"
    #device_class: safety
  - platform: template
    id: za20
    name: "za20 Baño Pieza Fondo"
    #device_class: safety
  - platform: template
    id: za21
    name: "za21 Pieza Cocina"
    #device_class: safety
  - platform: template
    id: za22
    name: "za22 Baño Cocina"
    #device_class: safety
  - platform: template
    id: za23
    name: "za23 Rayo 1 Living"
    #device_class: safety
  - platform: template
    id: za24
    name: "za24 Rayo 2 Cocina"
    #device_class: safety
  - platform: template
    id: za25
    name: "za25 Rayo 3 Sala Star"
    #device_class: safety
  - platform: template
    id: za26
    name: "za26 Dorm Matrimonial 1"
    #device_class: safety
  - platform: template
    id: za27
    name: "za27 Dorm Matrimonial 2"
    #device_class: safety
  - platform: template
    id: za28
    name: "za28 Dorm Matrimonial 3"
    #device_class: safety
  - platform: template
    id: za29
    name: "za29 Dorm Matrimonial 4"
    #device_class: safety
  - platform: template
    id: za30
    name: "za30 Closet Matrimonial"
    #device_class: safety
  - platform: template
    id: za31
    name: "za31 Baño Matrimonial"
    #device_class: safety

  - platform: template
    id: tr
    name: "$systemName Trouble Status"
    device_class: problem
  - platform: template
    id: bat
    name: "$systemName Battery Status"
    device_class: problem

  - platform: template
    id: ac
    name: "$systemName AC Status"
    device_class: plug

  - platform: template
    id: panic
    name: "$systemName Panic Status"
    device_class: safety

  - platform: template
    id: f1
    device_class: safety
    name: "$systemName Fire Status"

text_sensor:
  - platform: template
    id: system_status
    name: "$systemName System Status"
    icon: "mdi:shield"
  - platform: template
    id: p1
    name: "$systemName Partition 1 Status"
    icon: "mdi:shield"
  - platform: template
    id: p2
    name: "$systemName Partition 2 Status"
    icon: "mdi:shield"  
  - platform: template
    id: m1
    name: "$systemName Partition 1 Msg"
    icon: "mdi:alert-box"
  - platform: template
    id: m2
    name: "$systemName Partition 2 Msg"
    icon: "mdi:alert-box"

switch:
  - platform: template
    name: "$systemName Connection"
    id: connection_status_switch
    lambda: |-
      return dsc.keybusConnected;
    icon: "mdi:shield-link-variant"
    turn_on_action:
      - switch.toggle: restart_switch
    turn_off_action:
      - lambda: |-
          disconnectKeybus();
  - platform: restart
    id: restart_switch

[Dilbert66]

Can you try with the latest dev version at: https://github.com/Dilbert66/esphome-dsckeybus/tree/dev You will need to modify the yaml since this one will not use the zonealarmchange portion. The zone alarm process has now been changed to only one update field. This is because having too many fields can cause issues from what I've seen. There is also no need to have all those alarm zones anyhow since the idea is only to show which have triggered the alarm. Is there any reason you use the http web? To update via OTA you need to ensure you set the Connection field to off which disables the interrupts. How are you powering the nodemcu? You need to ensure you have at least 1 amp available. I have not encountered the issue but it sounds like the device is resetting due to the wdt. That would mean that it is too busy to handle all the processing it needs to do.

If you can try the dev version, set the debug=3 in the lambda then monitor via serial, we can perhaps see what is causing your reboot. You might see if you are getting a WDT reset .

[PipeDeveloper]

Hello! Thank you for your answer

I have done what you said, update the library to dev verson and the .yaml file. Yep got your point and thats why i put that the zonealarmchange is not so usefull because also the binary sensors are enough information if the alarm is triggered. You just need to see the history report, if all zones are closed when the alarm is armed, well is obvius that the one that changed to open triggered the alarm.

HTTP web is just in case that my home assistant loose comunication, to check via web if the nodeMCU still connected. Not so usefull and for memory save i will disable it.

For update via OTA thats very curious, but good to know that the switch "connection" is to disable the interrrups. For future updates like changing zone names or wherever, i'll have in mind that i have to switch off this 👍

About power Im ussing the "RED" and "BLK" terminal directly from the pannel. I have understand that it is a 12v power supply for sensors and many devices. The things connected to it are the pysical keypad, the RF receptor (for wireless sensors) and the NodeMCU.

Let me monitor the logs and stability for 24h and lets see if something happens in the way. Thank you!

[Dilbert66]

ok, I was able to sort of replicate your issue. I used 32 zones and also enabled the web server. without the web server (and the new code from dev), everything worked ok. Enable the web server and connecting kept rebooting the server. If I limited the yaml to about 10 zones only. Everything was fine even with the web server enabled. This looks like the esp code is hitting a resource limit especially with the web server using up even more ram. The esp reboots with an "Abort called" with a stack trace that needs decoding to know for sure what the cause is.

[Dilbert66]

I've pushed a new version to the dev branch that optimizes ram a bit. I can now run the web server with 32 zones without resets. I've also updated ESPHOME to the latest version. Try it if you want to see if it helps.

[Dilbert66]

As you can see also in the dev version, i've added your suggestion about wireless battery low status. They will show in the zone status field as "BL:xx"

[PipeDeveloper]

Running for about 2hours and seems to be stable.

If it is a memory issue, you recomend to use a more powerfull board such an ESP32? If everything is optimized for ESP8266, it should have enough ram/cpu to handle all. I don't know if there are compatibility issues with this library. Seems that ESPhome supports ESP32, so it should works as well.

About the battery status for wireless sensors the entitie detects the sensor but the value is "unknown" Im not sure about this, but maybe the value is an array of strings, in case that two sensors got low battery. I imagine that the value should be something like "BL:01","BL:04","BL19" (3 sensors low battery) or "null" (no sensors with low battery). Or maybe signals are diferent and depends on the Alarm Panel model, on your case i think you're testing on a different panel than mine.

My panel model is DSC PC1832

[Dilbert66]

Yes, the values are indeed an array of strings. ie BL:01, BL:02, etc. For alarms, it will show AL:01, AL:02, etc. As many are needed to display the values. This is why I used a text field because it is versatile in showing values. I also use a PC1832 and it works well for me. It starts as Unknown since it depends on the PC5132 wireless module to send data if it detects a low battery condition (after 3 low battery transmissions from the zone). The system can't know what is existing unless a 2 plus 7 are pressed to have the panel send the exact info. Currently the code does not read the results of the 2 response. It only uses the actual PC5132 module initial status update.

As far as the memory. You don't need to use the esp32, but you can as it should be supported. The problem is really the web server application. That uses a LOT of memory as noted in the esphome documentation and is really not recommended for use especially for a home security application. See the docs on the web server: https://esphome.io/components/web_server.html

The web_server component creates a simple web server on the node that can be accessed through any browser and a simple REST API. Please note that enabling this component will take up a lot of memory and can lead to problems, especially on the ESP8266.

[PipeDeveloper]

Excelent! Yea, definitely i will not use web server anymore.

I dont know if the wireless module have diferent packets. If this is the case my system got a DSC UA521

Another thing is how often sensors reports low battery. This is controvertial because on very low battery state, sensors sometimes doesn´t even change open/close state. This is absolutely normal and depends on the voltaje, RF signal strengh and interference factors specially on big houses. If it's working i should wait for the sensors transmission, but i imagine that it should update by itself in less than 5-10minutes. 4hours, the value still is "unkown"

BUT... For example, pressing *2 plus 7, system shows zone 1,3,4,etc... on the keypad zone1 which is the front door and it's pretty close to the wireless receptor. (sensor looks like this) This case i discard any interference and reception issue, keypad is clear and the detection of this zone always works fine. The expectation on this case is that at least show BL:01, so there is something missing on the script.

Just a suggestion It's more real to have a buttom to emulate *2 plus 7 on the keypad to force the full report of sensor battery. As i said that information is not so real if a sensor already ran out of battery and are unavailable to report the status (there is nothing to do unless you know that you got the report before, so you've been noticed) but you got the option to update the list manualy (also for development purposes) so you're sure that the correct info is displayed on HA.

[Dilbert66]

The way it works on the PC1832 I have with a PC5132 RF module is that the PC5132 will not send a low battery warning to the alarm panel until it receives at least 3 separate transmissions from the remote wireless module with the low battery attribute value set. For every transmission of course, the pc5132 will update the panel with open/close/motion status. On the 3rd transmission from the sensor with low battery still active, it will then inform the alarm panel which will then set the trouble light on. Pressing *2 plus 7 before that will not retrieve battery level unless it has been sent from the pc5132 module.

My software looks at the PC5132 initial notification and gets it at the same time the panel is notified. The trouble light will only disappear once the battery is replaced on the unit and the pc5132 notifies the panel (and my software). If a low battery notification is already present and you start my software after, it will not capture it since the pc5132 notification has already been sent previously. The pc5132 will not send it again as it knows that it has already updated the panel. Perhaps, a feature I could add is that if *2 7 is pressed (or sent) as you suggested, then my software can update it's memory record of the low battery zones and display that. I will see what is involved.
I'm not familiar with the DSA UA521 so its possible it functions differently. Perhaps you can capture some data. This will involve setting debug=3 in the yaml, and capturing the serial output data from the usb port of the esp8266 module.

[Dilbert66]

I believe will add the function to update status. It's a good idea. Since the trouble status is known always, I will add logic to check for all trouble light statuses (by sending the appropriate keys) and update any specific trouble status such as tamper, battery, etc

[Dilbert66]

From what I read, the dsc ua521 is the same as a north american PC5132 so they should function the same.

[Dilbert66]

Ok, i've pushed a new version to dev that updates the battery status from the 27 panel command. To get home assistant to update the zone status field, enter 27 manually. Sending *27 automatically from the firmware is not available at this time as it is problematic so you will need to do it once whenever you load new firmware or restart the esp8266. After that, it will update automatically the same way the panel does from the PC5132 status updates when a battery fails or is replaced.

[Dilbert66]

Ok, found a solution for the automatic send of the 27. The system will send 27## to trigger a panel update to the keypad of the current battery status on initial bootup only . It's now in dev.

[PipeDeveloper]

You're amazing!! it works instatly at the first boot. OTA update works fine.

This is a great and amazing feature, I love it, you love it, everyone who uses this will love it! hahaha Im really grateful 💯

I was watching some debug logs and i dont see anything outside place, also for the small stability changes holding 31 zones works fine and smoothly. 22hours online with stable connection, no cuts. I think this is done, clossing this issue.

If someone read this in the future, in summary and in simple words.

1.- Do not use HTTP web server application on your device when you got a lot of zones (31 on this case), this will consume your memory going into infinite loop rebooting.

2.- OTA update, you must first turn off the switch "connection", then you can update without problems.

3.- Added a feature: battery sensor report to show wireless zones that got low battery 👌

[Dilbert66]

Awesome! Glad it's working ok. Thanks for your help and the suggestion. It's a good add on feature.

[Dilbert66]

I think you need to go buy more batteries for your sensors! looks like you will be busy changing them all! :)

[PipeDeveloper]

@Dilbert66 That's right! hahaha They uses CR2032. In Chile some brands sell it for $5usd each. So i ordered a batch of 30units for less than $7usd on Aliexpress. arround November i'll have my system fully operative.