Dilbert66 / esphome-vistaECP

This is an implementation of an ESPHOME custom component and ESP Library to interface directly to a Safewatch/Honeywell/Ademco Vista 15/20 alarm system using the ECP interface and very inexpensive ESP8266/ESP32 modules .
GNU Lesser General Public License v2.1
130 stars 21 forks source link

Vista 25IT - Strange behavior on the bus #84

Closed lorenzodeveloper closed 1 year ago

lorenzodeveloper commented 1 year ago

Ciao Alain, Thank you very much for your effort! This project is amazing!

I currently have an Ademco Vista 25IT, that so far seems to be the italian version of the Vista 20SEa, with the following list of devices:

I followed the doc you wrote and assembled the ground isolated version of the circuit using a pair of 100kohm + 5.1kohm in parallel to make the 4.7kohm resistors (actually 4.85252 kohm) and a pair of 220ohm + 1kohm in parallel to make the 180 ohm resistor. Used ESP8266 wemos D1 R2. (pin mapping is the same of your example). (I used the 4N35 optocouplers)

Zones/addresses:

I'm not sure about the keypad address, looking at the manual, it seems that it should be at address 31, but I'm really not sure about it. The address I set on the yaml file is 36. lrr is set to true. (don't have one)

I made the following changes to the code to make it work with my panel:

// Italian language adaptation in vistaAlarm.h // hoping it is appreciated, here are the translations...

    const char *const FAULT = "APERT";
    const char *const BYPAS = "ESCL.";
    const char *const ALARM = "ALARM";
    const char *const FIRE = "INCEND";
    const char *const CHECK = "VERIF";
    const char *const CLOSED = "CLOSED"; //not used
    const char *const OPEN = "OPEN"; //not used
    const char *const ARMED = "INSERIM.";

    const char *const HITSTAR = "SPENTO Premi* e ";
    const char *const HITSTAR_ALT = "Premere *"; // sometimes it is displayed like this

...

    if (strstr(vista.statusFlags.prompt, HITSTAR) || strstr(vista.statusFlags.prompt, HITSTAR_ALT))
          vista.write('*');

// Status processing in vista.cpp I need to discard messages that contain only zeroes otherwise I'm not able to retrieve anything from the panel

void Vista::processStatus(char cbuf[], int *idx)
{
    // Before the for loop, discard only 00 packets
    bool onlyZeroes = true;
    for (int x = 0; x <= 10; x++)
    {
        if (cbuf[x] != 0)
        {
            onlyZeroes = false;
            break;
        }
    }
    if (onlyZeroes)
        return;
...

I plugged it to the bus and it works, I can read the bus and send commands to the panel.

The problem is that sometimes I get a lot of false positives, specially when I bypass a zone. This results in wrong reporting of the panel status (Alarm flag, Armed status, Bypassed zones reset, ...)

image image

After a brief investigation I found out that this is because these messages that are sometimes sent to the panel (not sent together)

[..:..:..][I][CMD:432]: 31 C3 0A 00 00 00 00 00 00 00 00 00 00 
or
[..:..:..][I][CMD:432]: B1 0D 6B 00 00 00 00 00 00 00 00 00 00 
or (most of the times)
[..:..:..][I][CMD:432]: 0E 00 00 00 00 00 00 00 00 00 00 00 00 

Only zeroes example

// Filtered out the "sending state C for sensor XYZ"

[16:07:02][I][CMD:436]: FB 02 20 82 61 00 00 00 00 00 00 00 00 
[16:07:02][I][EXT:436]: 00 24 03 D9 00 00 00 00 00 00 00 00 74 
[16:07:03][I][CMD:436]: 27 00 1C 18 00 00 00 00 00 00 00 00 00 
[16:07:03][I][CMD:436]: 00 00 00 00 00 00 00 00 00 00 00 00 00 
[16:07:05][I][CMD:436]: FE 65 72 20 20 00 00 00 00 00 00 00 00 
[16:07:05][I][INFO:696]: Prompt: SPENTO ZONE ESC                              //bypass is on on zone 10 that is not working
[16:07:05][I][INFO:697]: Prompt:  Si puo' inser  
[16:07:05][I][INFO:698]: Beeps: 0

[16:07:05][D][DEBUG:886]: ac=1,batt status = 0,systemflag=1,lightbat status=0,trouble=0
[16:07:10][D][DEBUG:566]: 30 second time check
[16:07:14][I][CMD:436]: 27 00 1C 18 00 00 00 00 00 00 00 00 00 
[16:07:14][I][CMD:436]: 00 00 00 00 00 00 00 00 00 00 00 00 00 
[16:07:16][I][CMD:436]: FE 65 72 20 20 00 00 00 00 00 00 00 00 
[16:07:16][I][INFO:696]: Prompt: SPENTO ZONE ESC                             //bypass is on on zone 10 that is not working
[16:07:16][I][INFO:697]: Prompt:  Si puo' inser  
[16:07:16][I][INFO:698]: Beeps: 0

Some other interesting logs

// Filtered out the "sending state C for sensor XYZ"

[16:10:55][I][CMD:436]: 27 00 1C 18 00 00 00 00 00 00 00 00 00 
[16:10:56][I][CMD:436]: 00 00 00 00 00 00 00 00 00 00 00 00 00 
[16:10:57][I][CMD:436]: FE 65 72 20 20 00 00 00 00 00 00 00 00 
[16:10:57][I][INFO:696]: Prompt: SPENTO ZONE ESC 
[16:10:57][I][INFO:697]: Prompt:  Si puo' inser  
[16:10:57][I][INFO:698]: Beeps: 0

[16:10:57][D][DEBUG:886]: ac=1,batt status = 0,systemflag=1,lightbat status=0,trouble=0
[16:11:06][I][CMD:436]: 27 00 1C 18 00 00 00 00 00 00 00 00 00 
[16:11:07][I][CMD:436]: 00 00 00 00 00 00 00 00 00 00 00 00 00 
[16:11:08][I][CMD:436]: FE 65 72 20 20 00 00 00 00 00 00 00 00 
[16:11:08][I][INFO:696]: Prompt: SPENTO ZONE ESC 
[16:11:08][I][INFO:697]: Prompt:  Si puo' inser  
[16:11:08][I][INFO:698]: Beeps: 0

[16:11:08][D][DEBUG:886]: ac=1,batt status = 0,systemflag=1,lightbat status=0,trouble=0
[16:11:09][D][DEBUG:566]: 30 second time check
[16:11:17][I][CMD:436]: FB 02 25 81 5D 00 00 00 00 00 00 00 00 
[16:11:18][I][CMD:436]: 27 00 1C 18 00 00 00 00 00 00 00 00 00 
[16:11:18][I][CMD:436]: B1 0D 6B 00 00 00 00 00 00 00 00 00 00 
[16:11:18][I][CMD:436]: FB 02 20 82 61 00 00 00 00 00 00 00 00 
[16:11:18][I][EXT:436]: 00 24 03 D9 00 00 00 00 00 00 00 00 74 
[16:11:20][I][CMD:436]: FE 65 72 20 20 00 00 00 00 00 00 00 00 
[16:11:20][I][INFO:696]: Prompt: SPENTO ZONE ESC 
[16:11:20][I][INFO:697]: Prompt:  Si puo' inser  
[16:11:20][I][INFO:698]: Beeps: 5

[16:11:20][D][text_sensor:067]: 'vistaalarm Beeps': Sending state '5'
[16:11:20][D][DEBUG:886]: ac=0,batt status = 0,systemflag=0,lightbat status=0,trouble=0
[16:11:20][D][text_sensor:067]: 'vistaalarm System Status': Sending state 'triggered'
[16:11:20][D][binary_sensor:036]: 'vistaalarm Fire': Sending state ON
[16:11:20][D][binary_sensor:036]: 'vistaalarm Trouble': Sending state ON
[16:11:20][D][binary_sensor:036]: 'vistaalarm AC': Sending state OFF
[16:11:20][D][binary_sensor:036]: 'vistaalarm Bypass': Sending state OFF
[16:11:20][D][binary_sensor:036]: 'vistaalarm Ready': Sending state OFF

[16:11:23][D][text_sensor:067]: 'vistaalarm System Status': Sending state 'triggered'
[16:11:25][D][text_sensor:067]: 'vistaalarm Beeps': Sending state '5'
[16:11:28][I][CMD:436]: FB 02 25 81 5D 00 00 00 00 00 00 00 00 
[16:11:28][I][EXT:436]: 00 21 40 9F 00 00 00 00 00 00 00 00 74 
[16:11:29][I][CMD:436]: 27 00 1C 18 00 00 00 00 00 00 00 00 00 
[16:11:29][I][CMD:436]: 00 00 00 00 00 00 00 00 00 00 00 00 00 
[16:11:31][I][CMD:436]: FE 65 72 20 20 00 00 00 00 00 00 00 00 
[16:11:31][I][INFO:696]: Prompt: SPENTO ZONE ESC 
[16:11:31][I][INFO:697]: Prompt:  Si puo' inser  
[16:11:31][I][INFO:698]: Beeps: 0

What can I do? Thank you in advance!

Dilbert66 commented 1 year ago

Ok, a couple things. The vista20se code version from dev has some issues as it's in the middle of some dev changes that I have not tested out yet. Use the version code from the Master branch. I should comment on that.

The keypad address you set for the vista20se in the yaml won't matter it's not needed. That's only used for the vista20 variants.

Your changes to remove the the zero values in processstatus is problematic as it will remove any cmd string that has a zero anywhere in it for the first 10 bytes. Also, 00 is a valid cmd string so should not be removed. The issue why you are getting all those zeros is odd. Might indicated a weak signal that can happen with the isolated version, but I don't see checksum errors on your FB command so that looks ok (0x77 on byte 12)

If you want to ensure you don't get 00 cmds, you can modify this section the vista::handle function:


         //if (x >=0 && x <0xf0 ) { //assume it's a status cmd //<- change this to 
             if (x >0 && x <0xf0 ) { //assume it's a status cmd // <- this only allow cmds above zero to be processed
            vistaSerial->setBaud(2400);
            newCmd = true;
            gidx = 0;
            cbuf[gidx++] = x;
            readChars(3, cbuf, & gidx, 5);
            processStatus(cbuf, & gidx);
            return 1;
        }

I have a very old vista20se that was donated to me by another user which I use to test. Unfortuantely, there are variants out there that might have some differences in the protocol.

lorenzodeveloper commented 1 year ago

Thank you for your quick response!

Yea, I checked the dev branch, but I've seen that it's missing something, so I haven't used it and I sticked to the master branch.

Regarding the 00 CMD: I should have thought about it, just implemented the suggested modification and now I'm discarding all the 00 CMDs. The reason why I discard it is that if don't do it I'm not able to detect faults or even report the ready status.

Ah ok, so in theory the resistors I used are ok, right? I had this doubt too since I've seen a lot of 74 bytes in the EXT logs after FB CMDs. I'm not able to find any log of it now, but sometimes (rare) I get weird readings for the prompt (like "VER_` 10 ZON 10" instead of "VERIF 10 ZONA 10")

What should I do instead for the B1, 31 and E0 commands? Should I discard them in that if condition too? I really cannot understand what they mean... they occur only during bypass or while armed they always have the same bytes. Thanks again.

Lorenzo

Dilbert66 commented 1 year ago

If you are getting missed characters in your prompts, then that indicates missed bits so your signal is not clean. I'm not sure why those zero's are causing you issues. It's obviously an artifact of some bits in the traffic flow that is not processed correctly.
The 0x74 in the EXT means it's an RF message that is unknown. That's ok. It just means that I don't know what those contain as I don't have much experience playing around with that system. As to the b1,31,e0, those could be valid as they most likely are status cmds. Anything below 0xf0 is and would normally be 5 bytes from what I saw. Even 0x00 is a valid status command usually. It could be different on your system.

I would recommend using the simple version of the circuit instead of the isolated one. I just find that it produces a much cleaner and less problematic signal.

lorenzodeveloper commented 1 year ago

Not sure, to me it looked more like a concurrency problem: what I thought is that it reads the right status update, e.g. during a FAULT event, but right after it reads the 00 00 ... 00, so when it enters on the strstr condition for the FAULT event in vistaAlarm.h the vista.statusFlags.zone is set to 0, so the actual zone is never updated. In short: a dirty read. Really not sure if it is possible actually, haven't checked it in details..

Thanks, I'll try in the weekend to assemble the simple version and let you know.

lorenzodeveloper commented 1 year ago

Just ordered a new breadboard, so I won't mess with the original circuit and I can simply swap between the two in case.. I think you're right, during the night I've experienced a lot of problems with missing bytes: image

Anyway, I had a spare RF antenna I used long time ago for a SDR project. I've found another interesting project on GitHub to decode the 345 MHz communication of the sensors to the panels and here are some logs. Let me know if it can help you with the RF decoding, I can post other logs.

// Panel data from your project when opening zone 16
[14:55:57][I][CMD:438]: FB 02 20 F1 F2 00 00 00 00 00 00 00 00 
[14:55:57][I][CMD:438]: FB 02 20 80 63 00 00 00 00 00 00 00 00 
[14:55:58][I][CMD:438]: FB 02 20 F1 F2 00 00 00 00 00 00 00 00 
[14:55:58][I][EXT:438]: 00 00 00 00 00 00 00 00 00 00 00 00 74 
[14:55:58][I][EXT:438]: 00 00 00 54 8D A8 6C 00 00 00 00 00 00 
[14:55:58][I][EXT:438]: 00 00 00 30 DB 00 00 00 00 00 00 00 00 
[14:55:58][I][CMD:438]: 16 00 00 08 00 00 00 00 00 00 00 00 00 

// Data from SDR
Honeywell Sensor - Valid Payload: FFFE8DA86C3084BA (Serial 895084/DA86C, Status 30)
Sensor Packet
device/895084/loop1    CLOSED
device/895084/loop2    OPEN
device/895084/loop3    OPEN
device/895084/tamper    OK
device/895084/battery    OK
device/895084/hb    PING
Honeywell Sensor - Valid Payload: FFFE8DA86C3084BA (Serial 895084/DA86C, Status 30)
Sensor Packet
Honeywell Sensor - Valid Payload: FFFE8DA86C3084BA (Serial 895084/DA86C, Status 30)
Sensor Packet
Honeywell Sensor - Valid Payload: FFFE8DA86C3084BA (Serial 895084/DA86C, Status 30)
Sensor Packet
Honeywell Sensor - Valid Payload: FFFE8DA86C3084BA (Serial 895084/DA86C, Status 30)
Sensor Packet
Honeywell Sensor - Valid Payload: FFFE8DA86C3084BA (Serial 895084/DA86C, Status 30)
Sensor Packet
Honeywell Sensor - Valid Payload: FFFE8DA86C3084BA (Serial 895084/DA86C, Status 30)
Sensor Packet
Honeywell Sensor - Valid Payload: FFFE8DA86C3084BA (Serial 895084/DA86C, Status 30)
Sensor Packet
Honeywell Sensor - Valid Payload: FFFE8DA86C3084BA (Serial 895084/DA86C, Status 30)
Sensor Packet

Here is the logs when closing zone 16

// Panel
[14:56:28][I][CMD:438]: FB 02 20 F1 F2 00 00 00 00 00 00 00 00 
[14:56:28][I][EXT:438]: 00 54 8D A8 00 00 00 00 00 00 00 00 73 
[14:56:28][I][EXT:438]: 00 54 8D 6C 10 FB 00 00 00 00 00 00 00 
[14:56:28][I][CMD:438]: 16 00 1C 08 00 00 00 00 00 00 00 00 00 
[14:56:30][I][CMD:438]: FE 52 49 52 45 00 00 00 00 00 00 00 00 

// SDR
Honeywell Sensor - Valid Payload: FFFE8DA86C100479 (Serial 895084/DA86C, Status 10)
Sensor Packet
device/895084/loop2    CLOSED
device/895084/hb    PING
Honeywell Sensor - Valid Payload: FFFE8DA86C100479 (Serial 895084/DA86C, Status 10)
Sensor Packet
Honeywell Sensor - Valid Payload: FFFE8DA86C100479 (Serial 895084/DA86C, Status 10)
Sensor Packet
Honeywell Sensor - Valid Payload: FFFE8DA86C100479 (Serial 895084/DA86C, Status 10)
Sensor Packet
Honeywell Sensor - Valid Payload: FFFE8DA86C100479 (Serial 895084/DA86C, Status 10)
Sensor Packet
Honeywell Sensor - Valid Payload: FFFE8DA86C100479 (Serial 895084/DA86C, Status 10)
Sensor Packet
Honeywell Sensor - Valid Payload: FFFE8DA86C100479 (Serial 895084/DA86C, Status 10)
Sensor Packet
Honeywell Sensor - Valid Payload: FFFE8DA86C100479 (Serial 895084/DA86C, Status 10)
Sensor Packet
Honeywell Sensor - Valid Payload: FFFE8DA86C100479 (Serial 895084/DA86C, Status 10)
Sensor Packet
Honeywell Sensor - Valid Payload: FFFE8DA86C100479 (Serial 895084/DA86C, Status 10)
Sensor Packet
Dilbert66 commented 1 year ago

I can sorta see the serial number in the EXT cmds but those zeros you are getting are really messing things up. I really can't say why that is. I would need to see logic analyzer traces to know for sure. I don't have any rf devices on the test board I have so really can't do much at this point. It's possible you are still getting some data corruption but I can't say for sure.

lorenzodeveloper commented 1 year ago

No worries.. I'll let you know when I'll try the simple version.

Anyway I've tried to experiment a bit with the baud rate when processing the FB commands in vista.cpp file.

// RF supervision
        if (x == 0xFB)
        {
            vistaSerial->setBaud(2400); //was 4800
            newCmd = true;
            gidx = 0;
            cbuf[gidx++] = x;
            readChars(4, cbuf, &gidx, 8);
            if (!validChksum(cbuf, 0, gidx))
                cbuf[12] = 0x77;

When it is set to 4800 it process the first messages correctly (I assume this because the same message on my board appeared also in other logs browsing the issues on this repo + I get no checksum error), but the answers (EXT, right?) are not processed in the right way cause I think the baud rate is 2400.

When setting to 2400 the first messages are in checksum error, but the next one are processed correctly since I'm also able to see the RFX message updated.

// Opening zone 16 with baud rate 2400

[19:00:59][I][CMD:441]: FB B1 2D 55 00 00 00 00 00 00 00 00 77 
[19:00:59][I][CMD:441]: FB B1 0D 6B 00 00 00 00 00 00 00 00 77 
[19:00:59][I][CMD:441]: FB 31 CB 3D 00 00 00 00 00 00 00 00 77 
[19:00:59][I][EXT:441]: FB 04 0D A8 6C 30 00 00 00 00 00 00 00 
[19:00:59][D][DEBUG:672]: RFX: 0895084,30
[19:00:59][D][text_sensor:067]: 'vistaalarm RF Msg': Sending state '0895084,30'

// Opening zone 16 with baud rate 4800

[14:55:57][I][CMD:438]: FB 02 20 F1 F2 00 00 00 00 00 00 00 00   // this is the message I've seen in other users' logs..
[14:55:57][I][CMD:438]: FB 02 20 80 63 00 00 00 00 00 00 00 00 
[14:55:58][I][CMD:438]: FB 02 20 F1 F2 00 00 00 00 00 00 00 00 
[14:55:58][I][EXT:438]: 00 00 00 00 00 00 00 00 00 00 00 00 74 
[14:55:58][I][EXT:438]: 00 00 00 54 8D A8 6C 00 00 00 00 00 00 
[14:55:58][I][EXT:438]: 00 00 00 30 DB 00 00 00 00 00 00 00 00 

Any idea on how to change baud rate depending on EXT or CMD? (if it is something that makes sense to try)

Dilbert66 commented 1 year ago

Excellent find. The way the vista works is that the cmd number is sent at 2400 baud for the FB and the rest of the data is sent at 4800 baud. So what needs to happen is that I reset the baud to 2400 after cmd receipt for data reception. This is done here:

Try this in vista.cpp for the fb cmd after line#1049 add "vistaSerial->setBaud(2400 );"

//RF supervision
        if (x == 0xFB) {
            vistaSerial->setBaud(4800);
            newCmd = true;
            gidx = 0;
            cbuf[gidx++] = x;
            readChars(4, cbuf, & gidx, 8);
            if (!validChksum(cbuf, 0, gidx)) 
                cbuf[12]=0x77;
            #ifdef MONITORTX
            memset(extcmd, 0, szExt); //store the previous panel sent data in extcmd buffer for later use
            memcpy(extcmd, cbuf, 5);
            vistaSerial->setBaud(2400 ); // <-- add this line to switch back to 2400
            #endif
            return 1;
        }
lorenzodeveloper commented 1 year ago

Thank you :)

I've applied the change, but it seems that it's ignored (like it still reads at 4800). // Opening zone 16

[09:22:10][I][CMD:441]: FB 02 25 80 5E 00 00 00 00 00 00 00 00  //correct
[09:22:10][I][CMD:441]: FB 02 20 F1 F2 00 00 00 00 00 00 00 00  //correct
[09:22:10][I][EXT:441]: 00 54 8D A8 00 00 00 00 00 00 00 00 73 // it seems like the log above when I kept only the 4800
[09:22:10][I][EXT:441]: 00 54 8D 6C 30 DB 00 00 00 00 00 00 00  //same of above

// Closing zone 16

[09:22:21][I][CMD:441]: FB 02 25 F1 ED 00 00 00 00 00 00 00 00 
[09:22:21][I][EXT:441]: 00 51 8D A8 00 00 00 00 00 00 00 00 73 
[09:22:21][I][EXT:441]: 00 51 8D 6C 10 00 00 00 00 00 00 00 00 
[09:22:21][I][EXT:441]: 00 51 8D FE 00 00 00 00 00 00 00 00 00 

Not sure on what is wrong :/

Dilbert66 commented 1 year ago

You can try adding is2400=true; after the setserial line to see if that helps any. Hard for me to know for sure what's going on with your data. I usually debug things like that using a logic analyzer trace. Honestly I'm surprised that the RF data would be sent at 2400 baud since on the vista it's normally at 4800.

lorenzodeveloper commented 1 year ago

Done, but same result... I tried also to change the baud rate of the vistaSerialMonitor, but no luck. Guess it was just a coincidence that I was able to get FB 04 messages when setting baud to 2400 for FB commands :/

Not sure if I'll buy a logic analyzer, but in case I'll let you know. In the meantime thank you very much for your support, you're super!

lorenzodeveloper commented 1 year ago

Ciao Alain, Just received a cheap logic analyzer. If I understood correctly, I need to disconnect the input wires (yellow and green inputs) to the esp and connect them to the logic analyzer and finally connect GND of the logic analyzer to the esp, right?

Then, if this setup is correct, after I setup also pulseview, what operations would like me to perform? I'd like also to debug the 00 cmds. For sure I think opening and closing doors with wireless sensors is one thing, then I - was thinking random commands from the keyboard. What do you think? Another question: would it be better to swap from gnd isolated version to simple version before getting the logs?

P.s. if the links from the other issues are working, I'll download the pulseview config zips from there.

Thank you again for your time!

Dilbert66 commented 1 year ago

You don't need to disconnect anything to use the logic analyzer. It just plugs in parallel to the lines. Sure, the wireless sensors are good ones to test. Keyboard cmds i already have.

You should not need to swap to the simple version unless you got a lot of 0x77 checksum issues. You need to be monitoring on the ESP side, not the panel side

Dilbert66 commented 1 year ago

I pushed an update that corrects some issues with RF checksum calc. Dont' think it will correct the baud rate issues you have though.

Dilbert66 commented 1 year ago

I pushed another update to deal with new unknown cmds or errant data. This might help your issue as well.

lorenzodeveloper commented 1 year ago

Super! I'll try and get back to you with the traces and feedback of the updates :) Unfortunately, it's been a busy weekend and I didn't have time to work on it.

lorenzodeveloper commented 1 year ago

Ciao Alain, sorry for the delay of getting back to you. I have a couple of updates:

  1. I flashed the updates you pushed, but no luck in the errant data: sometimes I keep getting missing chars in the Line1 item or I keep getting B1/31/something else commands that lead to changes in AC, lightbat status, etc. flags. (not so frequently, but still visible in the logs).
  2. Sometimes in the morning when I try to disarm the system the keyboard looks like it's not responding to the key presses. I mean, it does make the beep sound, but it won't disarm. I'm experiencing troubles with 1 wireless PIR sensor (it's faulty), so I don't know if it could be related to this one. It happened only 2 times, but the behavior was the following: disarm code -> nothing. disarm code -> nothing, but "check" appears on the keypad. disarm code -> nothing. disarm code -> disarmed -> check.
  3. Here you can find the trace files with PulseView software. Hope I wired everything correctly.. I used the config and instructions you posted in the issue of the Vista 20se (logic analizer -> https://www.az-delivery.de/en/products/saleae-logic-analyzer). Let me know if they're ok. I opened and closed zone 25, zone 22 and zone 24. All of them are wireless sensors, 25 and 22 same type, 24 is smaller (don't know if it's a different model). You will see also a lot of zone 3 and zone 4 triggers, but those are wired pir sensors... No key presses during the capturing traces.zip

Let me know if you need anything else. Thank you!

Dilbert66 commented 1 year ago

Thanks Lorenzo. I see the panel data and it looks correct but no rx data from the peripherals (green line). You might want to check that connection. I use the same type of logic analyzer, so you are good there. The issues you describe means that there is somethign interfering with the green line in the circuit. So this leads me to think there is something odd with your wiring.

lorenzodeveloper commented 1 year ago

Doh, I swapped the CH1 and CH2 and I also connected CH2 in the wrong place previously, I think I need the Christmas holidays :) Let me know if this trace is ok. Thank you! test ok.zip

I'll double check the wiring.

Dilbert66 commented 1 year ago

Ok, your traces are perfect! Thank you. So far I don't see anything amiss. Everything matches the protocol for the vista20se so far. I'll push a small change to see if it helps. Please post some esphome logs. I don't need any more traces right now as I've confirmed that your RF cmds are correct and the signals look good as the calculated checksum is correct. I've circled in blue the response from one of your RF sensors.
image

Dilbert66 commented 1 year ago

Going through it a bit better. I see a few missed cmd packets though, on the display side. For the vista20se , the display cmds are first a cmd with a cmd# below f0, then an FE cmd followed by 7 FF cmds that hold the ascii string. Some are missing some ff's . Not sure what's going on here. Usually you see missed bits, not whole cmd packets... I might have to tweak my code a bit.

lorenzodeveloper commented 1 year ago

Interesting! Just flashed the code. Here is some log, opening and closing zone 25. Let me know what you'd like to see better :) logs_esphome-web-fc535f_logs.txt

Not sure if it was present last time, but after I finished the OTA update I've seen these 2 commands

CMD:441 FB 02 20 81 62 00 00 00 ... 00
EXT:441   00 24 00 00 FF FF FF 00 .... 74

The other records do not contain those FF packets.

Could it be related to what you've seen? Unfortunately I cannot tell if this behavior was present the other times; I suppose yes, but I haven't paid attention to it..

Dilbert66 commented 1 year ago

Ok, that looks promising. The FB cmds are coming through. It's the RF devices responses that are corrupted. Most likely my code as your traces looked perfect for those. The EXT cmds after the FB are what we are looking for to get the serial# and on/off status of the RF sensor. Right now, they are not getting picked up correctly. Not sure yet if it's my code logic for this or a signal issue. I don't have RF sensors to test with so a bit more difficult to debug.

lorenzodeveloper commented 1 year ago

Let me know if I can do some other tests for you :) For sure I can try to push a little bit more the cables into the esp and also into the breadboard, might be that they're a little bit loose. That's the only thing I can think about.

Ah, what about the 00 cmds I've filtered out in vista.cpp? Is there something visible from the traces about that?

lorenzodeveloper commented 1 year ago

Small update: just reset the inclusion of 00 commands in vista.cpp ( reset to be >=) And they're not sent anymore after a CMD, don't know what has changed tbh, was it part of your previous commits?

But still I get the "messed up" EXTs of the FB commands

[12:19:49][I][CMD:441]: FB 02 20 81 62 00 00 00 00 00 00 00 00 
[12:19:49][I][EXT:441]: 00 24 00 00 00 00 00 00 00 00 00 00 74 
[12:19:49][I][EXT:441]: 00 24 00 40 9C 00 00 00 00 00 00 00 00 
[12:19:50][I][CMD:441]: 03 00 1C 08 00 00 00 00 00 00 00 00 00 

I'll check the wiring in the afternoon.

lorenzodeveloper commented 1 year ago

Small update pt 2: wiring looks correct.

Dilbert66 commented 1 year ago

Ok, tks for checking. I'm looking at the logic and testing on my system to streamline the code. It needs tweaks.

Dilbert66 commented 1 year ago

I've pushed a new update to master for the SE. It should fix a lot of those issues with the invalid characters you have and also hopefully fix your issue with the FB cmds.

lorenzodeveloper commented 1 year ago

Thank you, Alain. I will not be able to test it until the 29th, but I will let you know afterwards. Thanks again and happy holidays!!

lorenzodeveloper commented 1 year ago

Ciao Alain, I just flashed the changes and so far it looks good! The FB commands are parsed correctly.

I can still read weird commands like

[11:56:29][I][CMD:441]: D1 03 00 00 00 00 00 00 00 00 00 00 00 
[11:56:29][I][EXT:441]: 00 21 40 00 24 40 9C 00 00 00 00 00 00 

that cause false positives for AC and trouble flags and beeps variable

[11:56:30][D][text_sensor:067]: 'vistaalarm Beeps': Sending state '3'
[11:56:30][D][text_sensor:067]: 'vistaalarm System Status': Sending state 'unavailable'
[11:56:30][D][binary_sensor:036]: 'vistaalarm Trouble': Sending state ON
[11:56:30][D][binary_sensor:036]: 'vistaalarm AC': Sending state OFF
[11:56:30][D][binary_sensor:036]: 'vistaalarm Ready': Sending state OFF

Here is some extracted log logs_esphome-web-fc535f_upload_3012.txt

Lines 120 and 543 got my attention, but it happened only once in 1h monitoring, moreover the logbook is not showing anymore missing chars in Line1 and Line2. I'll keep an eye on it, but I think in general the issue is solved. Let me know what you think. Thank you very much for your work!

Dilbert66 commented 1 year ago

Ok, the fb cmds look good. The ones with the 0x74 are responses from devices other than the sensors? Dongles? They do not send a serial number. Only a 1 byte status. I don't know what those are.

So had another look at the differences between your system and my old SE and one thing differs and thats the status commands which are the ones with the first byte below 0xff. On my system they are 4 bytes long. On your system, your panel sends an extra 2 zeros after which explains the zeros you were always getting. I'll create a define in the file to allow the code to select the proper handling dependent on the setting. I don't know if this will correct the invalid cmds that you do see though as those can happen when a byte is corrupt.

Unfortunately those old systems, unlike the vista20, do not checksum their status cmds so it's difficult to filter out bogus data. I'll have to see if I can figure out a solution. For now, I'll post a change shortly to address the extra zero byte issues for your status cmds

Dilbert66 commented 1 year ago

I've pushed a fix for the extra zero bytes on the status cmd. Lets see if that helps any.

lorenzodeveloper commented 1 year ago

I don't know what they are to be honest :/ The only other wireless sensors I have are just the keyfobs, but they're never used. Not sure if they have a ping mechanism or something.

In the next days I will try to use the old antenna + raspberry pi to see if they're caught with the project I used some time ago (maybe we can get some hints on what they are from that), but I'm pretty sure last time I checked I was able to catch only the messages you're also catching with FB commands (open, close, tamper, battery, ping, ...).

Dilbert66 commented 1 year ago

Looks to be 3 of those unknown devices:

Device ID 81 with response status 40
[12:00:10][I][CMD:441]: FB 02 20 81 62 00 00 00 00 00 00 00 00 
[12:00:11][I][EXT:441]: 00 24 40 9C 00 00 00 00 00 00 00 00 74 

Device ID 82 with response status 03
[12:00:00][I][CMD:441]: FB 02 25 82 5C 00 00 00 00 00 00 00 00 
[12:00:00][I][EXT:441]: 00 21 03 DC 00 00 00 00 00 00 00 00 74 

Device ID 60 with response status 40
[11:59:15][I][CMD:441]: FB 02 25 60 7E 00 00 00 00 00 00 00 00 
[11:59:15][I][EXT:441]: 00 21 40 9F 00 00 00 00 00 00 00 00 74 
lorenzodeveloper commented 1 year ago

Just flashed the changes and I get various unknown CMDs (79, CB, B1, ...)

logs_esphome-web-fc535f_upload_3012_after 00.txt

Other records not in the uploaded file

[17:01:24][I][CMD:441]: FB 02 20 F1 F2 00 00 00 00 00 00 00 00 
[17:01:25][I][CMD:441]: FB 02 20 80 63 00 00 00 00 00 00 00 00 
[17:01:25][I][CMD:441]: 31 79 00 00 00 00 00 00 00 00 00 00 00 
[17:01:25][I][EXT:441]: FB 04 0B 57 5E 80 00 00 00 00 00 00 00 
[17:01:25][D][DEBUG:672]: RFX: 0743262,80
[17:01:25][D][text_sensor:067]: 'vistaalarm RF Msg': Sending state '0743262,80'
[17:01:25][I][CMD:441]: 27 00 00 08 00 00 00 00 00 00 00 00 00 
[17:01:26][I][CMD:441]: B1 CB 15 00 00 00 00 00 00 00 00 00 00 
[17:01:26][I][EXT:441]: FB 04 0B 57 5E 00 00 00 00 00 00 00 00 
[17:01:26][D][DEBUG:672]: RFX: 0743262,00
[17:01:26][D][text_sensor:067]: 'vistaalarm RF Msg': Sending state '0743262,00'
[17:01:26][I][CMD:441]: 27 00 1C 08 00 00 00 00 00 00 00 00 00 
[17:01:27][D][text_sensor:067]: 'vistaalarm RF Msg': Sending state '0743262,00'
[17:01:28][I][CMD:441]: FE 52 49 52 45 00 00 00 00 00 00 00 00 

...

[17:04:03][I][CMD:441]: FB 02 20 60 83 00 00 00 00 00 00 00 00 
[17:04:03][D][DEBUG:571]: 30 second time check
[17:04:03][I][EXT:441]: 00 24 40 9C 00 00 00 00 00 00 00 00 74 
[17:04:03][I][CMD:441]: 27 00 1C 08 00 00 00 00 00 00 00 00 00 
[17:04:05][I][CMD:441]: FE 52 49 52 45 00 00 00 00 00 00 00 00 
[17:04:05][I][INFO:701]: Prompt: **DISATTIVATO***
[17:04:05][I][INFO:702]: Prompt: SI PUO' INSERIRE
[17:04:05][I][INFO:703]: Beeps: 0

[17:04:06][D][DEBUG:891]: ac=1,low_batt status = 0,systemflag=1,lightbat status=0,trouble=0,check=0
[17:04:06][D][DEBUG:966]: Prev. sys. state=10, Curr. sys. state=10
[17:04:13][I][CMD:441]: 03 00 00 08 00 00 00 00 00 00 00 00 00 
[17:04:14][I][CMD:441]: B1 8B D5 00 00 00 00 00 00 00 00 00 00 
[17:04:14][I][CMD:441]: FE 20 20 20 20 00 00 00 00 00 00 00 00 
[17:04:14][D][text_sensor:067]: 'vistaalarm Line1': Sending state 'APERT 03 ZONA 03'
[17:04:14][D][text_sensor:067]: 'vistaalarm Line2': Sending state '                '
[17:04:14][I][INFO:701]: Prompt: APERT 03 ZONA 03
[17:04:14][I][INFO:702]: Prompt:                 
[17:04:14][I][INFO:703]: Beeps: 3
Dilbert66 commented 1 year ago

Very odd as this works fine on my system. Can you post analyzer traces of the times you get those unknown cmds. I need to see the sequence and timings of those cmds. I don't think the issue is related to the zero cmds changes. This looks more to a corrupt FB cmd request.

lorenzodeveloper commented 1 year ago

Here it is. I hope I selected the right files, one should contain a D1 command and the other a 79 command.

Logs.zip

Not sure if it was a coincidence, but when I had the panel opened those commands didn't happen. So it might be some sort of interference when I have the panel closed. I'll check better tomorrow if there is some cable or something that get pushed when I close the panel (it shouldn't be so, but I'll double check).

Dilbert66 commented 1 year ago

Thanks. The logs and traces look good! I'll go through them and let you know.

lorenzodeveloper commented 1 year ago

Little update: Just opened the panel and nothing is pushed when the panel is closed, so that's good, but the bad thing I just found out is that the 5881H (RF receiver I have) has a led that is continuously illuminated and that led is an indicator of strong local radio frequency interference, reading from the manual. image

When I disconnect from the AC the wemos, the led turns off, so I think I need to relocate the wemos outside the panel. To be honest, I wasn't expecting this since the 2 objects work using very different frequencies (2.4GHz wifi vs 345MHz of RF device), but I think it's worth a try. Also I'm not sure if this is the cause of the unknown commands on the bus. I will do it in the next days and let you know if I can find a good spot for it and if I notice any change.

Dilbert66 commented 1 year ago

Actually I did find one issue that my code was not handling correctly and was causing some missed characters as seen in your logs. Good logs by the way. Sometimes the panel sends two command one after another with no gap. I added a new check for that. Unfortunately my system doesnt or rarely sends those so I can't test it but the code works otherwise. I pushed an update so give it a shot

Dilbert66 commented 1 year ago

Interference can also be caused by power components and strong freqs can produce other harmonics on other frequencies especially in close proximity. I would keep the esp further from the RF module.

lorenzodeveloper commented 1 year ago

Ciao Alain, happy new year! I managed to change the location of the esp by extending the jumper cables and pushing them through a hole beneath the panel. Everything works fine and the LED is solid off.

I flashed the changes and it seems to not work sometimes, I can see the same CMDs (it's rare, but still happening). Would you like to see a trace?

In the meantime I attach some apps log.

Screenshot_20230102_171059_Home Assistant.jpg

Dilbert66 commented 1 year ago

Yes, I see that 31 79 there. The rest are normal. I'm at a loss right now as to what it could be. Yes, please post some traces of when the corrupt cmds show up. That's the only way really of seeing what is happening.

Dilbert66 commented 1 year ago

And a Happy New Year to you as well!!

Dilbert66 commented 1 year ago

Ok, I was doing some testing and found one possible cause of the issue. I will do more testing and post a fix when I can confirm.

lorenzodeveloper commented 1 year ago

Great! In the meantime here are other traces + logs that might be helpful. If it's of any help, I noticed that it is easier to reproduce if I open and close the door quickly.

Commands that I was able to obtain are: F0, 79, B1, 31.

Thank you again! logs.zip

Dilbert66 commented 1 year ago

Good logs. The issue is that intermittently the esp is not picking up some of the RF FB messages. It might be picking up noise or some other data that corrupts them so you get odd bytes displayed instead that look like a different cmd. The traces look fine but it's possible that the esp is not as sensitive as the logic analyzer to pick up the signal. I doin't see any thing too odd with the data so I think my code is working as it should. I did see one instance where my logic could be improved to reject certain generated characters but it doesnt explain the other ones. It's tough to know for sure. I can't duplicate the issue here so it's all a guess.

I added some debug logic to display more of the captured data to see if it will shed more light on the issue. Try it and and see what you can capture.

Again, unfortunately on these old version of the vista, they never implemented checksumming on the display cmd packets so it's very difficult to reject bad cmds as there is no good way of validating them. I might try a band solution later to do a check on the AC power bit which is normally consistent and see if that helps to prevent invalid flags. It still doesnt solve the issue of the data corruption though.

lorenzodeveloper commented 1 year ago

Thanks :) do you need both logs and traces? if only logs in the meantime I can give you this: test_logs_esphome-web-fc535f_upload.txt

There is one F0 9E and CB commands.

Dilbert66 commented 1 year ago

Thanks! That's ok for now. I have enough from the other logs too. It's a weird one for sure. I'll let you know if I need something else. Thanks again!

Dilbert66 commented 1 year ago

I've made a small addition to check for parity to improve serial reception. Lets see if that helps with your issue.