search

zwave-js / node-zwave-js

Z-Wave driver written entirely in JavaScript/TypeScript
https://zwave-js.github.io/node-zwave-js/
MIT License
750 stars 599 forks source link

Intermatic PE653 compatibility issues, firmware dependent #4588

Open j9brown opened 2 years ago

j9brown commented 2 years ago

Is your problem within Home Assistant (Core or Z-Wave JS Integration)?

NO, my problem is NOT within Home Assistant or the ZWave JS integration

Is your problem within ZWaveJS2MQTT?

NO, my problem is NOT within ZWaveJS2MQTT

Checklist

Describe the bug

The Intermatic PE653 doesn't seem to work correctly with zwave-js. The nature of the problems depends on the firmware running on the device.

Firmware v3.3:

Firmware v3.4:

I recently upgraded the firmware of my device to v3.4 since that's what's shipped from the factory these days and what I presume most other users have, with the hope of resolving the thermostat compatibility issue. I didn't want to submit compatibility fixes for the thermostat that might break behavior for users of v3.4. That said, v3.4 seems quite broken so I may revert to v3.3 after all.

(I'd be keen to know if anyone is actually using v3.4 successfully with zwave-js. FWIW, I originally had this device fully working with v3.3 and OpenZwave, including the thermostat.)

I'm going to keep investigating to determine a course of action to improve compatibility.

See also: discussion

Device information

Manufacturer: Intermatic Model name: PE653 Node ID in your network: 42

How are you using node-zwave-js?

Which branches or versions?

zwavejs2mqtt: 6.8.1 zwave-js: 9.0.7 home id: 4146670975 home hex: 0xf7292d7f

Did you change anything?

no

If yes, what did you change?

No response

Did this work before?

No, it never worked anywhere

If yes, where did it work?

No response

Attach Driver Logfile

zwavejs-pe653-discovery-with-firmware-v3.4.log

zwave-js-bot commented 2 years ago

👋 Hey @j9brown!

It looks like you attached a logfile, but its filename doesn't look like it a driver log that came from Z-Wave JS. Please make sure you upload the correct one.

j9brown commented 2 years ago

Yes it's the correct log, you silly bot, I just gave the file a more descriptive name since I'm also going to capture a log with the v3.3 firmware later. 😉

j9brown commented 2 years ago

Relevant part for thermostat discovery issue (for firmware v3.4 in this case). Slightly redacted (removed transmit stats).

2022-05-09T05:59:31.563Z CNTRLR   [Node 042] Interviewing Thermostat Setpoint...
2022-05-09T05:59:31.564Z CNTRLR » [Node 042] retrieving supported setpoint types...
2022-05-09T05:59:31.569Z SERIAL » 0x010900132a0243042552fd                                            (11 bytes)
2022-05-09T05:59:31.569Z DRIVER » [Node 042] [REQ] [SendData]
                                  │ transmit options: 0x25
                                  │ callback id:      82
                                  └─[ThermostatSetpointCCSupportedGet]
2022-05-09T05:59:31.573Z SERIAL « [ACK]                                                                   (0x06)
2022-05-09T05:59:31.579Z SERIAL « 0x0104011301e8                                                       (6 bytes)
2022-05-09T05:59:31.580Z SERIAL » [ACK]                                                                   (0x06)
2022-05-09T05:59:31.599Z SERIAL « 0x011800135200000200bc7f7f7f7f010103000000000201000018              (26 bytes)
2022-05-09T05:59:31.599Z SERIAL » [ACK]                                                                   (0x06)
2022-05-09T05:59:31.681Z SERIAL « 0x010b0004002a03430541be0060                                        (13 bytes)
2022-05-09T05:59:31.684Z CNTRLR   [Node 042] [+] [Thermostat Setpoint] supportedSetpoint [Endpoint 0] [internal]
                                  Types: 0,6
2022-05-09T05:59:31.684Z SERIAL » [ACK]                                                                   (0x06)
2022-05-09T05:59:31.685Z DRIVER « [Node 042] [REQ] [ApplicationCommand]
                                  └─[ThermostatSetpointCCSupportedReport]
                                      supported setpoint types:
                                      · N/A
                                      · unknown (0x06)
2022-05-09T05:59:31.686Z CNTRLR   [Node 042] uses Thermostat Setpoint bitmap interpretation A
2022-05-09T05:59:31.686Z CNTRLR » [Node 042] querying current value of setpoint N/A...
2022-05-09T05:59:31.689Z SERIAL » 0x010a00132a034302002553f8                                          (12 bytes)
2022-05-09T05:59:31.689Z DRIVER » [Node 042] [REQ] [SendData]
                                  │ transmit options: 0x25
                                  │ callback id:      83
                                  └─[ThermostatSetpointCCGet]
                                      setpoint type: N/A
2022-05-09T05:59:31.692Z SERIAL « [ACK]                                                                   (0x06)
2022-05-09T05:59:31.700Z SERIAL « 0x0104011301e8                                                       (6 bytes)
2022-05-09T05:59:31.700Z SERIAL » [ACK]                                                                   (0x06)
2022-05-09T05:59:31.721Z SERIAL « 0x011800135300000200bb7f7f7f7f01010300000000020100001e              (26 bytes)
2022-05-09T05:59:31.721Z SERIAL » [ACK]                                                                   (0x06)
2022-05-09T05:59:32.762Z CNTRLR   [Node 042] Timed out while waiting for a response from the node (ZW0201)
2022-05-09T05:59:32.763Z CNTRLR « [Node 042] Setpoint N/A is not supported
2022-05-09T05:59:32.763Z CNTRLR » [Node 042] querying current value of setpoint Auto Changeover...
2022-05-09T05:59:32.772Z SERIAL » 0x010a00132a0343020a2554f5                                          (12 bytes)
2022-05-09T05:59:32.773Z DRIVER » [Node 042] [REQ] [SendData]
                                  │ transmit options: 0x25
                                  │ callback id:      84
                                  └─[ThermostatSetpointCCGet]
                                      setpoint type: Auto Changeover
2022-05-09T05:59:32.777Z SERIAL « [ACK]                                                                   (0x06)
2022-05-09T05:59:32.784Z SERIAL « 0x0104011301e8                                                       (6 bytes)
2022-05-09T05:59:32.790Z SERIAL » [ACK]                                                                   (0x06)
2022-05-09T05:59:32.803Z SERIAL « 0x011800135400000300bc7f7f7f7f01010300000000020100001f              (26 bytes)
2022-05-09T05:59:32.804Z SERIAL » [ACK]                                                                   (0x06)
2022-05-09T05:59:33.850Z CNTRLR   [Node 042] Timed out while waiting for a response from the node (ZW0201)
2022-05-09T05:59:33.851Z CNTRLR « [Node 042] Setpoint Auto Changeover is not supported
2022-05-09T05:59:33.854Z CNTRLR   [Node 042] [~] [Thermostat Setpoint] supportedSetpoint [Endpoint 0] [internal]
                                  Types: 0,6 => 
2022-05-09T05:59:33.855Z CNTRLR   [Node 042] [+] [Thermostat Setpoint] setpointTypesInte [Endpoint 0] [internal]
                                  rpretation: "A"
2022-05-09T05:59:33.855Z CNTRLR   [Node 042] [+] [Thermostat Setpoint] interviewComplete [Endpoint 0] [internal]
                                  : true
j9brown commented 2 years ago

Tentative observation about firmware v3.4 switch behavior.

The PE953 remote (node 43) sends Multi Instance Cmd Encap (0x60 0x06) with COMMAND_CLASS_SWITCH_BINARY and SWITCH_BINARY_SET to turn switches on and off, successfully:

31106 08.05.2022 23:54:29.698 40Kbit/s 57 1 3180 043 042 XX XX XX XX Singlecast Multi Instance Cmd Encap XX XX XX XX 2B 41 09 10 2A 60 06 02 25 01 00 6A

Conversely, zwave-js (node 1) sends Multi Channel Cmd Encap (0x60 0x0D) with COMMAND_CLASS_SWITCH_BINARY and SWITCH_BINARY_SET to turn switches on and off, unsuccessfully:

31406 08.05.2022 23:56:31.803 40Kbit/s 55 1 61 001 042 XX XX XX XX Singlecast Multi Channel Cmd Encap XX XX XX XX 01 41 09 10 2A 60 0D 00 01 25 02 4B

AlCalzone commented 2 years ago

I originally had this device fully working with v3.3 and OpenZwave, including the thermostat

This is most likely because OZW didn't query thermostats for their supported modes, but rather relied on information in their config files. Z-Wave JS mostly relies on what the devices report.

The problem here seems to be a really poor implementation of the device firmware.

Problem 1: Like so often, the support bitmasks aren't implemented correctly by this device 🙄 It reports the bitmask as (binary) 0100_0001, which stands for setpoint 6 (not defined) and 0 (off), when it should be reporting 1000_0010, which is 7 and 1. We currently don't have a compat flag overriding the reported values -> https://github.com/zwave-js/node-zwave-js/issues/1625

Problem 2: The device reports support for Multi Channel CC V2, but doesn't seem to understand commands encapsulated with V2. It sends its own reports V1 encapsulated, which has a different format. Maybe forcing the CC version to be 1 via the existing compat flag might help here.

j9brown commented 2 years ago

Ok, I've crafted a configuration file that supports firmware v3.4 with conditional logic by forcing the use of the MultiChannelCC v1 and disabling certain broken command classes that confuse discovery. The lifeline association on endpoint 0 also seems to work now so no more polling.

I'm uncertain how to resolve the thermostat issue. I think we'll need a new compat flag for that.

AlCalzone commented 2 years ago

I think we'll need a new compat flag for that.

Yeah. I prefer it to be very generic though to be able to override all kinds of incorrectly reported capabilities.

Can you put up a PR with the first part of the fix?

JDogg016 commented 1 year ago

I am curious whether anybody made any headway on further integrating the Intermatic PE653 to display the temperature as well as operate a variable speed pump.

j9brown commented 1 year ago

I haven’t spent any more time on it and have been living with the broken thermostat control.

Working around the firmware off-by-one encoding bug would be pretty easy with a compatibility flag but I don’t believe such a fix would be accepted by this project. I considered contributing a more general compatibility feature but it looked like it would take a lot of refactoring to implement.

I have dumped the firmware of the PE653 and reverse engineered the firmware updater protocol. So another option would be to patch the bug in the firmware itself (probably only requires changing a couple of bytes). I wasn’t able to determine the correct firmware checksum encoding after a few hours though so I shelved that idea too.

I don’t have a variable speed pump to experiment with so I’m not sure what’s involved with that.

JDogg016 commented 1 year ago

Hmmm.... that seems pretty hopeless to me from a compatibility standpoint.

j9brown commented 1 year ago

I wouldn’t say it’s hopeless. We know exactly what the problem is and we have several viable solutions. It’s just a matter of investing some effort to implement one of them to completion.

JDogg016 commented 1 year ago

I appreciate that and I would prefer to get the equipment running on HA with some simple automations (as opposed to buying new pool automation hardware).

I saw on another github thread about executing HA service calls that caused entities to be exposed.

https://github.com/zwave-js/node-zwave-js/issues/5335

The challenge is I would not know the "device_id" unless it as simple as '7' which is what is stated in the HA Device Info tab.

philh30 commented 1 year ago

Another compatibility issue for this device... When a variable speed pump is connected, it can be controlled using endpoints 0x10 through 0x13. Each endpoint represents one of the VSP speeds, and they accept Binary Switch commands with only one switch ever on at a time.

Another endpoint (0x27) is used when the P5043ME expansion module is connected, taking Binary Switch commands to toggle Pool/Spa mode. I don't have a P5043ME so can't confirm, but this is what the history of drivers for the PE653 shows.

The device doesn't advertise these endpoints at all. I'd like to be able to force discovery of these endpoints through the config file, but the only way I've gotten them to work is by manually adding them to the .jsonl files.

I'm running fw3.4 on my PE653, but my understanding is that the VSP endpoints function the same for the other firmware versions. The P5043ME expansion module requires fw3.4 to function, so endpoint 0x27 wouldn't apply to earlier versions.

AlCalzone commented 1 year ago

@philh30 I don't remember right now if simply adding the CCs via compat flags to individual endpoints works for Multi Channel CC v1 - just like some are removed in the current file. Have you tried that?

philh30 commented 1 year ago

@AlCalzone Below are the compat flags I've tried, mostly all at once though there were a few intermediate steps before I had all of these in my file. I found #5540 that seemed similar with a missing endpoint and that led me down the path of editing the .jsonl files. I'm happy to experiment if you'd like me to try anything.

    "endpoints": {
        "0": {
            "label": "Root",
            "associations": {
                "1": {
                    "label": "Lifeline",
                    "maxNodes": 5,
                    "isLifeline": true
                }
            }
        },
        "1": {
            "label": "Circuit 1"
        },
        "2": {
            "label": "Circuit 2"
        },
        "3": {
            "label": "Circuit 3"
        },
        "4": {
            "label": "Circuit 4"
        },
        "5": {
            "label": "Circuit 5"
        },
        "16": {
            "label": "VSP 1"
        },
        "17": {
            "label": "VSP 2"
        },
        "18": {
            "label": "VSP 3"
        },
        "19": {
            "label": "VSP 4"
        }
    },
    "preserveEndpoints": [1, 2, 3, 4, 5, 16, 17, 18, 19],
    "compat": [
        {
            // Fixes #4588: Firmware v3.4 has numerous bugs related to multi-endpoint support.
            // Firmware v3.3 and v3.1 do not appear to have the same issues.
            "$if": "firmwareVersion === 3.4",
            "commandClasses": {
                "add": {
                    // Binary Switch
                    "0x25": {
                        "isSupported": true,
                        "endpoints": [1, 2, 3, 4, 5, 16, 17, 18, 19],
                        "version": 1
                    },
                    // Force use of MultiChannelCC v1.
                    "0x60": {
                        "isSupported": true,
                        "endpoints": [1, 2, 3, 4, 5, 16, 17, 18, 19],
                        "version": 1
                    }
                },
                // The firmware handles requests on some endpoints incorrectly, often reporting garbage
                // that confuses discovery or inhibits operation. Remove all of these broken CCs.
                "remove": {
                    // BasicCC: All endpoints control the state of Switch 1 so only keep the root endpoint
                    // to reduce clutter and to handle received BASIC_SET events.
                    "0x20": {
                        "endpoints": [1, 2, 3, 4, 5, 16, 17, 18, 19]
                    },
                    // ManufacturerSpecificCC: Endpoint 1 erroneously reports an incorrect manufacturer
                    // and product ID, unlike on the root endpoint.
                    "0x72": {
                        "endpoints": [1]
                    },
                    // ClockCC: Endpoint 1 erroneously reports a time with an invalid minute field,
                    // unlike on the root endpoint.
                    "0x81": {
                        "endpoints": [1]
                    },
                    // AssociationCC: Endpoint 1 erroneously reports that it supports 133 associated nodes
                    // but association commands don't work at all, unlike on the root endpoint.
                    "0x85": {
                        "endpoints": [1]
                    },
                    // VersionCC: Endpoint 1 reports an unknown version, unlike on the root endpoint.
                    "0x86": {
                        "endpoints": [1]
                    }
                }
            },
            // The device sometimes sends BASIC_SET to the lifeline association when the state of Switch 1
            // changes but the value is always 0 so treat it as an event.
            "treatBasicSetAsEvent": true
        }
    ]
AlCalzone commented 1 year ago

Hm, ok that would be what I suggested. Guess that needs more work from my side then.