Incorrect Cluster Name On Device Response

[danieledwardgeorgehitchcock]

manuSpecificAssaDoorLock and manuSpecificUbisysDeviceSetup share the same Cluster ID (64512 / 0xFC00).

When I send a read command as so: topic: zigbee2mqtt/0x000d6f0011141333/1/set data: {"read":{"attributes":["wrongCodeAttempts"],"cluster":"manuSpecificAssaDoorLock","options":{}}}

I receive the following response: Zigbee2MQTT:debug 2023-12-14 11:17:01: Received Zigbee message from '0x000d6f0011141333', type 'readResponse', cluster 'manuSpecificUbisysDeviceSetup', data '{"19":5}' from endpoint 1 with groupID 0

When I try and use manuSpecificAssaDoorLock in a fromZigbee converter, the process flow isn't triggered however, when using manuSpecificUbisysDeviceSetup it is. I have also tried replacing the cluster names with the ID 64512 however, this doesn't seem to work either.

Current working external converter (issue in c4_lock_attribute):

const fz = require('zigbee-herdsman-converters/converters/fromZigbee');
const tz = require('zigbee-herdsman-converters/converters/toZigbee');
const exposes = require('zigbee-herdsman-converters/lib/exposes');
const reporting = require('zigbee-herdsman-converters/lib/reporting');
const e = exposes.presets;

const fzLocal = {
    c4_lock_operation_event: {
        cluster: 'genAlarms',
        type: ['commandAlarm'],
        convert: async (model, msg, publish, options, meta) => {
            let result = {};
            if (msg.data.clusterid == 64512) {
                const alarmcode = msg.data.alarmcode;
                const lookup = {
                    9: {action: 'error_jammed', state: 'UNLOCKED', lock_state: 'not_fully_locked'},
                    21: {action: 'manual_lock', state: 'LOCKED', lock_state: 'locked'},
                    22: {action: 'manual_unlock', state: 'UNLOCKED', lock_state: 'unlocked'},
                    24: {action: 'lock', state: 'LOCKED', lock_state: 'locked'},
                    25: {action: 'unlock', state: 'UNLOCKED', lock_state: 'unlocked'},
                    27: {action: 'auto_lock', state: 'LOCKED', lock_state: 'locked'},
                };
                if (!lookup[alarmcode]) {
                    result.action = 'unknown';
                    meta.logger.warn(`zigbee-herdsman-converters:Yale Lock: Unrecognized Operation Event (${alarmcode})`);
                    // We need to read the lock state as the alarm code is unknown
                    try {
                        await msg.endpoint.read('closuresDoorLock', ['lockState']);
                    } catch (error) {
                        meta.logger.warn(`zigbee-herdsman-converters:Yale Lock: failed to read lock state`);
                    }

                } else {
                    result = lookup[alarmcode];
                }
            } else {
                meta.logger.warn(`zigbee-herdsman-converters:Yale Lock: received unknown alarm state: ${JSON.stringify(msg.data)}`);
            }
            // We need to read the lock attributes as these are not reported by the device
            try {
                await msg.endpoint.read('manuSpecificAssaDoorLock', ['batteryLevel']);
            } catch (error) {
                meta.logger.warn(`zigbee-herdsman-converters:Yale Lock: failed to read lock attributes`);
            }
            return result;
        },
    },
    c4_lock_attribute: {
        cluster: 'manuSpecificUbisysDeviceSetup',
        type: ['readResponse'],
        convert: async (model, msg, publish, options, meta) => {
            const data = msg.data
            let result = {};
            if (data["18"]) {
                const lookup = {
                    0: "off",
                    30: "30seconds",
                    60: "60seconds",
                    120: "2minutes",
                    180: "3minutes",
                };
                result.auto_lock_time = lookup[data["18"]];
            }
            if (data["19"]) {
                result.wrong_code_attempts = data["19"];
            }
            if (data["20"]) {
                result.shutdown_time = data["20"];
            }
            if (data["21"]) {
                result.battery_level = data["21"];
            }
            if (data["22"]) {
                result.inside_escutcheon_led = data["22"] == 1 ? true : false;
            }
            if (data["23"]) {
                const lookup = {
                    1: "silent",
                    2: "low",
                    3: "high",
                };
                result.volume = lookup[data["23"]];
            }
            if (data["24"]) {
                const lookup = {
                    0: "normal",
                    1: "vacation",
                    2: "privacy",
                };    
                result.lock_mode = lookup[data["24"]];
            }
            return result;
        },
    },
};

const definition = {
    fingerprint: [{
        type: 'EndDevice',
        manufacturerName: 'Yale',
        manufacturerID: 43690,
        powerSource: 'Battery',
        endpoints: [
            {ID: 1, profileID: 260, deviceID: 10, inputClusters: [0, 9, 10, 257, 64512, 1], outputClusters: []},
            {ID: 196, profileID: 260, deviceID: 10, inputClusters: [1], outputClusters: []},
        ]},
    ],
    model: 'ZYA-C4-MOD-S',
    vendor: 'Yale',
    description: 'Control4 module for Yale KeyFree/Keyless/Doorman/Assure/nexTouch locks',
    fromZigbee: [fz.lock, fzLocal.c4_lock_operation_event, fzLocal.c4_lock_attribute],
    toZigbee: [tz.lock],
    exposes: [e.lock(), e.lock_action()],
    configure: async (device, coordinatorEndpoint, logger) => {
        const endpoint = device.getEndpoint(1);
        await endpoint.read('closuresDoorLock', ['lockState']);
        await endpoint.read('manuSpecificAssaDoorLock', ['autoLockTime', 'wrongCodeAttempts', 'shutdownTime', 'batteryLevel', 'volume']);
        await reporting.bind(endpoint, coordinatorEndpoint, ['manuSpecificAssaDoorLock']);
    },
};

module.exports = definition;

[danieledwardgeorgehitchcock]

This also seems to interfere with device command responses too:

Zigbee2MQTT:debug 2023-12-14 12:50:37: Received MQTT message on 'zigbee2mqtt/0x000d6f0011141333/1/set' with data '{"command":{"cluster":"manuSpecificAssaDoorLock","command":18,"payload":{}}}'
Zigbee2MQTT:debug 2023-12-14 12:50:37: Publishing 'set' 'command' to '0x000d6f0011141333'
2023-12-14T12:50:37.464Z zigbee-herdsman:controller:endpoint Command 0x000d6f0011141333/1 manuSpecificAssaDoorLock.getBatteryLevel({}, {"sendWhen":"immediate","timeout":10000,"disableResponse":false,"disableRecovery":false,"disableDefaultResponse":false,"direction":0,"srcEndpoint":null,"reservedBits":0,"manufacturerCode":null,"transactionSequenceNumber":null,"writeUndiv":false})
2023-12-14T12:50:39.255Z zigbee-herdsman:adapter:zStack:adapter Response timeout (0x000d6f0011141333:27176,1)
2023-12-14T12:50:39.256Z zigbee-herdsman:adapter:zStack:adapter sendZclFrameToEndpointInternal 0x000d6f0011141333:27176/1 (0,0,1)
2023-12-14T12:50:39.257Z zigbee-herdsman:adapter:zStack:znp:SREQ --> AF - dataRequest - {"dstaddr":27176,"destendpoint":1,"srcendpoint":1,"clusterid":64512,"transid":61,"options":0,"radius":30,"len":3,"data":{"type":"Buffer","data":[1,28,18]}}
2023-12-14T12:50:39.257Z zigbee-herdsman:adapter:zStack:unpi:writer --> frame [254,13,36,1,40,106,1,1,0,252,61,0,30,3,1,28,18,185]
2023-12-14T12:50:39.258Z zigbee-herdsman:controller:endpoint Command 0x000d6f0011141333/1 manuSpecificAssaDoorLock.getBatteryLevel({}, {"sendWhen":"immediate","timeout":10000,"disableResponse":false,"disableRecovery":false,"disableDefaultResponse":false,"direction":0,"srcEndpoint":null,"reservedBits":0,"manufacturerCode":null,"transactionSequenceNumber":null,"writeUndiv":false}) failed (Timeout - 27176 - 1 - 27 - 64512 - 11 after 10000ms)
Zigbee2MQTT:error 2023-12-14 12:50:39: Publish 'set' 'command' to '0x000d6f0011141333' failed: 'Error: Command 0x000d6f0011141333/1 manuSpecificAssaDoorLock.getBatteryLevel({}, {"sendWhen":"immediate","timeout":10000,"disableResponse":false,"disableRecovery":false,"disableDefaultResponse":false,"direction":0,"srcEndpoint":null,"reservedBits":0,"manufacturerCode":null,"transactionSequenceNumber":null,"writeUndiv":false}) failed (Timeout - 27176 - 1 - 27 - 64512 - 11 after 10000ms)'
Zigbee2MQTT:debug 2023-12-14 12:50:39: Error: Command 0x000d6f0011141333/1 manuSpecificAssaDoorLock.getBatteryLevel({}, {"sendWhen":"immediate","timeout":10000,"disableResponse":false,"disableRecovery":false,"disableDefaultResponse":false,"direction":0,"srcEndpoint":null,"reservedBits":0,"manufacturerCode":null,"transactionSequenceNumber":null,"writeUndiv":false}) failed (Timeout - 27176 - 1 - 27 - 64512 - 11 after 10000ms)
    at Timeout._onTimeout (/app/node_modules/zigbee-herdsman/src/utils/waitress.ts:64:35)
    at listOnTimeout (node:internal/timers:569:17)
    at processTimers (node:internal/timers:512:7)
2023-12-14T12:50:39.267Z zigbee-herdsman:adapter:zStack:unpi:parser <-- [254,1,100,1,0,100]
2023-12-14T12:50:39.267Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext [254,1,100,1,0,100]
2023-12-14T12:50:39.272Z zigbee-herdsman:adapter:zStack:unpi:parser --> parsed 1 - 3 - 4 - 1 - [0] - 100
2023-12-14T12:50:39.273Z zigbee-herdsman:adapter:zStack:znp:SRSP <-- AF - dataRequest - {"status":0}
2023-12-14T12:50:39.273Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext []
2023-12-14T12:50:41.503Z zigbee-herdsman:adapter:zStack:unpi:parser <-- [254,3,68,128,0,1,61,251]
2023-12-14T12:50:41.503Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext [254,3,68,128,0,1,61,251]
2023-12-14T12:50:41.503Z zigbee-herdsman:adapter:zStack:unpi:parser --> parsed 3 - 2 - 4 - 128 - [0,1,61] - 251
2023-12-14T12:50:41.503Z zigbee-herdsman:adapter:zStack:znp:AREQ <-- AF - dataConfirm - {"status":0,"endpoint":1,"transid":61}
2023-12-14T12:50:41.504Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext []
2023-12-14T12:50:41.537Z zigbee-herdsman:adapter:zStack:unpi:parser <-- [254,24,68,129,0,0,0,252,40,106,1,1,0,69,0,17,127,96,0,0,4,9,28,18,100,40,106,29,16]
2023-12-14T12:50:41.537Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext [254,24,68,129,0,0,0,252,40,106,1,1,0,69,0,17,127,96,0,0,4,9,28,18,100,40,106,29,16]
2023-12-14T12:50:41.537Z zigbee-herdsman:adapter:zStack:unpi:parser --> parsed 24 - 2 - 4 - 129 - [0,0,0,252,40,106,1,1,0,69,0,17,127,96,0,0,4,9,28,18,100,40,106,29] - 16
2023-12-14T12:50:41.538Z zigbee-herdsman:adapter:zStack:znp:AREQ <-- AF - incomingMsg - {"groupid":0,"clusterid":64512,"srcaddr":27176,"srcendpoint":1,"dstendpoint":1,"wasbroadcast":0,"linkquality":69,"securityuse":0,"timestamp":6323985,"transseqnumber":0,"len":4,"data":{"type":"Buffer","data":[9,28,18,100]}}
2023-12-14T12:50:41.539Z zigbee-herdsman:controller:log Received 'raw' data '{"clusterID":64512,"data":{"type":"Buffer","data":[9,28,18,100]},"address":27176,"endpoint":1,"linkquality":69,"groupID":0,"wasBroadcast":false,"destinationEndpoint":1}'
Zigbee2MQTT:debug 2023-12-14 12:50:41: Received Zigbee message from '0x000d6f0011141333', type 'raw', cluster 'manuSpecificUbisysDeviceSetup', data '{"data":[9,28,18,100],"type":"Buffer"}' from endpoint 1 with groupID 0
Zigbee2MQTT:debug 2023-12-14 12:50:41: No converter available for 'ZYA-C4-MOD-S' with cluster 'manuSpecificUbisysDeviceSetup' and type 'raw' and data '{"data":[9,28,18,100],"type":"Buffer"}'
2023-12-14T12:50:41.546Z zigbee-herdsman:adapter:zStack:unpi:parser --- parseNext []

[danieledwardgeorgehitchcock]

Might be the same issue as #789

[Koenkk]

This is because the device doesn't respond with a manufacturerCode, therefore we don't know with what cluster to match. Does adding a manufacturerCode to manuSpecificAssaDoorLock in cluster.ts fix this?

[danieledwardgeorgehitchcock]

This doesn't do anything - I have sniffed the traffic and checked the logs, and the manufacturerCode isn't held in the response.

Could we not do some kind of device->cluster mapping to ensure the correct clusters are being used?

[Koenkk]

Ideally we should remove all the manufacturer specific cluster from cluster.ts and inject them for certain device in zigbee-herdsman-converters, but it will take some time before we get there (I believe @sjorge will also be interested in this)

[danieledwardgeorgehitchcock]

Any prototype code / hints I can play about with in my external converter?

I'm guessing there'd also have to be quite a significant change to the way Z2M processes traffic (especially responses)..

Happy to help out where I can!

[Koenkk]

Currently this is not possible from the ext converter, as a temp workaround for your problem, can we merge the clusters?

[danieledwardgeorgehitchcock]

I don't think we can - the command / attribute / response ID's clash

[danieledwardgeorgehitchcock]

Maybe a better solution would be (if you are happy to) merge the current PR I have open for the herdsman converter change, then squirrel away in the background on a more robust fix..?

[sjorge]

I believe @sjorge will also be interested in this

I hope this will become easier once we're using modernExtends everywhere. Were we can do something similar to merge in zcl definitions. (This obviously needs to happen before the modernExtends are evaluated.)

But I see no easy way of doing it, aside from have a evaluation step for each device were we merge a device specific cluster definition over a standard ZCL one.

[Koenkk]

@danieledwardgeorgehitchcock done!