matterbridge device recognition from z2m

[jpadie]

reopening issue 17. issue remains unresolved.

recap:

2 gang zigbee dimmer module is recognised perfectly by Z2M. on/off and dimmer functionality is available.

matterbridge-z2m ingests the device as a composed light and assigns a device type of 0x0013.

the composed device is then picked up by alexa as a composed pair of lights however no dimmer functionality is exposed.

[Luligu]

Can you send me please the file bridge-devices.json that you find in Matterbridge/matterbridge-zigbee2mqtt directory. I need also that you tell me the exact friendly name of the device that causes the issue. Thanks

[jpadie]

Please see the original issue that you marked as closed.  There is no substantive change in behaviour and you already have sanitised version of the json files.On 7 Jun 2024 21:43, Luligu @.***> wrote: Can you send me please the file bridge-devices.json that you find in Matterbridge/matterbridge-zigbee2mqtt directory. I need also that you tell me the exact friendly name of the device that causes the issue. Thanks

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

[Luligu]

Do you have a single dimmer? Is it recognizable by alexa with brightness and is it working correctly? I need to figure out if the issue with Alexa is general or limited to devices with multiple endpoints. Furthermore there is evidence that Alexa has issues showing more than a device type like the typical occupancy sensor with also a light sensor... Can you make a screenshot of the log in the part where your device is added to the bridge? I need specifically the addChildDeviceType...

[jpadie]

I don't have a single gang dimmer module to test with.

I think the issue is in how you handle composed devices.

for example I have just created some nodes in matter js to test whether this is an alexa bug. the structure I used was

root:
  aggregator
      composedLights
         dimmable light 1
         dimmable light 2
      composedSensor
         humidity sensor
         temperature sensor

all devices are picked up properly by Alexa. even the humidity sensor.

the code I used to create these devices is as follows

import "@project-chip/matter-node.js";
import { requireMinNodeVersion } from "@project-chip/matter-node.js/util";
import { VendorId } from "@project-chip/matter.js/datatype";
import { logEndpoint } from "@project-chip/matter.js/device";
import { Endpoint, EndpointServer } from "@project-chip/matter.js/endpoint";
import { AggregatorEndpoint } from "@project-chip/matter.js/endpoint/definitions";
import { Environment, StorageService } from "@project-chip/matter.js/environment";
import { ServerNode } from "@project-chip/matter.js/node";
import { Time } from "@project-chip/matter.js/time";
import { DimmableLightDevice } from "@project-chip/matter.js/endpoint/definitions";
import { TemperatureSensorDevice } from "@project-chip/matter.js/endpoint/definitions";
import { HumiditySensorDevice } from "@project-chip/matter.js/endpoint/definitions";
import { BridgedNodeEndpoint } from "@project-chip/matter.js/endpoint/definitions";
requireMinNodeVersion(16);

const {
    deviceName,
    vendorName,
    passcode,
    discriminator,
    vendorId,
    productName,
    productId,
    port,
    uniqueId
} = await getConfiguration();

const server = await ServerNode.create({
    id: uniqueId,
    network: {
        port,
    },
    commissioning: {
        passcode,
        discriminator,
    },
    productDescription: {
        name: deviceName,
        deviceType: AggregatorEndpoint.deviceType,
    },
    basicInformation: {
        vendorName,
        vendorId: VendorId(vendorId),
        nodeLabel: productName,
        productName,
        productLabel: productName,
        productId,
        serialNumber: `matterjs-${uniqueId}`,
        uniqueId,
    },
});

const aggregator = new Endpoint(AggregatorEndpoint, { id: "aggregator" });

await server.add(aggregator);

const composedLights = new Endpoint(BridgedNodeEndpoint,
    {
        id: "composedLights-1",
        bridgedDeviceBasicInformation: {
            nodeLabel: "composedLights-1",
            productName: "composedLights-1",
            productLabel: "composedLights-1",
            serialNumber: "composedLights-1",
            reachable: true
        }
    }
);

const composedSensor = new Endpoint(BridgedNodeEndpoint,
    {
        id: "composedSensor-1",
        bridgedDeviceBasicInformation: {
            nodeLabel: "composedSensor-1",
            productName: "composedSensor-1",
            productLabel: "composedSensor-1",
            serialNumber: "composedSensor-1",
            reachable: true
        }
    }
);

await aggregator.add(composedSensor);
await aggregator.add(composedLights);

const light1 = new Endpoint(DimmableLightDevice, { id: "dimmable light 1" });
const light2 = new Endpoint(DimmableLightDevice, { id: "dimmable light 2" });

await composedLights.add(light1);
await composedLights.add(light2);

const tempSensor = new Endpoint(TemperatureSensorDevice, { id: "tempSensor-1" });
const humiditySensor = new Endpoint(HumiditySensorDevice, { id: "humiditySensor-1" });

await composedSensor.add(tempSensor);
await composedSensor.add(humiditySensor);

await server.bringOnline();

//create some dummy values
tempSensor.set({
    temperatureMeasurement: {
        measuredValue: 20.5 * 100
    }
});

humiditySensor.set({
    relativeHumidityMeasurement: {
        measuredValue: 53.5 * 100
    }
});

logEndpoint(EndpointServer.forEndpoint(server));

async function getConfiguration() {
    const environment = Environment.default;
    const storageService = environment.get(StorageService);
    const deviceStorage = (await storageService.open("device")).createContext("data");

    const deviceName = "test matter aggregator";
    const vendorName = "JPADIE";
    const passcode = environment.vars.number("passcode") ?? (await deviceStorage.get("passcode", 20202021));
    const discriminator = environment.vars.number("discriminator") ?? (await deviceStorage.get("discriminator", 3840));
    // product name / id and vendor id should match what is in the device certificate
    const vendorId = environment.vars.number("vendorid") ?? (await deviceStorage.get("vendorid", 0xfff1));
    const productName = "test matter aggregator";
    const productId = environment.vars.number("productid") ?? (await deviceStorage.get("productid", 0x8000));

    const port = 5540;

    const uniqueId = environment.vars.string("uniqueid") ?? (await deviceStorage.get("uniqueid", Time.nowMs().toString()));

    // Persist basic data to keep them also on restart
    await deviceStorage.set({
        passcode,
        discriminator,
        vendorid: vendorId,
        productid: productId,
        uniqueid: uniqueId,
    });

    return {
        deviceName,
        vendorName,
        passcode,
        discriminator,
        vendorId,
        productName,
        productId,
        port,
        uniqueId,
    };
};

the endpoint structure that is generated and logged by my test is attached. endpoint log.txt

I cannot see any log entry in /var/log/journal that contains the text 'addChildDeviceType'. Where would you expect it to be?

[jpadie]

in case useful the z2m definitions are these:

Sonoff SNZB-02

const definition = {
    zigbeeModel: ['TH01'],
    model: 'TH01',
    vendor: 'eWeLink',
    description: 'Automatically generated definition',
    extend: [identify(), temperature(), humidity(), battery()],
    meta: {},
};

Moes 2 gang dimmer (recognised as TS0601_dimmer_2)

const definition = {
    zigbeeModel: ['TS0601'],
    model: 'TS0601',
    vendor: '_TZE200_e3oitdyu',
    description: 'Automatically generated definition',
    extend: [],
    meta: {},
};

that last one is not very useful. attached is what the device exposes so far as z2m is concerned.

[Luligu]

Hi, thanks you, that's very interesting. I guess I understood, could you be so kind to add another composed device to your test with occupancy sensor and light sensor. That is not picked up too by alexa in the test made by Tammer (all other controllers show it correctly of course).

[jpadie]

yes - no problem.
i've created a third composed node with those two devices.

both are picked up by Alexa. the motion sensor shows detected motion (which is the state preset in the virtual device). the light sensor does not show a value. at the moment, I don't see why unless the implementation in Alexa does not use the same units (contrary to the spec which allows 0 to 0xfffe).

adding the light sensor as a direct child node of the aggregator does not appear to change the behaviour.

[Luligu]

Hi, thank you for your test. Besides the value of the light measurement, is the device type itself exposed on Alexa? I mean do you see a light sensor with a motion sensor for the first device and a light sensor for the second device?

[jpadie]

I see

no that is not the way that Alexa appears to present this. Instead you see three devices. One is the composed node itself. the second and third are the motion and the light sensor. Deleting the composed device from Alexa deletes the constituent devices as well. Which is useful sometimes.

However it's not a great UX. For virtual devices I suspect it will always be better to avoid composed devices and present each component device as direct children of the aggregator.

see the pics below

[Luligu]

It seems that Alexa doesn't support the light sensor device type. That confirms the test already done with Tammer. If you want to check your idea about the child endpoints create a virtual device composed with temperature sensor and humidity sensor. Alexa should have them hopefully and check if they show up correctly. If you put them on the same endpoint we already know that alexa doesn't show the second device type.

[jpadie]

I would say that there may be a bug in the support, but not that it is not supported. After all Alexa correctly recognises the device, assigns the correct icon to it and displays the device in its manifest.

whereas if you build a "combined device" with two clusters on one endpoint where the spec does not explicitly permit that then there is nothing in the spec that says what controllers should do; so it is not that Alexa is not compliant there, but that the "combined device" is not something that is recognised. the usual reported behaviour of controllers is to acknowledge only the first "endpoint".

FWIW, given that this project is innately virtualising devices, I'd think that it is best to pass through each device from wherever the bridge originates (e.g. z2m) as a direct child of the aggregator.

[Luligu]

Sorry but I cannot follow you... the matter specs are absolutely simple. We are finishing to figure out all the issues with Alexa and Matter. It doesn't show level and color for a light even with the shelly plugin. We still have another test to do and I will tell you the results.

[jpadie]

can you point out where in the matter specs it specifies how a controller should handle a custom device with two or more clusters that are not known device types? e.g. a temp + humidity sensor.

I don't know what you mean by saying that Alexa does not show level and colour for lights? I showed you just yesterday that dimmable lights provisioned via matter.js are correctly shown with Alexa and provided sample code. do you need me to show you a test with a colour capable light as well?

[Luligu]

Hi,

can you point out where in the matter specs it specifies how a controller should handle a custom device with two or more clusters that are not known device types? e.g. a temp + humidity sensor.

All this sentence has no meaning in term of matter terminology.

I don't know what you mean by saying that Alexa does not show level and colour for lights? I showed you just yesterday that dimmable lights provisioned via matter.js are correctly shown with Alexa and provided sample code. do you need me to show you a test with a colour capable light as well?

I meant in a composed device: zigbee2mqtt plugin, since your device has 2 dimmer, exposes it creating 2 child endpoint (level not shown by Alexa), shelly plugin always use child endpoint even for a single dimmer (also there the level is not shown by Alexa).

Conclusion Alexa is not able to show a light with dimmer in a child endpoint and there is nothing I can do about it. (You can create yourself a composed device with 2 child endpoint of dimmer type and you will get the same results as in zigbee2mqtt and shelly plugin). All other controllers (Apple, Google, SmartThings, HA and eWeLink) have no issue in this configuration.

Alexa shows correctly lights with onOff, level and color only if they are not in a child endpoint (if you add the dynamic example plugin in Matterbridge it will show in Alexa a light with onOff level and color without any difficulty).

Hopefully one day Alexa will upgrade the matter support but right now the situation is far from optimal.

You can close the issue.

[jpadie]

This is not correct. I showed you two days ago that two dimmable lights in a composed device attached to an aggregator functioned entirely correctly with Alexa.

[jpadie]

why is this marked as completed? have you now fixed your codebase to support this class of device? please indicate the release/changelog that fixed the issue.

[Luligu]

Conclusion Alexa is not able to show a light with dimmer in a child endpoint and there is nothing I can do about it.

Alexa shows correctly lights with onOff, level and color only if they are not in a child endpoint (if you add the dynamic example plugin in Matterbridge it will show in Alexa a light with onOff level and color without any difficulty).

Hopefully one day Alexa will upgrade the matter support but right now the situation is far from optimal.

[jpadie]

but quite evidently you are not correct as the code that I sent you proved conclusively that a dimmable light device is properly recognised by alexa as a child device of a bridgedendpoint.

do I take it that you do not believe that the code I wrote for you works? or do you suggest that I am lying in suggesting that it works? would you like visual proof?

[Luligu]

Listen I don't know how the bug in Alexa shows up... The possibilities are a lot... Maybe the number of endpoints... Maybe the presence of other device types on the endpoint... in real implementation there is also a power source device type 0x0011... Maybe the simple order of the endpoint components... You can try in your simple code to recreate the exact situation of the real implementation and check all variables...

[jpadie]

this is what I have done and it works. please post the results of what happens in your environment when you run the code that I provided.

[Luligu]

Yeah, we tested matter.js and is working (like you said). We tested Matterbridge api and they are working (you saw the screenshot). Now you can just try in your code to add PowerSource device type and maybe also a label cluster to simulate the real implementation and understand where Alexa stops showing correctly the child endpoints. These two are the first differences that come to my mind about the endpoint composition difference between your simple code and the real implementation. Let's say that my intuition about the PowerSource device type is correct then what do we do? We remove something for all controllers cause Alexa has problems? When you understand where is the problem, simulating the real implementation in your code, we can eventually think how to change something for Alexa without modifying what is working 100% correctly with all other controllers.

[jpadie]

where do you log the endpoint structure that matter.js generates for you? I cannot see these in the journal (on rPi) and I note that you do not use logEndpoint in your codebase. If I have the endpoint structure that your code generates then I can emulate it.

in the meantime in some journal logs that I downloaded earlier today I am seeing these errata (to note the messages about no level control):

Jun 11 10:25:03 raspberrypi matterbridge[2851]: [10:25:03.753] [0x00124b002a5095ef] MQTT message for device 0x00124b002a5095ef payload: { battery: 100, humidity: 66.13, last_seen: '2024-06-11T10:25:03+01:00', linkquality: 92, temperature: 13.42, voltage: 3100 } 
Jun 11 10:25:03 raspberrypi matterbridge[2851]: [10:25:03.797] [0x00124b002a5095ef] Update endpoint 3 attribute RelativeHumidityMeasurement-measuredValue from 0 to 6613 
Jun 11 10:25:03 raspberrypi matterbridge[2851]: [10:25:03.802] [0x00124b002a5095ef] Update endpoint 3 attribute TemperatureMeasurement-measuredValue from 0 to 1342 
Jun 11 10:25:03 raspberrypi matterbridge[2851]: [10:25:03.813] [0x00124b002a5095ef] Update endpoint 3 attribute PowerSource-batVoltage from 1500 to 3100 
Jun 11 10:25:06 raspberrypi matterbridge[2851]: [10:25:06.282] [Matterbridge] The plugin matterbridge-zigbee2mqtt is up to date. Current version: 2.0.16, Latest version: 2.0.16 
Jun 11 10:25:20 raspberrypi matterbridge[2851]: [10:25:20.510] [0x00124b002a5095ef] MQTT message for device 0x00124b002a5095ef payload: { battery: 100, humidity: 66.34, last_seen: '2024-06-11T10:25:20+01:00', linkquality: 94, temperature: 13.42, voltage: 3100 } 
Jun 11 10:25:20 raspberrypi matterbridge[2851]: [10:25:20.521] [0x00124b002a5095ef] Update endpoint 3 attribute RelativeHumidityMeasurement-measuredValue from 6613 to 6634 
Jun 11 10:25:44 raspberrypi matterbridge[2851]: [10:25:44.605] [0x187a3efffef17704] MQTT message for device 0x187a3efffef17704 payload: { brightness_l1: 5, brightness_l2: 137, last_seen: '2024-06-11T10:25:44+01:00', linkquality: 36, min_brightness_l1: 0, min_brightness_l2: 0, state_l1: 'OFF', state_l1_l1: 'OFF', state_l2: 'OFF', state_l2_l2: 'OFF' } 
Jun 11 10:25:44 raspberrypi matterbridge[2851]: [10:25:44.639] [0x187a3efffef17704] Update endpoint 5 (l1) cluster 8-LevelControl not found: is z2m converter exposing all features? 
Jun 11 10:25:44 raspberrypi matterbridge[2851]: [10:25:44.652] [0x187a3efffef17704] Update endpoint 6 (l2) cluster 8-LevelControl not found: is z2m converter exposing all features? 
Jun 11 10:25:51 raspberrypi matterbridge[2851]: 2024-06-11 10:25:50.782 ERROR  UdpChannelNode       ENXIO: no such device or address, uv_if_indextoiid
Jun 11 10:25:51 raspberrypi matterbridge[2851]: 2024-06-11 10:25:51.191 ERROR  UdpChannelNode       ENXIO: no such device or address, uv_if_indextoiid
Jun 11 10:25:51 raspberrypi matterbridge[2851]: 2024-06-11 10:25:51.242 ERROR  UdpChannelNode       ENXIO: no such device or address, uv_if_indextoiid
Jun 11 10:25:51 raspberrypi matterbridge[2851]: 2024-06-11 10:25:51.380 ERROR  UdpChannelNode       ENXIO: no such device or address, uv_if_indextoiid
Jun 11 10:25:51 raspberrypi matterbridge[2851]: [10:25:51.978] [Matterbridge] Setting reachability to true for Matterbridge 
Jun 11 10:25:59 raspberrypi matterbridge[2851]: [10:25:59.916] [0x00124b002a5095ef] MQTT message for device 0x00124b002a5095ef payload: { battery: 100, humidity: 65.32, last_seen: '2024-06-11T10:25:57+01:00', linkquality: 92, temperature: 13.42, voltage: 3100 } 
Jun 11 10:25:59 raspberrypi matterbridge[2851]: [10:25:59.926] [0x00124b002a5095ef] Update endpoint 3 attribute RelativeHumidityMeasurement-measuredValue from 6634 to 6532 

[Luligu]

Sorry but this log is not readable. Copy paste the console log please. I saw some errors that could be interesting to check. I already answered you about log. Use the frontend Devices and you see the whole node with all info.

[jpadie]

The console log does not show past events and resets when moving browser etc. It is not useful for serious debugging of large logs.

The journal provided is the output that your code logs when started as a service.

Where else does your code save its historic logs?