pvvx / ZigbeeTLc

Custom firmware for Zigbee 3.0 IoT devices on the TLSR825x chip
Other
365 stars 20 forks source link

LYWSD03MMC configuration Error in Z2M #58

Open Bodengriller opened 7 months ago

Bodengriller commented 7 months ago

Since the Z2M update to 1.35.3-1 I get the following configuration-error for all my LYWSD03MMC:

"Failed to configure""TempHumLCD_Schlafzimmer",
"attempt 2 (Error":Read 0xa4c138a130e2c2f3/1 hvacUserInterfaceCfg([
   16
],
{
   "timeout":10000,
   "disableResponse":false,
   "disableRecovery":false,
   "disableDefaultResponse":true,
   "direction":0,
   "srcEndpoint":null,
   "reservedBits":0,
   "manufacturerCode":null,
   "transactionSequenceNumber":null,
   "writeUndiv":false
}") 
failed (Status""UNSUPPORTED_ATTRIBUTE"") 
at Endpoint.checkStatus (/app/node_modules/zigbee-herdsman/src/controller/model/endpoint.ts":"319":"28) 
at Endpoint.read (/app/node_modules/zigbee-herdsman/src/controller/model/endpoint.ts":"469":"22) 
at setupAttributes (/app/node_modules/zigbee-herdsman-converters/src/lib/modernExtend.ts":"63":"13) 
at configure (/app/node_modules/zigbee-herdsman-converters/src/lib/modernExtend.ts":"84":"17) 
at Object.configure (/app/node_modules/zigbee-herdsman-converters/src/index.ts":"141":"25) 
at Configure.configure (/app/lib/extension/configure.ts":"120":"13) 
at EventEmitter.wrappedCallback (/app/lib/eventBus.ts":"174":17))

I have already tested this external converter - same problem

There are no problems with this external converter, but all settings are missing (smiley off etc.):

const fz = require('zigbee-herdsman-converters/converters/fromZigbee');
const exposes = require('zigbee-herdsman-converters/lib/exposes');
const reporting = require('zigbee-herdsman-converters/lib/reporting');
const ota = require('zigbee-herdsman-converters/lib/ota');
const e = exposes.presets;

const definition = {
    zigbeeModel: ['LYWSD03MMC'],
    model: 'LYWSD03MMC',
    vendor: 'Xiaomi',
    description: 'Temperature & humidity sensor',
    fromZigbee: [fz.temperature, fz.humidity, fz.battery],
    toZigbee: [],
    configure: async (device, coordinatorEndpoint, logger) => {
        const endpoint = device.getEndpoint(1);
        const bindClusters = ['msTemperatureMeasurement', 'msRelativeHumidity', 'genPowerCfg'];
        await reporting.bind(endpoint, coordinatorEndpoint, bindClusters);
        await reporting.temperature(endpoint, {min: 10, max: 300, change: 10});
        await reporting.humidity(endpoint, {min: 10, max: 300, change: 50});
        await reporting.batteryVoltage(endpoint);
        await reporting.batteryPercentageRemaining(endpoint);
    },
    exposes: [e.temperature(), e.humidity(), e.battery()],
    ota: ota.zigbeeOTA,
};
module.exports = definition;

There is only this warning: OTA: Images currently unavailable for device 'LYWSD03MMC', hardwareVersion '16', manufacturerName Xiaomi, {"fieldControl":0,"manufacturerCode":4417,"imageType":522,"fileVersion":18427905}'

I'm sure that all the settings worked until recently even without an external converter. However, I can't find any issue or report on this problem. Am I doing something wrong or is it a known problem? Is there a way to solve this?

Installed Version on devices: 1141-020a-01193001-Z03MMC.zigbee

(until yesterday all devices were on v0.1.1.8 - as the problem was recognized)

TreborR2 commented 7 months ago

So I set it up according to your instructions (I also tried force false) and it gives me this error, see picture. I tried searching on Google and couldn't find anything interesting. Anyone have any ideas or advice? image image

pvvx commented 7 months ago

@pvvx maybe it's a nightmare, but in ZHA half of my devices aren't working ;)

With Zigbee this is true everywhere. Therefore, for a long time I did not want to make firmware for thermometers on Zigbee.

Bodengriller commented 7 months ago

@TreborR2 I notice that the device with V0.1.1.9 is recognised as a "Custom Device (DIY)". This indicates that the z2m-devbis-converter has been loaded here.

Try the following: Additionally save this external converter in your z2m folder (name it "lywsd03mmc.js") Then add this converter to your Z2M config

image

After that reboot your Z2M. Your "Teplomer-Koupelna" should now show "Xiaomi_TEST" as vendor. if not so -> reconfigure this device (yellow button) If everything up to this point is correct, Z2M is already using a converter that is not completely wrong for the pvvx-FW (but has only minimal functionality)

Than try to do a OTA-Update

When everything is fine, you can delete this converter and remove it from Z2M config

MortenVinding commented 7 months ago

@pvvx maybe it's a nightmare, but in ZHA half of my devices aren't working ;)

With Zigbee this is true everywhere. Therefore, for a long time I did not want to make firmware for thermometers on Zigbee.

So put the firmware file in the dir, point at it in a json file the same place, and add the json file in Z2M GUI…?

Just read up on ZHA, looks not much different 🤷‍♂️

MortenVinding commented 7 months ago

So I set it up according to your instructions (I also tried force false) and it gives me this error, see picture. I tried searching on Google and couldn't find anything interesting. Anyone have any ideas or advice?

I got the same when I tried to go the other way: ie. from ZigbeeTLC/pvvx firmware to devbis. I guess the to firmware images are not compatible with each other?

Guess that's why pvvx explains how to create a conversion file here: https://github.com/pvvx/ZigbeeTLc/issues/58#issuecomment-1940163912

robvanoostenrijk commented 7 months ago

Based on the LYWSD03MMC external converter for Zigbee2MQTT posted above, here is one for the CGDK2 (Qingping Temp & RH Monitor Lite)

Since the CGDK2 does not have Smiley or Comfort settings, these are removed from the convertor.

ZCGDK2 external convertor: ZCGDK2.js.txt

TreborR2 commented 7 months ago

@TreborR2 I notice that the device with V0.1.1.9 is recognised as a "Custom Device (DIY)". This indicates that the z2m-devbis-converter has been loaded here.

Try the following: Additionally save this external converter in your z2m folder (name it "lywsd03mmc.js") Then add this converter to your Z2M config image After that reboot your Z2M. Your "Teplomer-Koupelna" should now show "Xiaomi_TEST" as vendor. if not so -> reconfigure this device (yellow button) If everything up to this point is correct, Z2M is already using a converter that is not completely wrong for the pvvx-FW (but has only minimal functionality)

Than try to do a OTA-Update

When everything is fine, you can delete this converter and remove it from Z2M config

I'm only responding now because I was away. Thank you for answer. I did what you wrote and I think it shows what you think it should show? But to be honest, I don't quite understand now, what did we verify / find out with it? Even with this conversion file it gives the same error "Failed to check if update available for 'Teploměr-Koupelna' (Attempt to access memory outside buffer bounds)" when I try to OTA update the FW? Any advice? I admit that I am not completely knowledgeable about this. Thanks in advance for any further help.

image

TreborR2 commented 7 months ago

@Bodengriller What exactly was the conversion file supposed to test? Because I deleted it and removed it from the Z2M settings as well and that device is still showing me as "Xiaomi_TEST". I tried resetting the button several times and restarted the whole system.

image image

TreborR2 commented 7 months ago

I have one more question. Do you also find that the newer FW 3001-0120 drains the battery faster than the older 3001-0119? I started using the one with the newer FW later than the older FW and it shows that the one with the newer one already has 81.5% and the other one still has 100%. And the strength in the zigbee signal is not that different to be the reason??? I'm just asking? Maybe there was a weak battery or another problem?

image

Bodengriller commented 7 months ago

What exactly was the conversion file supposed to test? Because I deleted it and removed it from the Z2M settings as well and that device is still showing me as "Xiaomi_TEST". I tried resetting the button several times and restarted the whole system.

My thought was that your OTA update might fail because the device is configured with the wrong converter (Custom Device DIY is definitely the Z2M converter for the devbis firmware, which is stored at Z2M, but does not work correctly for the pvvx firmware) So that you can distinguish it better, I have changed the name in the converter to "xiaomi_test".

However, since the OTA update does not work for you even with the stripped-down, functioning converter, it was probably not due to this.

After you have deleted the converter, you must reconfigure the device (yellow button) - then the completely wrong converter from Z2M should be loaded again

I've no clue why you cannot update OTA, sorry

I have one more question. Do you also find that the newer FW 3001-0120 drains the battery faster than the older 3001-0119? I started using the one with the newer FW later than the older FW and it shows that the one with the newer one already has 81.5% and the other one still has 100%.

I can't make a judgement on this, all the updates and gimmicks have almost drained my batteries :D In any case, an update consumes significantly more power than normal operation

JayFoxRox commented 7 months ago

Do you also find that the newer FW 3001-0120 drains the battery faster than the older 3001-0119? I started using the one with the newer FW later than the older FW and it shows that the one with the newer one already has 81.5% and the other one still has 100%.

Keep in mind that the OTA can drain your battery a lot already.

Also I noticed that some of my devices used wrong report intervals, so I'd advice to occassionally reconfigure the thermometer through zigbee2mqtt (yellow reconfigure / refresh button at bottom in "About" tab of device). I'd definitely do this after pairing / reflashing.

Therefore I'm still not 100% confident that this firmware stores the configuration (such as report intervals!) correctly in flash because I also noticed different battery drain and report intervals on some of my devices - usually resolved by manually reconfiguring them. There was a similar issue with the smiley in #47. For the other settings we appear to trust the telink SDK to store/load the intervals etc.

Nephiel commented 6 months ago

I too was using LYWSD03MMC with ZigbeeTLc <= 0.1.1.9, updated to 0.1.2.0 using the OTA suggested by z2m, and ran into the UNSUPPORTED_ATTRIBUTE error.

I wanted to stay on ZigbeeTLc rather than devbis (mostly for battery life) but was unsure whether the z2m OTA had updated my devices to the latter. A major source of confusion for me was that z2m kept reporting the model as LYWSD03MMC and not as LYWSD03MMC-z, even after deleting all my affected devices from z2m and attempting to re-pair them.

FWIW, what finally worked for me was adding the custom converter from here, but twice (one as-is, and a second copy edited without the -z suffix). After that, z2m finally reported model as LYWSD03MMC-z and all affected devices re-paired successfully. Once that was done, I removed the second copy of the converter leaving only the one with the -z and restarted z2m one last time.

fhempy commented 6 months ago

@Nephiel I have the same issue, that the device with the new firmware is still recognized as the one without -z. I tried to use the external converters twice, but unfortunately even the converter without -z is used. I have done the following:

Do you have any further hints? Thanks!

pvvx commented 6 months ago

A similar problem exists in ZHA. After updating, the device name is not updated. Removing and re-pairing does not help.

image

Although the device name changes.

image

Something is not completed in the Home Assistant software with Z2M and ZHA.

But in ZHA, the device name has no effect. The device operates according to the Zigbee 3.0 standard and does not require converters or “quirks” for basic operation. Converters or "quirks" are required solely to handle non-Zigbee 3.0 device functions.

In this case, non-standard functions are setting up a smiley face and shifting the measurements of the temperature and humidity sensor. But they are available in the “Services” menu or configured in the “yaml” of Home Assistant.

fhempy commented 6 months ago

I'm using FHEM, not Home Assistant, but the issue is within zigbee2mqtt I think. The model is not updated. I checked the data directory where z2m saves all data, but even there the device was removed completely before re-pairing. I don't know where z2m gets the wrong model when pairing a new device. Another z2m installation, which hasn't been paired with any LYWSD03MMC devices before, works.

pvvx commented 6 months ago

This means that the command to remove the device from the list has not been sent to the Zigbee coordinator. And old data is used.

After updating the firmware, the Zigbee 3.0 device may change functionality. For example, expand. But this is not tracked in z2m. This greatly limits the expansion of the device's functionality.

In z2m, users are tied to a list of supported devices approved by z2m programmers. Everything depends only on the wishes of the z2m developers.

It was recently announced that support for the Zigbee 3.0 standard will be implemented. But I don’t know how things are in z2m.

PS: Similar to BLE, it typically takes open source writers 10 years to implement a new standard. Linux still does not support Bluetooth 5.0, which was released in 2016.

fhempy commented 6 months ago

The coordinator information is also stored in the data directory of z2m. I don't know of any other place.

Just one thing to mention regarding the external converter. Since 1.36.1 (released a few hours ago) the batteryPercentage function isn't implemented any more https://github.com/pvvx/ZigbeeTLc/blob/master/z2m/lywsd03mmc-z.js#L8 battery({percentage: true}) needs to be called instead

pvvx commented 6 months ago

I can't help you with anything other than corrections or additions to the thermometer firmware. There is no such opportunity to monitor and participate in writing the code for each user application. And additions to old products usually take a long time due to previously incorrectly chosen algorithms and other conditions. This time is significantly reduced if someone creates a new product. Often, an old product simply cannot be adapted to new formats...


There is always a workaround - you can introduce a function to change the MAC device. But this will be a “plug” for the sake of compatibility with outdated software and will not add functionality to the device, and may create new problems. You can also introduce the ability for the user to change the name and manufacturer of the device. Then z2m will be a mess and fixes will be needed faster :) But this is a more popular feature among users, as it will allow you to tag devices with a unique name for any application.


In general, the software for Zigbee is very crude and unfinished. Example - a device for relaying BLE thermometers in Zigbee has 3 temperatures, 3 humidity, 3 batteries. According to Zigbee 3.0 this is described as 3 devices on different endpoints. ZHA thinks and creates 3 temperatures, 3 humidity, but 1 battery. But there is no way to distinguish which endpoint number this temperature is from. The assignment of the sensor name in HA occurs according to unclear conditions - random. :) And there is still a lot of such chaos in the existing software for users in Zigbee...

Nephiel commented 6 months ago

Do you have any further hints? Thanks!

I was just trying to recall any other steps I took to get it to work for me. I do remember not enabling the "force remove" and "prevent from joining again" options in z2m web UI when removing the old devices. Other than that, I'm not sure, but I'd probably try restarting z2m right after removing all the non -z LYWSD03 devices, before pairing them again. Hopefully that ensures they're fully removed and z2m picks them up as the correct model.

pvvx commented 6 months ago

@fhempy Removing the device, rebooting Home Assistant, “resetting” to GND in Xiaomi LYWSD03MMC for a few seconds, new pairing with ZHA. This led to the name being updated. All "Object IDs" in Home Assistant for the LYWSD03MMC-z device have been updated. All panels and connections had to be rewritten. In the ZHA, these actions are not required, since there is no connection to the name of the device and crutches are not required to work with the device. But after Zigbee OTA update nothing is updated.

fhempy commented 6 months ago

I removed the device in z2m and did the "resetting" with the GND-RST pin connection. After that I had to remove the battery again to pair it again in z2m. Unfortunately it still remains without -z. image

pvvx commented 6 months ago

I never disconnected the batteries. A temporary GND-reset contact or pressing a button on thermometers with a button is always sufficient. To save battery, polling of all Zigbee channels for binding occurs with a random period of several minutes. This process requires a lot of energy and, taking into account pauses of several minutes, the average consumption still increases to 0.1 mA, which is a lot for the CR2032. If there is no pairing, the battery will drain quickly...

maaad commented 5 months ago

To change zigbeeModel: LYWSD03MMC to zigbeeModel: LYWSD03MMC-z with 0121 firmware I've removed device from z2m, then restarted z2m, permitted join and reset (GND+RST) LYWSD03MMC. After that my z2m has correctly defined modelID and used converter from that PR.