pvvx / ZigbeeTLc

Custom firmware for Zigbee 3.0 IoT devices on the TLSR825x chip
Other
321 stars 15 forks source link

LYWSD03MMC configuration Error in Z2M #58

Open Bodengriller opened 4 months ago

Bodengriller commented 4 months ago

Since the Z2M update to 1.35.3-1 I get the following configuration-error for all my LYWSD03MMC:

"Failed to configure""TempHumLCD_Schlafzimmer",
"attempt 2 (Error":Read 0xa4c138a130e2c2f3/1 hvacUserInterfaceCfg([
   16
],
{
   "timeout":10000,
   "disableResponse":false,
   "disableRecovery":false,
   "disableDefaultResponse":true,
   "direction":0,
   "srcEndpoint":null,
   "reservedBits":0,
   "manufacturerCode":null,
   "transactionSequenceNumber":null,
   "writeUndiv":false
}") 
failed (Status""UNSUPPORTED_ATTRIBUTE"") 
at Endpoint.checkStatus (/app/node_modules/zigbee-herdsman/src/controller/model/endpoint.ts":"319":"28) 
at Endpoint.read (/app/node_modules/zigbee-herdsman/src/controller/model/endpoint.ts":"469":"22) 
at setupAttributes (/app/node_modules/zigbee-herdsman-converters/src/lib/modernExtend.ts":"63":"13) 
at configure (/app/node_modules/zigbee-herdsman-converters/src/lib/modernExtend.ts":"84":"17) 
at Object.configure (/app/node_modules/zigbee-herdsman-converters/src/index.ts":"141":"25) 
at Configure.configure (/app/lib/extension/configure.ts":"120":"13) 
at EventEmitter.wrappedCallback (/app/lib/eventBus.ts":"174":17))

I have already tested this external converter - same problem

There are no problems with this external converter, but all settings are missing (smiley off etc.):

const fz = require('zigbee-herdsman-converters/converters/fromZigbee');
const exposes = require('zigbee-herdsman-converters/lib/exposes');
const reporting = require('zigbee-herdsman-converters/lib/reporting');
const ota = require('zigbee-herdsman-converters/lib/ota');
const e = exposes.presets;

const definition = {
    zigbeeModel: ['LYWSD03MMC'],
    model: 'LYWSD03MMC',
    vendor: 'Xiaomi',
    description: 'Temperature & humidity sensor',
    fromZigbee: [fz.temperature, fz.humidity, fz.battery],
    toZigbee: [],
    configure: async (device, coordinatorEndpoint, logger) => {
        const endpoint = device.getEndpoint(1);
        const bindClusters = ['msTemperatureMeasurement', 'msRelativeHumidity', 'genPowerCfg'];
        await reporting.bind(endpoint, coordinatorEndpoint, bindClusters);
        await reporting.temperature(endpoint, {min: 10, max: 300, change: 10});
        await reporting.humidity(endpoint, {min: 10, max: 300, change: 50});
        await reporting.batteryVoltage(endpoint);
        await reporting.batteryPercentageRemaining(endpoint);
    },
    exposes: [e.temperature(), e.humidity(), e.battery()],
    ota: ota.zigbeeOTA,
};
module.exports = definition;

There is only this warning: OTA: Images currently unavailable for device 'LYWSD03MMC', hardwareVersion '16', manufacturerName Xiaomi, {"fieldControl":0,"manufacturerCode":4417,"imageType":522,"fileVersion":18427905}'

I'm sure that all the settings worked until recently even without an external converter. However, I can't find any issue or report on this problem. Am I doing something wrong or is it a known problem? Is there a way to solve this?

Installed Version on devices: 1141-020a-01193001-Z03MMC.zigbee

(until yesterday all devices were on v0.1.1.8 - as the problem was recognized)

slingel commented 4 months ago

Had the same thing after upgrade. Make sure binds look like this:

Unbenannt

For me the hvacuserinterfacecfg wasnt bound anymore.

slingel commented 4 months ago

nevermind...

Failed to configure 'test_pvvx', attempt 2 (Error: Read 0xa4c138222d75d9cf/1 hvacUserInterfaceCfg([16],
{"timeout":10000,"disableResponse":false,"disableRecovery":false,"disableDefaultResponse":true,"direction":0,"srcEndpoint":null,"
eservedBits":0,"manufacturerCode":null,"transactionSequenceNumber":null,"writeUndiv":false}) failed (Status
UNSUPPORTED_ATTRIBUTE') at Endpoint.checkStatus (/app/node_modules/zigbee-
herdsman/src/controller/model/endpoint.ts:319:28) at Endpoint.read (/app/node_modules/zigbee-
herdsman/src/controller/model/endpoint.ts:469:22) at setupAttributes (/app/node_modules/zigbee-herdsman-
converters/src/lib/modernExtend.ts:63:13) at configure (/app/node_modules/zigbee-herdsman-
converters/src/lib/modernExtend.ts:84:17) at Object.configure (/app/node_modules/zigbee-herdsman-
converters/src/index.ts:141:25) at Configure.configure (/app/lib/extension/configure.ts:120:13) at EventEmitter.wrappedCallback
(/app/lib/eventBus.ts:174:17))
Bodengriller commented 4 months ago

For me the hvacuserinterfacecfg wasnt bound anymore.

you are right - hvacuserinterfacecfg wasnt bound at all my LYWSD03MMC-Sensors, too But that didn't solve the problem with failed config

pvvx commented 4 months ago

New ver 0.1.2.0 - zigbeeModel: ['LYWSD03MMC-z'] https://github.com/pvvx/ZigbeeTLc/pull/50#issuecomment-1925906616

Bodengriller commented 4 months ago

New ver 0.1.2.0 - zigbeeModel: ['LYWSD03MMC-z'] #50 (comment)

Do you want to say there will be a new Version soon, that will fix this Problem?

pvvx commented 4 months ago

The devices will have different names. "-z" will be added to the name Zigbee devices, "-bz" will be added to the BLE & Zigbee devices, "-zb" to the Zigbee2BLE devices. Z2M identifies a device by name, since it does not know how to poll clusters and identifiers to accurately determine which version it is connected to. This causes many errors in Z2M, especially with Tuya devices. With the same names, Tuya has different versions of firmware and functions. And LYWSD03MMC in Z2M gets confused with the devbis version, also due to the fact that it cannot poll the version. Devbis cannot change the device name. :) Z2M users must write their own scripts. This is how Z2M is designed. Z2M also requires too many resources to operate. That's why I don't support or use Z2M.

Bodengriller commented 4 months ago

thank you for clarification. just for my understanding:

image

And LYWSD03MMC in Z2M gets confused with the devbis version, also due to the fact that it cannot poll the version. Devbis cannot change the device name. :)

I realise you're not a friend of Z2M - that's OK. I don't want to start a discussion about why you prefer ZHA :) So if I understand you correctly, the external converter from devbis simply doesn't work because we Z2M users have to build our own external converter for your PVVE version? And the official integration for the LYWSD03MMC from Z2M probably refers to the devbis-FW, which is probably why this is the same as the external converter from it.

Knowing that would be a good approach, then I could deal with it and see if I can get it built such an converter somehow, if I want to stay on your pvvx-FW with Z2M. I assume you don't have anything ready in the drawer that I just couldn't find?

pvvx commented 4 months ago

And the official integration for the LYWSD03MMC from Z2M probably refers to the devbis-FW

Yes. I'm on ZHA, devbis on Z2M.

In ZHA the name is indifferent. But I alone cannot support all types of Zigbee applications and write scripts for them... At the moment, there are already 20 devices with different BLE/Zigbee functions - different supported firmware...

I realise you're not a friend of Z2M - that's OK. I don't want to start a discussion about why you prefer ZHA :)

Preference has nothing to do with it.

TreborR2 commented 4 months ago

I have the same problem. A few days ago I did a fw update to the zigbee version and everything was OK. I updated another same device today but it shows as unsupported with "-z". And I see that what works has a slightly older fw, see picture. Does anyone have any ideas how to fix this when the bluetooth is not working? I don't have the option to flash it somehow via cables. And I have 3 more thermometers that I wanted to convert to zigbee as well, but I don't have the courage now and I don't want to turn them into another useless device.

I uploaded the fw via this page https://pvvx.github.io/atc_mithermometer/telinkmiflasher.html

Sorry, I'm a beginner. Will it be fixed somehow in cooperation with zigbee2mqtt and it will start working again on its own, or is it necessary to reset something somewhere? If so, what exactly? Thank you.

FW-zigbee-notsupported

jhonny001 commented 4 months ago

thank you for clarification. just for my understanding: image

And LYWSD03MMC in Z2M gets confused with the devbis version, also due to the fact that it cannot poll the version. Devbis cannot change the device name. :)

I realise you're not a friend of Z2M - that's OK. I don't want to start a discussion about why you prefer ZHA :) So if I understand you correctly, the external converter from devbis simply doesn't work because we Z2M users have to build our own external converter for your PVVE version? And the official integration for the LYWSD03MMC from Z2M probably refers to the devbis-FW, which is probably why this is the same as the external converter from it.

Knowing that would be a good approach, then I could deal with it and see if I can get it built such an converter somehow, if I want to stay on your pvvx-FW with Z2M. I assume you don't have anything ready in the drawer that I just couldn't find?

Did you manage to get this to work? Is there any guide to adding a new custom converter to Z2M for us newbies? I have half of all my thermomethers with fw 119 working fine and the other hald with fw 120 not working at all... :(

MortenVinding commented 4 months ago

So if I understand you correctly, the external converter from devbis simply doesn't work because we Z2M users have to build our own external converter for your PVVE version? And the official integration for the LYWSD03MMC from Z2M probably refers to the devbis-FW, which is probably why this is the same as the external converter from it.

I just took devbis converter and changed the name to -z in it. Seems to not work with special features like turnoff smiley or display, but otherwise fine.

Might look in to how to flash devbis firmware using OTA, but for now this is good enough.

I don't want to start religious discussion either, but I don't understand the augment about Z2M using too many resources. On my system it is at 0.0% cpu and 0.2% memory (of my 2GB), for the whole container...

pvvx commented 4 months ago

Might look in to how to flash devbis firmware using OTA, but for now this is good enough.

python3 zb_bin_ota.py z03mmc.bin z03mmc -m0x1141 -i0x020a -v0x09993001 -s"Zigbee ver: ZigbeeTLc to devis"

zb_bin_ota.py

https://github.com/pvvx/ZigbeeTLc/tree/master/bin/devbis

Added to TelinkMiFlasher.html : image

Warning: The devbis version consumes twice as much as ZigbeeTlc!

I don't want to start religious discussion either, but I don't understand the augment about Z2M using too many resources. On my system it is at 0.0% cpu and 0.2% memory (of my 2GB), for the whole container...

You forgot MQTT and stuff. 200 MB RAM minimum With 2 GB of RAM, Home Assistant does not work well. The database does not fit into the cache. For comfortable operation, the media speed (PCIe NVME) must be at least 1 GB/s.

TreborR2 commented 4 months ago

So far I have solved it like this https://www.reddit.com/r/homeassistant/comments/1aok4l4/help_im_stuck_with_an_unknown_zigbee_device/ . Apparently my posts were published "Hairy-Plum-3118", so you need to be logged in.

pvvx commented 4 months ago

@TreborR2 - The device names have been renamed because the devbis converter (at https://github.com/devbis/z03mmc/blob/master/converters/lywsd03mmc.js) is not compatible with the firmware from the ZigbeeTlc repository.

TreborR2 commented 4 months ago

I don't understand your message. Does that mean you don't recommend this solution for some reason? Is not working? Or why? For now, it's blaming me for the data, see picture. Xiaomi-zigbee

ediguidedog commented 4 months ago

Might look in to how to flash devbis firmware using OTA, but for now this is good enough.

python3 zb_bin_ota.py z03mmc.bin z03mmc -m0x1141 -i0x020a -v0x09993001 -s"Zigbee ver: ZigbeeTLc to devis"

zb_bin_ota.py

https://github.com/pvvx/ZigbeeTLc/tree/master/bin/devbis

Added to TelinkMiFlasher.html :

Sorry I'm going to need my hand held here. Is the python script above something I run from the command line on the box with my zigbee coordinator in it and if so how does it know which device to actually update as I have a few and only one is not working or is it something incorporated into the TelinkMiFlasher which I understands needs Bluetooth to connect and the device I want to flash no longer has Bluetooth available.

What I want to achieve is to get one device I flashed OTA that is now zigbee only and now appearing with a -z suffix back to the state that Z2M recognises it. If that's using the devis firmware that I'm all for it.

pvvx commented 4 months ago

Disable other thermometers during Zigbee OTA. Remove the battery. Then delete the Zigbee OTA file and reboot the system.

If that's using the devis firmware that I'm all for it.

Update all thermometers to devbis version. The batteries in Devbis will last up to 6 months.

ediguidedog commented 4 months ago

Disable other thermometers during Zigbee OTA. Remove the battery. Then delete the Zigbee OTA file and reboot the system.

If that's using the devis firmware that I'm all for it.

Update all thermometers to devbis version. The batteries in Devbis will last up to 6 months.

So 1 copy python script to folder on linux box hosting z2m which has sonos stick plugged into it 2 copy z03mmc.bin to same folder 3 execute python3 zb_bin_ota.py z03mmc.bin z03mmc -m0x1141 -i0x020a -v0x09993001 -s"Zigbee ver: ZigbeeTLc to devis" 4 reboot my linux box

Sorry I'm really lost here Is that correct ?

pvvx commented 4 months ago

Is that correct ?

No. In Zigbee2MQTT OTA is more difficult to do.

Like this: https://github.com/pvvx/ZigbeeTLc/issues/43#issuecomment-1929097808 but you need to edit the links and numbers in accordance with the new OTA file.

In ZHA you need to copy the *.zigbee file to the directory for OTA and enable OTA.

pvvx commented 4 months ago

I don't understand your message. Does that mean you don't recommend this solution for some reason? Is not working? Or why?

The "Calibration" and "comfort" settings do not match the ZigbeeTlc version.

Bodengriller commented 4 months ago

I'm still puzzled by the discussion about the name - I don't really understand something here.

I have updated one of my 3 thermometers to 0.1.2.0. All 3 still show up as LYWSD03MMC, not as LYWSD03MMC-z,

image

If I connect it via the following external converter, everything works without errors and can be configured. So this part of the script seems to be correct. Of course, the settings for temp calibration, smiley and so on are missing here

const fz = require('zigbee-herdsman-converters/converters/fromZigbee');
const exposes = require('zigbee-herdsman-converters/lib/exposes');
const reporting = require('zigbee-herdsman-converters/lib/reporting');
const ota = require('zigbee-herdsman-converters/lib/ota');
const e = exposes.presets;

const definition = {
    zigbeeModel: ['LYWSD03MMC'],
    model: 'LYWSD03MMC',
    vendor: 'Xiaomi',
    description: 'Temperature & humidity sensor',
    fromZigbee: [fz.temperature, fz.humidity, fz.battery],
    toZigbee: [],
    configure: async (device, coordinatorEndpoint, logger) => {
        const endpoint = device.getEndpoint(1);
        const bindClusters = ['msTemperatureMeasurement', 'msRelativeHumidity', 'genPowerCfg'];
        await reporting.bind(endpoint, coordinatorEndpoint, bindClusters);
        await reporting.temperature(endpoint, {min: 10, max: 300, change: 10});
        await reporting.humidity(endpoint, {min: 10, max: 300, change: 50});
        await reporting.batteryVoltage(endpoint);
        await reporting.batteryPercentageRemaining(endpoint);
    },
    exposes: [e.temperature(), e.humidity(), e.battery()],
    ota: ota.zigbeeOTA,
};

The current external converter vor devbis-FW runs into our known problems - that is clear

Well, that's 147 very manageable lines of code, the last part of which is no problem.

If I understand everything correctly, we would only have to use this converter as a template and adapt it to the corresponding variables/parameters/attributes of the pvvx-FW. Then all Z2M users would be served by simply using this converter. The advantage for all of us would be that we would benefit from the more energy-efficient pvvx-FW.

Do I understand that correctly?

Let's make an example: @pvvx how or where can we find the correct attribut for this part in your sourcecode?

binary({
            name: 'show_smiley',
            valueOn: ['SHOW', 1],
            valueOff: ['HIDE', 0],
            cluster: 'hvacUserInterfaceCfg',
            attribute: {ID: 0x0010, type: dataType.boolean},
            description: 'Whether to show a smiley on the device screen.',
        })

Maybe the ID is the only thing to change?! I think if we tackle this all together, everyone will benefit from making this firmware even more popular :)

pvvx commented 4 months ago

Maybe the ID is the only thing to change?!

ZigBee Cluster Library Specification -> 2.6.2 Data Types -> Table 2-10. Data Types

Setting offsets for temperature and humidity Switch temperature between Celsius and Fahrenheit, Show smiley, Comfort parameters

MortenVinding commented 4 months ago

Finally got some time to look in to this.

Might look in to how to flash devbis firmware using OTA, but for now this is good enough.

python3 zb_bin_ota.py z03mmc.bin z03mmc -m0x1141 -i0x020a -v0x09993001 -s"Zigbee ver: ZigbeeTLc to devis"

zb_bin_ota.py

https://github.com/pvvx/ZigbeeTLc/tree/master/bin/devbis

Added to TelinkMiFlasher.html : image

Warning: The devbis version consumes twice as much as ZigbeeTlc!

Thank you so much for your help 👍

But I have run in to a couple of problems:

  1. Your python script seems to add some gibberish at the begging of the firmware file (just removed in editor)
  2. Zigbee2MQTT complain about some memory mapping: "Update of '0xa4a1281ad787cb93' failed (Attempt to access memory outside buffer bounds)"

I don't want to start religious discussion either, but I don't understand the augment about Z2M using too many resources. On my system it is at 0.0% cpu and 0.2% memory (of my 2GB), for the whole container...

You forgot MQTT and stuff. 200 MB RAM minimum With 2 GB of RAM, Home Assistant does not work well. The database does not fit into the cache. For comfortable operation, the media speed (PCIe NVME) must be at least 1 GB/s.

Ahh sorry I didn't look the at the right place. My Zigbee2MQTT node.js process is consuming 6.5% of the RAM, but HA is using 25%. So still rather insignificant all in all. I'm using MQTT for many other things besides Z2M, but it is only using 0.6% of the RAM.

My HA is rock stable and super fast running on the 2GB Proxmox is allocating to it. It is not actively using any swap, and it only has 650MB swap anyway:

# free -h
               total        used        free      shared  buff/cache   available
Mem:           1.9Gi       1.0Gi        79Mi       3.0Mi       837Mi       883Mi
Swap:          650Mi       331Mi       319Mi

zwave is using 3.1% RAM, and I only have 2 zwave devices (as compared with 41 zigbee.

Don't you think ZHA is about just as demanding, only not as visible because it buried in the rest of HA python processes?

MortenVinding commented 4 months ago

Maybe the ID is the only thing to change?!

ZigBee Cluster Library Specification -> 2.6.2 Data Types -> Table 2-10. Data Types

Setting offsets for temperature and humidity Switch temperature between Celsius and Fahrenheit, Show smiley, Comfort parameters

That is awesome.

I have changed this to get the smiley working again:

        binary({
            name: 'show_smiley',
            valueOn: ['SHOW', 0],
            valueOff: ['HIDE', 1],
            cluster: 'hvacUserInterfaceCfg',
            attribute: 'programmingVisibility',
            description: 'Whether to show a smiley on the device screen.',
        }),

havn't looked at calibration yet, but it seems to pretty straight forward.

pvvx commented 4 months ago

available 883Mi

Consequently, assistant.db no longer fits into the RAM buffers. When you set to display the graph for a year, it will slow down mercilessly...

 free -h
               total        used        free      shared  buff/cache   available
Mem:           3.6Gi       590Mi       2.7Gi       5.6Mi       441Mi       3.0Gi
Swap:          1.8Gi          0B       1.8Gi

hdparm -tT /dev/nvme0n1

 Timing cached reads:   2062 MB in  2.00 seconds = 1031.08 MB/sec
 Timing buffered disk reads: 1198 MB in  3.00 seconds = 399.29 MB/sec

NanoPi_R5S

home-assistant_v2.db 3.5G

Bodengriller commented 4 months ago

@pvvx : thank you very much for this info!

https://github.com/pvvx/ZigbeeTLc/issues/30#issue-2034396126 https://github.com/pvvx/ZigbeeTLc/issues/28

@MortenVinding

I have changed this to get the smiley working again:

        binary({
            name: 'show_smiley',
            valueOn: ['SHOW', 0],
            valueOff: ['HIDE', 1],
            cluster: 'hvacUserInterfaceCfg',
            attribute: 'programmingVisibility',
            description: 'Whether to show a smiley on the device screen.',
        }),

hmm... why:

attribute: 'programmingVisibility'

?

devbis:

attribute: {ID: 0x0010, type: dataType.boolean},

ZHA-Settings from pvvx say ENUM8 (id:0x30)

so why not:

attribute: {ID: 0x30, type: dataType.boolean}

or at least:

attribute: 'ScheduleProgrammingVisibility'

? How do you come up with 'programmingVisibility'?

TreborR2 commented 4 months ago

I'm still puzzled by the discussion about the name - I don't really understand something here.

I have updated one of my 3 thermometers to 0.1.2.0. All 3 still show up as LYWSD03MMC, not as LYWSD03MMC-z,

image

If I connect it via the following external converter, everything works without errors and can be configured. So this part of the script seems to be correct. Of course, the settings for temp calibration, smiley and so on are missing here

const fz = require('zigbee-herdsman-converters/converters/fromZigbee');
const exposes = require('zigbee-herdsman-converters/lib/exposes');
const reporting = require('zigbee-herdsman-converters/lib/reporting');
const ota = require('zigbee-herdsman-converters/lib/ota');
const e = exposes.presets;

const definition = {
    zigbeeModel: ['LYWSD03MMC'],
    model: 'LYWSD03MMC',
    vendor: 'Xiaomi',
    description: 'Temperature & humidity sensor',
    fromZigbee: [fz.temperature, fz.humidity, fz.battery],
    toZigbee: [],
    configure: async (device, coordinatorEndpoint, logger) => {
        const endpoint = device.getEndpoint(1);
        const bindClusters = ['msTemperatureMeasurement', 'msRelativeHumidity', 'genPowerCfg'];
        await reporting.bind(endpoint, coordinatorEndpoint, bindClusters);
        await reporting.temperature(endpoint, {min: 10, max: 300, change: 10});
        await reporting.humidity(endpoint, {min: 10, max: 300, change: 50});
        await reporting.batteryVoltage(endpoint);
        await reporting.batteryPercentageRemaining(endpoint);
    },
    exposes: [e.temperature(), e.humidity(), e.battery()],
    ota: ota.zigbeeOTA,
};

The current external converter vor devbis-FW runs into our known problems - that is clear

Well, that's 147 very manageable lines of code, the last part of which is no problem.

If I understand everything correctly, we would only have to use this converter as a template and adapt it to the corresponding variables/parameters/attributes of the pvvx-FW. Then all Z2M users would be served by simply using this converter. The advantage for all of us would be that we would benefit from the more energy-efficient pvvx-FW.

Do I understand that correctly?

Let's make an example: @pvvx how or where can we find the correct attribut for this part in your sourcecode?

binary({
            name: 'show_smiley',
            valueOn: ['SHOW', 1],
            valueOff: ['HIDE', 0],
            cluster: 'hvacUserInterfaceCfg',
            attribute: {ID: 0x0010, type: dataType.boolean},
            description: 'Whether to show a smiley on the device screen.',
        })

Maybe the ID is the only thing to change?! I think if we tackle this all together, everyone will benefit from making this firmware even more popular :)

How is this possible? And are you really using zigbee2mqtt or ZHA image

Bodengriller commented 4 months ago

How is this possible? And are you really using zigbee2mqtt or ZHA

That is what I don't understand Maybe because my devices were already paired?

definetly Z2M:

image
TreborR2 commented 4 months ago

And are you using the conversion file? If Z2M fix it??? And I still ask, perhaps stupidly, what good is the conversion file if the thermometer has the right name and the Z2M recognizes it? The picture shows the thermometer with the functional FW, where I don't use the conversion file, and I have all the setting options there. On the other hand, I don't see some of these options with the new FW and converter??? Otherwise, I tried putting 3 thermometers in one place (1. with original BT FW, 2. with older FW without conversion file and 3. with new FW and preview file) and the temperature and humidity data were almost identical. Just for fun.

image

Bodengriller commented 4 months ago

I think I got it.

Here I have modified the devbis version of the ext. converter so that it works with Z2M for the pvvx-FW. I have tested everything so far and could not find any errors.

Open questions:

  1. Why is the -z still missing on my devices? I can't explain that. @pvvx do you have any ideas?

Perhaps lines 29 and 30 still need to be adjusted for other users who want to use this converter

I left a lot of comments in to have a comparison to the devbis version - these can all still be removed

  1. line 51 VS line 55: attribute: {ID: 0x0000, type: dataType.enum8}, attribute: 'tempDisplayMode',

I don't understand where to get the attributes like 'tempDisplayMode' both works fine, the first one with ID fits better to the publications of pvvx, could therefore be adapted more easily

  1. Line 77 ff. enable_display: couldn't find the right attribute for pvvx-FW @pvvx is "display off" implemented in your FW? (do you know the attribute?)
pvvx commented 4 months ago

"display off" implemented in your FW? (do you know the attribute?)

There is no display shutdown. The devbis implementation of "Display off" does not turn off the LCD, but rather displays a blank screen. This does not reduce battery consumption.

No documentation found for all display chip variants...

Bodengriller commented 4 months ago

Thanks for the clarification. Then this option really makes no sense and can be omitted. (I would actually have thought that an LCD display that doesn't show anything would also require less energy...but anyway, if you want sensors without a display, the Aquara is a better option anyway).

Bodengriller commented 4 months ago

so here is "my" converter with everything cleaned up so far.

only open questions are line 29/30

zigbeeModel: ['LYWSD03MMC'],
model: 'LYWSD03MMC',

[Edit] updated version below

pvvx commented 4 months ago

(I would actually have thought that an LCD display that doesn't show anything would also require less energy...but anyway, if you want sensors without a display, the Aquara is a better option anyway).

TS0201_TZ3000 + soon TH03Z, ...

Bodengriller commented 4 months ago

finally got it. I had to remove all of my three Devices and then I started a fresh Interview. Now all devices have the -z extension. It seems that Z2M somehow identify the devices, even when they get a new zigbee-name with an OTA-update. ... I'm not sure what to make of this

So this is the final converter, that should work well with v.0.1.2.0 for every new paired devices.

Thanks a lot for your support @pvvx !

pvvx commented 4 months ago

Is that suitable? https://github.com/pvvx/ZigbeeTLc/tree/master/z2m

MortenVinding commented 4 months ago

Consequently, assistant.db no longer fits into the RAM buffers. When you set to display the graph for a year, it will slow down mercilessly...

 free -h
               total        used        free      shared  buff/cache   available
Mem:           3.6Gi       590Mi       2.7Gi       5.6Mi       441Mi       3.0Gi
Swap:          1.8Gi          0B       1.8Gi

That's a good point! My db is currently only 420MB. I only have about half a year of data, can't remember why, but at some point I had to wipe it. But reading in that half year is snappy and dosen't seem to consume much memory.

But you got me thinking. I will keep an eye on it when my db grows.

hdparm -tT /dev/nvme0n1

 Timing cached reads:   2062 MB in  2.00 seconds = 1031.08 MB/sec
 Timing buffered disk reads: 1198 MB in  3.00 seconds = 399.29 MB/sec

NanoPi_R5S

home-assistant_v2.db 3.5G

hdparm does not work in the HA VM for some reason, but directly on the Proxmox OS it is:

root@pve:~# hdparm -tT /dev/sda

/dev/sda:
 Timing cached reads:   26538 MB in  1.98 seconds = 13389.94 MB/sec
 Timing buffered disk reads: 1576 MB in  3.00 seconds = 524.85 MB/sec
MortenVinding commented 4 months ago

hmm... why:

attribute: 'programmingVisibility'

?

devbis:

attribute: {ID: 0x0010, type: dataType.boolean},

ZHA-Settings from pvvx say ENUM8 (id:0x30)

so why not:

attribute: {ID: 0x30, type: dataType.boolean}

or at least:

attribute: 'ScheduleProgrammingVisibility'

? How do you come up with 'programmingVisibility'?

I didn't actually notice the difference. programmingVisibility I got from the Development console in Z2M: image

Don't why it's called like that, but it works...

pvvx commented 4 months ago

Zigbee Alliance Cluster Library Specification, Revision 6..8 https://github.com/Koenkk/zigbee-herdsman/blob/master/docs/07-5123-08-Zigbee-Cluster-Library.pdf

6.6.2.1 Attributes The attributes of this cluster are summarized in Table 6-49.

and Table 2-11. Data Types

My db is currently only 420MB. I only have about half a year of data, can't remember why, but at some point I had to wipe it.

I have a base depth limit set to 1 month. Next, HA performs data averaging. Rewriting (trimming) of the database in HA occurs every day. This causes all 3.5Gb to be overwritten.

image

image

MortenVinding commented 4 months ago

Zigbee Alliance Cluster Library Specification, Revision 6..8 https://github.com/Koenkk/zigbee-herdsman/blob/master/docs/07-5123-08-Zigbee-Cluster-Library.pdf

6.6.2.1 Attributes The attributes of this cluster are summarized in Table 6-49.

and Table 2-11. Data Types

Yes I see. Funny why Koenkk has decided not to use the full name...

My db is currently only 420MB. I only have about half a year of data, can't remember why, but at some point I had to wipe it.

I have a base depth limit set to 1 month. Next, HA performs data averaging. Rewriting (trimming) of the database in HA occurs every day. This causes all 3.5Gb to be overwritten.

Okay it sounds like you have spend whole lot more time tuning your recorder. Honestly I haven't touched a thing.

But I do sync some of the data to InfluxDB though.

MortenVinding commented 4 months ago

finally got it. I had to remove all of my three Devices and then I started a fresh Interview. Now all devices have the -z extension. It seems that Z2M somehow identify the devices, even when they get a new zigbee-name with an OTA-update. > ... I'm not sure what to make of this

Could it be related to this in the 1.35 release: https://github.com/Koenkk/zigbee2mqtt/releases/tag/1.35.0

This release contains a quite fundamental change on how Zigbee2MQTT handles unsupported devices. Zigbee2MQTT will now attempt to automatically detect features supported by the device and discover those to e.g. the frontend and Home Assistant. This means that if you, for example pair an unsupported light, there is no need to wait for official support. Basic features will work out-of-the-box.

slingel commented 4 months ago

If you read the modelID from the dev console after update to 0.1.2.0 and restart with the new converter devices will be updated and don't have to be readded.

pvvx commented 4 months ago

If the functionality has changed after the OTA, the Zigbee coordinator does not know this. In the Zigbee coordinator, the device is registered by MAC address.

Bodengriller commented 4 months ago

I didn't actually notice the difference. programmingVisibility I got from the Development console in Z2M

And I didn't know the dev-console yet, thank you :D

Zigbee Alliance Cluster Library Specification, Revision 6..8

That's exactly what I meant. In the docu it's called 'ScheduleProgrammingVisibility' But in the converter you should use only 'ProgrammingVisibility' So I did not find a docu where the correct atrribute-names for the converters are listed. That is why I just used the hex-IDs as in the official docu (or in your examples for ZHA :) )

If you read the modelID from the dev console after update to 0.1.2.0 and restart with the new converter devices will be updated and don't have to be readded.

Didn't Work for me, nor restart of Z2M than the complete host

In the Zigbee coordinator, the device is registered by MAC address.

That would explain everything, I almost thought so.

TreborR2 commented 4 months ago

Aktualizoval jsem jeden ze svých 3 teploměrů na 0.1.2.0. Všechny 3 se stále zobrazují jako LYWSD03MMC, ne jako LYWSD03MMC-z

@Bodengriller

  1. Thank you for the conversion file! Works great with the new FW!
  2. I have a question about the update to the latest FW. If you did the update from 0119 to 0120 and via OTA and via Z2M? Could you write the exact procedure how to do this? I'm not entirely sure how to do this and would hate to brick the device. When I try to update via Z2M now, it fails and reports an error, see image, but if I understand correctly, this is "normal" behavior?

image

rwronski commented 4 months ago

Can anyone please tell me or make a short summary what to do if I have a Z2M and thermometer with error "Exception converter: LYWSD03MMC-z has no endpoint"? OTA isn't possible in Z2M now.

slingel commented 4 months ago

@TreborR2

this is how I do updates through z2m:

Unbenannt

Bodengriller commented 4 months ago

that's my OTA-config:

image

make sure that you store the files in the "zigbee2mqtt" subfolder (not directly next to the HA-config-File)

pvvx commented 4 months ago

What a nightmare in Z2M... Well, it was easier to write everything from scratch in C... :)

rwronski commented 4 months ago

OK guys, but I don't have an "update" button on this thermometer. In the OTA panel there's no this thermometer. @pvvx maybe it's a nightmare, but in ZHA half of my devices aren't working ;)