search

dresden-elektronik / deconz-rest-plugin

deCONZ REST-API plugin to control ZigBee devices
BSD 3-Clause "New" or "Revised" License
1.9k stars 506 forks source link

Aqara multi sensor not refreshing values regularly anymore after update to deconz 2.17.01 #6276

Closed centrofly closed 1 year ago

centrofly commented 2 years ago

Describe the bug

After updating to Deconz 2.17.01 and firmware 26720700 the update interval of my Aqara temperature/humidity/pressure sensors dropped significantly.

Usually the sensors should update when temperature changes for at least 0.5 degree Celsius or after 1 hour. Before the update this worked fine.

Now the sensors often do not update for several hours. You can see this when looking at attached CSV file which shows the logged temperatures. Beginning at 2022-08-09 in the evening (when I updated the deconz firmware package), the readings are coming in a lot less frequent, sometimes several hours apart.

TEMPERATURE 07 temperature.csv

As you can see in the file, before the update the was at least an update every hour.

Please note, I logged the values using iobroker for better visibility, but the problem is present in Deconz GUI as well, where the values not updating as well. So this is not a iobroker/adapter issue.

Steps to reproduce the behavior

Update to Deconz 2.17.01 and monitor update frequency of Aqara temperature sensors.

Expected behavior

Values should at least update once every hour, or when temperature changes 0.5 degree Celsius

Screenshots

Environment

Deconz Version: 2.17.01 / 1.7.2022 Firmware Version: 26720700

deCONZ Logs

See attached file for couple of minutes of deconz log, donot know if it contains anything useful. Let me know if you need something specific.

deconz_log.txt

Note: The sensor that corresponds to the value readings in the csv file is named named MULTI_07

Additional context

SwoopX commented 2 years ago

This is not a bug, just to be clear on that right away 🙂

I assume you upgraded from a version prior to 2.15.0? This PR is intended to bring back what you interpret as bug https://github.com/dresden-elektronik/deconz-rest-plugin/pull/6268

centrofly commented 2 years ago

Thanks for the info, but I must admit I do not fully understand what you described in the PR.

Why would you not classify the behavior I found as a bug, if the quality of data received is not as good (up to date) as before?

What caused this new behaviour (less value updates)?

SwoopX commented 2 years ago

Well, what the device does is sending the reports for the 3 values in a non-configurable way, more or less zigbee compliant, in more randomly chosen intervals. Deconz receives and processes them, so everything works as expected.

Now Xiaomi has some, not exactly zigbee compliant "special" reports, which are sent in a roughly 50 mins interval, content-wise different for every device. Those are what you perceive as reduced readings.

With the new code base, it had to be chosen what will be taken to give you updated values, which was the regular reports, as they occur more often (and you cannot have both). The PR is intended to bring back leveraging the special report in addition.

LeoeLeoeL commented 2 years ago

I can say, since Aqara TVOC DDF went to GOLD (the latest beta) air quality entity updates often while humidity not. In previous release it was the opposite.

tamtam1111 commented 2 years ago

This is not a bug, just to be clear on that right away 🙂

I assume you upgraded from a version prior to 2.15.0? This PR is intended to bring back what you interpret as bug #6268

The "fix“ would be very welcome

MaxCere commented 2 years ago

Any workaround or ETA on when we will see the changes here? https://github.com/deconz-community/deconz-docker

MaxCere commented 2 years ago

Any workaround

Rolled back to 2.14.01 as a temporary workaround works, waiting for the improvement for a stable solution.

Hakko31 commented 2 years ago

I have the same error pattern. The sensors no longer update for hours.

MaxCere commented 2 years ago

I upgraded to 2.18.1 but still, sensors are refreshing less frequently.

Smanar commented 2 years ago 
Expected behavior

Values should at least update once every hour, or when temperature changes 0.5 degree Celsius

what is your reporting interval ?

MaxCere commented 2 years ago

It's 0.5 degrees like you said but on 2.14.01 and using other Zigbee dongle updates are much more frequent....how can you tell that this is expected behavior? Take a look at the below graph, flat line is with 2.18.01 and the more frequent is with 2.14.01

User experience it's a lot better with more updates, I will migrate to another dongle if it remains the expected behavior.

image

Smanar commented 2 years ago

Your device is the "lumi.weather" ? Strange I haven't the same result on my side, I have 0.1 °C variation.

You don't have special setting on DDF stuff ?

MaxCere commented 2 years ago

Device is "lumi.sensor_ht", didn't touch any DDF.

I will try again run 2.18.01 to avoid my errors

Smanar commented 2 years ago

Ok So I don't known at all this device but from the Z2M project it seem this device use the both cluster for temperature report, the temperature one and the Xiaomi one.

The second one is broken or WIP https://github.com/dresden-elektronik/deconz-rest-plugin/pull/6268

Can you try like in this PR https://github.com/dresden-elektronik/deconz-rest-plugin/pull/6309

With replacing 

        {
          "name": "state/temperature",
          "public": false
        },

by

        {
          "name": "state/temperature",
          "awake": true,
          "parse": {
            "at": "0xff01",
            "ep": 1,
            "eval": "Item.val = Attr.val * 100",
            "fn": "xiaomi:special",
            "idx": "0x03"
          }
        },

If I m right the device will use the Xiaomi cluster instead of the temperature cluster, but hopping for better one.

grandyann commented 2 years ago

I've got the exact same behavior here with a lumi.sensor_ht:

Smanar commented 2 years ago

If I m right the "idx" to try for light level is 0x0b not 0x03 like on the sample.

grandyann commented 2 years ago

Note that my device is a xiaomi temperature and humidity sensor (not a light).

So I tried with 0x03, and also 0x0b, but nothing did really work. Note that I also tried with 0x02 because I found some info here: https://zigate.fr/xiaomi-capteur-temperature-humidite-clusters/ (sorry, in french), but I'm really not sure which value I should take. Is there any place with the datasheet for each device ?

xavax59 commented 2 years ago

Hello, i've the same problem. To solve the problem, I change the status DDF xiaomi_wsdcgq11lm_temp_hum_press_sensor.json (here /usr/share/deCONZ/devices/xiaomi/). I change Gold to Bronze

stigvi commented 2 years ago

Hello, i've the same problem. To solve the problem, I change the status DDF xiaomi_wsdcgq11lm_temp_hum_press_sensor.json (here /usr/share/deCONZ/devices/xiaomi/). I change Gold to Bronze

Yes, this is what I did, too. IMO, these sensors should still be in a state other than gold "officially"

Edit: Both

needs attention.

Smanar commented 2 years ago

Ha right, you can disable the "Gold" status to disable DDF stuff. Better to correct the report, but yes you are right, better to set the status "Bronze" than "Gold" officialy.

grandyann commented 2 years ago

Thx a lot for the tip @xavax59 and @stigvi , it made a huge difference, and now everything seems to work the way it used to before updating 👍

BabaIsYou commented 2 years ago

IMO, these sensors should still be in a state other than gold "officially"

Agree, more and more often the DDF seems published in "Gold" without having been completely validated, for convenience. I think they should be released minimally in "Silver" when first included in a release, then set to "Gold" on the next release if no issue has been reported.

Smanar commented 2 years ago

Right, but I think you are the only one that doing that. Even me I put always "Gold" out of laziness. But always for new devices, because the DDF can't broke an old one.

BabaIsYou commented 2 years ago

Right, but I think you are the only one that doing that. Even me I put always "Gold" out of laziness.

lol ! I'm just following this documentation : DDF levels

But always for new devices, because the DDF can't broke an old one. Not quite sure about that ... The new DDF will superseed the old one or legacy code auto generated one.

Smanar commented 2 years ago

lol ! I'm just following this documentation

And I m sure no one can say you are wrong, but set the "Glod" state is easier and faster, the"least effort syndrome"

Not quite sure about that ... The new DDF will superseed the old one or legacy code auto generated one.

It's new device, so there is no legacy code, no previous DDF. It can be a broken DDF or nothing ^^. I have only /2 devices that are not in "Gold", but return are too much random for the moment ....

niawag commented 2 years ago

Hi all, I'm having connection problems with a xiaomi sensor (Xiaomi Mi Smart Temperature and Humidity Sensor - v 20160516 in phoscon), I'm not sure my problem are related to this issue but it may be. The sensor is really close to the conbee2 (less than 2m with a soft wall between eachother). How can I troubleshoot it ? Should I try to change the DDF inside the docker container? Or should I get back to a prior deconz version, if so, which one?

Smanar commented 2 years ago

Perhaps https://github.com/dresden-elektronik/deconz-rest-plugin/wiki/Xiaomi:-Lost-devices-or-functional-issues

DDF don't help for connection issue, only recognition. Can be the router if the sensor use one.

github-actions[bot] commented 2 years ago

As there has not been any response in 21 days, this issue has been automatically marked as stale. At OP: Please either close this issue or keep it active It will be closed in 7 days if no further activity occurs.

centrofly commented 2 years ago

Thx a lot for the tip @xavax59 and @stigvi , it made a huge difference, and now everything seems to work the way it used to before updating 👍

@grandyann: What exactly did you change? I don not understand how/where to change DDF status from gold to bronze.

Smanar commented 2 years ago

Hello, use a text editor to edit the file with this path. You will see in the file "Status" in the first 10 line.

github-actions[bot] commented 1 year ago

As there has not been any response in 21 days, this issue has been automatically marked as stale. At OP: Please either close this issue or keep it active It will be closed in 7 days if no further activity occurs.

rasssta commented 1 year ago

Is there any work going on for this bug? Because we are many that consider it to be a bug... I thought some of my sensors were broken recently because of this issue. Applying the workaround did magic for me.

image

Take a guess when I applied the workaround...

stigvi commented 1 year ago

Is there any work going on for this bug? Because we are many that consider it to be a bug... I thought some of my sensors were broken recently because of this issue. Applying the workaround did magic for me.

Take a guess when I applied the workaround...

I got tired of this and the Philips Hue Dimmer bug not fixed for months. So I decided to switch over to zigbee2mqtt with a Sonoff coordinator and that was a positive experience.

rasssta commented 1 year ago

Yeah, I'm leaning towards doing the same (again, sigh). I moved away from Homey as I got tired of the crappy update interval/0.5c increments for my temp sensors, so if this is going to stay as a "feature" I'm off to the next one.

sceiler commented 1 year ago

@rasssta can you please point me to a step-by-step guide or tell me yourself how to apply this workaround? I have got 2 different Aqara sensors with the same issue

rasssta commented 1 year ago

@rasssta can you please point me to a step-by-step guide or tell me yourself how to apply this workaround? I have got 2 different Aqara sensors with the same issue

You have one above: https://github.com/dresden-elektronik/deconz-rest-plugin/issues/6276#issuecomment-1261927464

sceiler commented 1 year ago

Yeah I saw this but where do I find these JSON files? How do I apply it to my Conbee II/deConz?

stigvi commented 1 year ago

I have switched over to zigbee2mqtt and dont have deconz anymore, so I have to write this out of memory. But if you open or go into the user interface for deconz, you will find several small strips at the top or bottom of the screen. That is more sections or screens on the VNC session. One of those small strips is the DDF editor.

So you start with clicking on the node in the network map so that the node is the active one. Then select something like "edit DDF" in the menu. Then you open the DDF editor with clicking on the smal strip at the top or bottom. Select silver for that DDF and save and finally hot reload.

If you have this as an addon in Home Assistant, you have to do this at every restart as the changes you do are reset when the docker is restarted.

sceiler commented 1 year ago

I have switched over to zigbee2mqtt and dont have deconz anymore, so I have to write this out of memory. But if you open or go into the user interface for deconz, you will find several small strips at the top or bottom of the screen. That is more sections or screens on the VNC session. One of those small strips is the DDF editor.

So you start with clicking on the node in the network map so that the node is the active one. Then select something like "edit DDF" in the menu. Then you open the DDF editor with clicking on the smal strip at the top or bottom. Select silver for that DDF and save and finally hot reload.

If you have this as an addon in Home Assistant, you have to do this at every restart as the changes you do are reset when the docker is restarted.

thanks a lot. I got it working with your instructions. unfortunately it doesn't seem to help in my case. might be because it is a different sensor than the above mentioned or something else. I ordered a new Sonoff gateway and will replace my conbee ii with it. hopefully that will help long-term

Mimiix commented 1 year ago

I have switched over to zigbee2mqtt and dont have deconz anymore, so I have to write this out of memory. But if you open or go into the user interface for deconz, you will find several small strips at the top or bottom of the screen. That is more sections or screens on the VNC session. One of those small strips is the DDF editor. So you start with clicking on the node in the network map so that the node is the active one. Then select something like "edit DDF" in the menu. Then you open the DDF editor with clicking on the smal strip at the top or bottom. Select silver for that DDF and save and finally hot reload. If you have this as an addon in Home Assistant, you have to do this at every restart as the changes you do are reset when the docker is restarted.

thanks a lot. I got it working with your instructions. unfortunately it doesn't seem to help in my case. might be because it is a different sensor than the above mentioned or something else. I ordered a new Sonoff gateway and will replace my conbee ii with it. hopefully that will help long-term

You can use the conbee with Z2m :)

sceiler commented 1 year ago

tbh I was considering a lot if I should change at all with all the migration work required but after researching around the internet it seems the conbee ii is a bit outdated from a hardware perspective. even though it should not matter at all with my rather basic setup I need to / want to redo most of my zigbee stuff anyway so I might as well switch to the latest zigbee hardware available.

Smanar commented 1 year ago

it seems the conbee ii is a bit outdated from a hardware perspective

From memory ? The microcontroller speed ?

I still have a conbee 1

cputoaster commented 1 year ago

Just a me-too comment: I cannot pair the lumi.weather sensors anymore with latest stable docker. And I don't know how to change it easily (just copying the json to the docker mounted devices folder and changing to bronze did not work, its probably still taking the one from the image). Maybe reverting it to Bronze for the next release would be a good intermediary fix?

Smanar commented 1 year ago

There is soon a new deconz version, without all bugs on 2.19.X branch To disable a DDF you can set in to "Bronze" yes, but you can too delete it.

But no, for me the solution is correcting the DDF.

github-actions[bot] commented 1 year ago

As there has not been any response in 21 days, this issue has been automatically marked as stale. At OP: Please either close this issue or keep it active It will be closed in 7 days if no further activity occurs.

github-actions[bot] commented 1 year ago

As there has not been any response in 28 days, this issue will be closed. @ OP: If this issue is solved post what fixed it for you. If it is not solved, request to get this opened again.

deluxestyle commented 1 year ago

Can this be opened again? I still face this issue

Smanar commented 1 year ago

Have you tried to disable the DDF ?

deluxestyle commented 1 year ago

Have you tried to disable the DDF ?

Means to set the DDF to Bronze? Yes, but everytime a new version comes I have to do it again

Smanar commented 1 year ago

Yes, but everytime a new version comes I have to do it again

Ha ok, what is your OS ? you can put it in an "devices" user folder, but if the user folder is prioritary on the "deconz" folder I m not sure if it still same if the deconz one have an highter status.