search

dresden-elektronik / deconz-rest-plugin

deCONZ REST-API plugin to control ZigBee devices
BSD 3-Clause "New" or "Revised" License
1.9k stars 499 forks source link

IAS fingerprint check fails for TUYA TS0202 devices, fails enrolment leading erratic behaviour. #6368

Closed Monofin closed 1 year ago

Monofin commented 2 years ago

Describe the bug

During testing of one of the TUYA TS0202 series of motion sensors, I discovered that the sensor was not being enrolled properly post the integration of: https://github.com/dresden-elektronik/deconz-rest-plugin/pull/6297

The fingerprint of the device causes a failure for the check for cluster 0x0500 at lines 177-178 of ias_zone.cpp

https://github.com/dresden-elektronik/deconz-rest-plugin/blob/50c532e6866fb97f1091f7d81baa7f536696eb7c/ias_zone.cpp#L178

(This causes very erratic behaviour of the device, occasionally updating and not at other times)

Explicitly specifying the fingerprint of the device in the DDF solves the issue, but I suspect there's something more subtle going on

  "fingerprint": {
    "profile": "0x0104",
    "device": "0x0402",
    "endpoint": "0x01",
    "in": [
      "0x0000",
      "0x0001",
      "0x0500"
    ]
  },

I'm not sure this fingerprint is 100% accurate for the device, (I'm lacking understanding of how the fingerprint mechanism works in depth), but works on the above sensor.

(If the fingerprint is definitely a requirement of the DDF, then I will update the PR that added the support in the first place)

Steps to reproduce the behavior

Include any TUYA TS0202 based (e.g. _TZ3000_otvn3lne) in the network, and check the debug logs in IAS and ERROR mode: Will show lines similar to:

No IAS sensor found for endpoint: 0x01

And: 15:48:48:651 void DeRestPluginPrivate::handleIasZoneClusterIndication(const deCONZ::ApsDataIndication&, deCONZ::ZclFrame&),196: assertion 'itemIasState' failed 15:48:48:651 void DeRestPluginPrivate::handleIasZoneClusterIndication(const deCONZ::ApsDataIndication&, deCONZ::ZclFrame&),197: assertion 'itemPending' failed

Where the sensor is failed to be enrolled due to the fingerprint check.

Expected behavior

Sensor should be enrolled successfully, fingerprint check should pass (?) as the device does contain the 0x0500 cluster.

Screenshots

Environment

deCONZ v2.18.2-beta, Qt: 5.11.3, Commit: 5eaa4b Firmware: 0x26780700

deCONZ Logs

Additional context

Smanar commented 2 years ago

I'm not sure this fingerprint is 100% accurate for the device, (I'm lacking understanding of how the fingerprint mechanism works in depth), but works on the above sensor.

Ha yes, had same issue too and have updated the fingerprint in the DDF too. But you are right, now after reflexion, the fingerprint in the DDF is a fingerprint to check (according the device and enable the DDF or not) or a fingerprint to set (to set the fingerprint on the API entry)

Monofin commented 2 years ago

Cool, some things to look at there I guess!

There’s a secondary issue which I’m looking into, is that after the IAS bindings and enrolment are working, if the battery sensor cluster is bound to, the REST interface reports two consecutive battery reports, one at 0% and then one at the correct value, about every 5 seconds or so regardless of config… very odd.

Smanar commented 2 years ago

the REST interface reports two consecutive battery reports, one at 0% and then one at the correct value, about every 5 seconds

Ha ? Can you explain better ? You have that at inclusion only or all the time ? and you have a battery report every 5 s ? It realy too much

On the DDF setting are

               "min":60,
               "max":3600,
               "change":"0x00000001"

IDK where the 0 value is coming but you can be specific too, with replacing for exemple

            {
               "name":"config/battery",
               "default":0
            },

by

            {
               "name":"config/battery",
               "parse":{
                  "at":"0x0021",
                  "cl":"0x0001",
                  "ep":1,
                  "eval":"Item.val = Attr.val / 2",
                  "fn":"zcl"
               },
               "read":{
                  "ep":1,
                  "cl":"0x0001",
                  "at":"0x0021"
               }
            },
Monofin commented 2 years ago

Mea Culpa: Battery status problem above looks plausibly like a deconz-homebridge issue related to the device, I'm not seeing any incoming Zigbee activity that would cause the behaviour, and it occurs exactly on every deconz-homebridge heartbeat.

Some output (Homebridge, with Homebridge-deconz): (Correction from me, its every 15 seconds, deconz-homebridge heartbeat is every 15.)

[29/09/2022, 22:05:59] [deCONZ] Kitchen Motion 1 Battery: set Battery Level from 94% to 0% [29/09/2022, 22:05:59] [deCONZ] Kitchen Motion 1 Battery: set Status Low Battery from 0 to 1 [29/09/2022, 22:05:59] [deCONZ] Kitchen Motion 1 Battery: set Battery Level from 0% to 94% [29/09/2022, 22:05:59] [deCONZ] Kitchen Motion 1 Battery: set Status Low Battery from 1 to 0 [29/09/2022, 22:06:14] [deCONZ] Kitchen Motion 1 Battery: set Battery Level from 94% to 0% [29/09/2022, 22:06:14] [deCONZ] Kitchen Motion 1 Battery: set Status Low Battery from 0 to 1 [29/09/2022, 22:06:14] [deCONZ] Kitchen Motion 1 Battery: set Battery Level from 0% to 94% [29/09/2022, 22:06:14] [deCONZ] Kitchen Motion 1 Battery: set Status Low Battery from 1 to 0 [29/09/2022, 22:06:14] [deCONZ] Kitchen Motion 2 Battery: set Battery Level from 100% to 0% [29/09/2022, 22:06:14] [deCONZ] Kitchen Motion 2 Battery: set Status Low Battery from 0 to 1 [29/09/2022, 22:06:14] [deCONZ] Kitchen Motion 2 Battery: set Battery Level from 0% to 100% [29/09/2022, 22:06:14] [deCONZ] Kitchen Motion 2 Battery: set Status Low Battery from 1 to 0 [29/09/2022, 22:06:29] [deCONZ] Kitchen Motion 1 Battery: set Battery Level from 94% to 0% [29/09/2022, 22:06:29] [deCONZ] Kitchen Motion 1 Battery: set Status Low Battery from 0 to 1 [29/09/2022, 22:06:29] [deCONZ] Kitchen Motion 1 Battery: set Battery Level from 0% to 94% [29/09/2022, 22:06:29] [deCONZ] Kitchen Motion 1 Battery: set Status Low Battery from 1 to 0

Using the polling method (how it was originally defaulted to before the enrolment started working), its fine - as you suggested above - that works well, but looks like there's something odd in the bindings.

Probably a separate issue, but it only occurs when the enrolment is successful, so it is vaguely linked to this one...

Smanar commented 2 years ago

I have checked the code, and it seem the "fingerprint" part is not used as check, but to force the fingerprint used by the device, so it's a good thing, it help for device detection and better to set it for boring device.

For the battery, there is the problem about report, they are too fast, not good for battery, can explain why some user have battery drain (but not all ....)

Another problem, why 2 values ? the good one and the zero one, for me there is a problem with the 2 cores the legacy and the DDF one, as each of them is able to work alone. And I don't think the 0 value is from the device (so from a bind or a report)

So you can perhaps see a difference using this one, full DDF core

{
   "schema":"devcap1.schema.json",
   "manufacturername":[
      "_TZ3000_msl6wxk9",
      "_TZ3000_otvn3lne"
   ],
   "modelid":[
      "TS0202",
      "TS0202"
   ],
   "product":"TS0202 Presence sensor",
   "sleeper":true,
   "status":"Gold",
   "subdevices":[
      {
         "type":"$TYPE_PRESENCE_SENSOR",
         "restapi":"/sensors",
         "uuid":[
            "$address.ext",
            "0x01",
            "0x0500"
         ],
         "fingerprint":{
            "profile":"0x0104",
            "device":"0x0402",
            "endpoint":"0x01",
            "in":[
               "0x0000",
               "0x0001",
               "0x0500"
            ]
         },
         "items":[
            {
               "name":"attr/id"
            },
            {
               "name":"attr/lastannounced"
            },
            {
               "name":"attr/lastseen"
            },
            {
               "name":"attr/manufacturername"
            },
            {
               "name":"attr/modelid"
            },
            {
               "name":"attr/name"
            },
            {
               "name":"attr/swversion"
            },
            {
               "name":"attr/type"
            },
            {
               "name":"attr/uniqueid"
            },
            {
               "name":"config/battery",
               "parse":{
                  "at":"0x0021",
                  "cl":"0x0001",
                  "ep":1,
                  "eval":"Item.val = Attr.val / 2",
                  "fn":"zcl"
               },
               "read":{
                  "ep":1,
                  "cl":"0x0001",
                  "at":"0x0021"
               }
            },
            {
               "name":"config/duration"
            },
            {
               "name":"config/enrolled",
               "public":false
            },
            {
               "name":"config/on"
            },
            {
               "name":"config/pending"
            },
            {
               "name":"config/reachable"
            },
            {
               "name":"state/lastupdated"
            },
            {
               "name":"state/presence",
               "awake":true,
               "parse":{
                  "fn":"ias:zonestatus",
                  "mask":"alarm1"
               }
            }
         ]
      }
   ],
   "bindings":[
      {
         "bind":"unicast",
         "src.ep":1,
         "cl":"0x0500"
      },
      {
         "bind":"unicast",
         "src.ep":1,
         "cl":"0x0001",
         "report":[
            {
               "at":"0x0021",
               "dt":"0x20",
               "min":60,
               "max":3600,
               "change":"0x00000001"
            }
         ]
      }
   ]
}

And this one using more legacy code

{
   "schema":"devcap1.schema.json",
   "manufacturername":[
      "_TZ3000_msl6wxk9",
      "_TZ3000_otvn3lne"
   ],
   "modelid":[
      "TS0202",
      "TS0202"
   ],
   "product":"TS0202 Presence sensor",
   "sleeper":true,
   "status":"Gold",
   "subdevices":[
      {
         "type":"$TYPE_PRESENCE_SENSOR",
         "restapi":"/sensors",
         "uuid":[
            "$address.ext",
            "0x01",
            "0x0500"
         ],
         "fingerprint":{
            "profile":"0x0104",
            "device":"0x0402",
            "endpoint":"0x01",
            "in":[
               "0x0000",
               "0x0001",
               "0x0500"
            ]
         },
         "items":[
            {
               "name":"attr/id"
            },
            {
               "name":"attr/lastannounced"
            },
            {
               "name":"attr/lastseen"
            },
            {
               "name":"attr/manufacturername"
            },
            {
               "name":"attr/modelid"
            },
            {
               "name":"attr/name"
            },
            {
               "name":"attr/swversion"
            },
            {
               "name":"attr/type"
            },
            {
               "name":"attr/uniqueid"
            },
            {
               "name":"config/battery"
            },
            {
               "name":"config/duration"
            },
            {
               "name":"config/enrolled",
               "public":false
            },
            {
               "name":"config/on"
            },
            {
               "name":"config/pending"
            },
            {
               "name":"config/reachable"
            },
            {
               "name":"state/lastupdated"
            },
            {
               "name":"state/presence"
            }
         ]
      }
   ],
   "bindings":[
      {
         "bind":"unicast",
         "src.ep":1,
         "cl":"0x0500"
      },
      {
         "bind":"unicast",
         "src.ep":1,
         "cl":"0x0001",
         "report":[
            {
               "at":"0x0021",
               "dt":"0x20",
               "min":60,
               "max":3600,
               "change":"0x00000001"
            }
         ]
      }
   ]
}

No need to re include the device.

Monofin commented 2 years ago

Thanks for the DDF: I've resolved the Battery issue to a problem with the device having two resources from when it was presenting a ZHABattery as part of an older DDF file - so we can discount that.

(It's now fully working as a device properly once removed from the homebridge-deconz and re-added, sorry for the extra noise!)

The fingerprint in the DDF is great for forcing the fingerprint to a set value, but I don't think it should be needed in this case.

Without the forced fingerprint in the DDF, the check for hasInCluster() is failing for the device, despite definitely having 0x0500 as one of the clusters on the actual device: https://github.com/dresden-elektronik/deconz-rest-plugin/blob/2d72807bbeccbb91b6cb1d6c7d2228009f9e7784/sensor.cpp#L122

I haven't got GDB on the case yet (I'm on aarch64 and trying to get the Ubuntu tooling and dependencies satisfied to build deconz...), but I suspect that either the cluster is being marked as an 'out' cluster, or some other cluster registration issue is at play.

I'll see if I can get a build done today.

Smanar commented 2 years ago

the check for hasInCluster() is failing for the device, despite definitely having 0x0500 as one of the clusters on the actual device:

The problem is the inclusion is event based, so the information can be here but too late. And some bugged device can hide a cluster (detected later by deconz, so still displayed)

For me forcing the fingerprint is a good thing, as it's based on model id and manufacture name, so device dependent.

There is too some device with moving cluster ^^, you can see different cluster at every inclusion, like for some consumption sensors.

Monofin commented 2 years ago

Hi there - it looks as if there is a (faintly) possible race condition between the event arrival, the processing of the IAS data, and the population of the cluster information that is causing the test failure. I've finally got the build going and some debug happening :-)

Monofin commented 1 year ago

Appears to be fixed by #6398, tested as working for Tuya TS0202 for that PR commit. Enrolment now seems fully dealt with when triggered by incoming ZCL event.

Forcing fingerprint not necessary for 0x500 cluster using device.