Enhance robustness of TwinGuard measurment handling on weak wireless connections

[MarkJonas]

I have a SHC with 5 TwinGuards connected. One of the TwinGuards has better and worse days in regards of the wireless connection to the gateway. In the official Bosch Smart Home app the reading during a wacky connection will look like this:

In OpenHAB it will look like this:

The good: There seems to be more valid data than the Bosch Smart Home app shows.

The bad: OpenHAB receives bad data the Bosch Smart Home app is able to filter out.

It would be great if the binding could also filter out the bad data.

[MarkJonas]

I can now see the issue on a second TwinGuard. Here also there are "drop spikes", i.e. much lower values than there should be. It seems as if there is one reading per minute as each drop is exactly one minute wide. The drops happen simultaneously for temperature, humidity, and purity. The major difference this time is that the Bosch Smart Home app does not show data loss and the data is still clean.

I am wondering if - as a workaround - I could do a median filter over 5 (or more) minutes. A new reading every five minutes instead one every minute would still be good. But right now I am still too unexperienced with OpenHAB to know how to do this.

[MarkJonas]

Here an example where 4 of 5 TwinGuards are affected. The Smart Home App does not show an interruption. The TwinGuards are situated in a condominion with hollow brick walls. Only the TwinGuard without interruption is in line of sight with the gateway.

[coeing]

@MarkJonas Could you post the logs (with DEBUG logging enabled) around the time when the drops happen? This way I can have a look at the raw data which was send by the controller. Maybe there is a flag or something like that which indicates that the data is invalid. Otherwise it may be impossible for our binding to decide which data is valid and which is not.

[MarkJonas]

@coeing OK, I just re-enabled the DEBUG logs. I'll now let it run for 24 hours and then send you the logs and a screenshot to locate the problematic times.

Edit: I just see that there is v1.0 Beta 6. So I will immediately upgrade to this version.

Edit 2: After updating the binding I now have both Beta 5 and Beta 6 in the system. I already removed the Beta 5 .jar from the addons directory and restarted openHAB. But that did not help. I hope this will not create problems. How do I remove the outdated versions of the binding?

232 │ Active │  80 │ 3.0.0.202012261941      │ org.openhab.binding.boschshc
233 │ Active │  80 │ 3.1.0.202101110957      │ org.openhab.binding.boschshc

[tuxianerDE]

@coeing With the latest build 3.1.0 I dont observe these drops in the graph anymore. So far it seems more stable without any drops.

[coeing]

@coeing OK, I just re-enabled the DEBUG logs. I'll now let it run for 24 hours and then send you the logs and a screenshot to locate the problematic times.

Edit: I just see that there is v1.0 Beta 6. So I will immediately upgrade to this version.

Edit 2: After updating the binding I now have both Beta 5 and Beta 6 in the system. I already removed the Beta 5 .jar from the addons directory and restarted openHAB. But that did not help. I hope this will not create problems. How do I remove the outdated versions of the binding?

232 │ Active │  80 │ 3.0.0.202012261941      │ org.openhab.binding.boschshc
233 │ Active │  80 │ 3.1.0.202101110957      │ org.openhab.binding.boschshc

You should uninstall the old version with bundle:uninstall 232, see https://github.com/stefan-kaestle/openhab2-addons/issues/62#issuecomment-759351999 how another user solved the issue.

[MarkJonas]

@coeing I uninstalled the old bundle, restarted openHAB and now there is only the newest version of the binding.

I'll let it run for 1-2 days and then report back in regards of the drops. @tuxianerDE's experience sounds promising.

[tuxianerDE]

@coeing - It seems that it is stalling again at least I can see that the graph dont obtain new data anymore.

P.S. and I checked the temperature change in my sleeping room ;)

[coeing]

@MarkJonas Did you have time to check out if the drops occur in the latest version (v1.1) of our binding? If so, let me know, otherwise I will close the issue for now.

[MarkJonas]

@MarkJonas Did you have time to check out if the drops occur in the latest version (v1.1) of our binding? If so, let me know, otherwise I will close the issue for now.

I missed the v1.1 release. I'll give it a try on the weekend.

[MarkJonas]

@MarkJonas Did you have time to check out if the drops occur in the latest version (v1.1) of our binding? If so, let me know, otherwise I will close the issue for now.

I missed the v1.1 release. I'll give it a try on the weekend.

I installed v1.1-rc1 yesterday. Today I was able to verify that the problem is still still there. That is, sometimes temperatures are dropping close to 0 °C.

[coeing]

@MarkJonas Did you have time to check out if the drops occur in the latest version (v1.1) of our binding? If so, let me know, otherwise I will close the issue for now.

I missed the v1.1 release. I'll give it a try on the weekend.

I installed v1.1-rc1 yesterday. Today I was able to verify that the problem is still still there. That is, sometimes temperatures are dropping close to 0 °C.

Thanks for verifying the issue! :) Did you have Debug or Trace logging enabled when the issue occurred? And could you send us your log, so we have a chance to find out what is causing the issue?

[MarkJonas]

@coeing @GerdZanker I sent the logs to you by email. Log level is TRACE.

[coeing]

Thanks to the logs from @MarkJonas I found the following lines:

2021-03-05 10:47:50.093 [DEBUG] [devices.bridge.BoschSHCBridgeHandler] - Found child: org.openhab.binding.boschshc.internal.devices.twinguard.BoschTwinguardHandler@152bece - calling processUpdate with {"smokeSensitivity":"MIDDLE","@type":"smokeSensitivityState","preAlarmEnabled":true}
2021-03-05 10:47:50.095 [DEBUG] [ices.twinguard.BoschTwinguardHandler] - Twinguard: received update: SmokeSensitivity {"smokeSensitivity":"MIDDLE","@type":"smokeSensitivityState","preAlarmEnabled":true}
2021-03-05 10:47:50.098 [DEBUG] [ices.twinguard.BoschTwinguardHandler] - Parsed switch state of hdm:ZigBee:xxx: org.openhab.binding.boschshc.internal.devices.twinguard.dto.AirQualityLevelState@1d15e7c

This indicates that the Twinguard handler casts a smokeSensitivityState update to an AirQualityLevelState. This would cause the temperature (and other) fields to have their default value (which is 0 for numbers) and would explain the observed bug.

I will try to find time to look into the handler this week to fix the issue. @MarkJonas I will probably have to send you a version with the fix you have to test as I do not own a Twinguard device.

[coeing]

@MarkJonas I fixed the check if the received state has the expected type. This should make sure that no invalid state is passed to the handler. Please test with this version: https://github.com/stefan-kaestle/openhab2-addons/releases/tag/v1.1-bugfix.66.1

You should see some "Received invalid, expected type org.openhab.binding.boschshc.internal.devices.twinguard.dto.AirQualityLevelState" warnings in your log whenever a smokeSensitivityState was received. But at least the state should not update any of the channels of your thing.

[coeing]

As far as I see it you should not even get a log warning. The TwinguardHandler was changed to use the new service architecture which checks the ID of the received state. The ID is the service name and a service only processes their own state updates, so my additional check should never fail. Nonetheless, please test the version I provided so we can be sure that your bug is fixed :)

[MarkJonas]

@coeing Thank you very much for the updated version. I just installed it. The plan is to let it run for ~36 hours and then check for success. So far there was always at least one even during a period of 24 hours.

236 │ Active │  80 │ 3.1.0.202103191712      │ org.openhab.binding.boschshc

[MarkJonas]

@coeing Thank you very much for the updated version. I just installed it. The plan is to let it run for ~36 hours and then check for success. So far there was always at least one even during a period of 24 hours.

Until now it is looking good. :smile:

[MarkJonas]

@coeing There are still no misreadings, neither at temperature nor at humidity. So I think the bug has been successfully resolved. :1st_place_medal: