SensorPush sensors not updating in Home Assistant

[bifurcated]

The problem

I have a SensorPush HTP.xw that, prior to updating to either 2022.8.2 or 2022.8.3 (not sure which), was updating values in HA on a frequent basis and working just fine.

Without changing the location of that sensor (and thus introducing possible connectivity issues), it began to update very infrequently, to the point where the actual temperature/humidity/pressure readings were substantially different from what HA was reporting. I confirmed that the SensorPush app was reading correctly, and was accurate.

I enabled debug for bluetooth. Updates appear to be sent to HA, but not recorded. In an 8 hour period today, HA has received 477 updates from the sensor, but 477 data points are not logged in the sensor entity.

Here's summary proof of the updates being received by HA:

[core-ssh config]$ grep SensorPush home-assistant.log | grep advertisement_data I wc -l 477 [core-ssh config]$

I've attached a screen cap of the sensor graph, as I wasn't sure how else to show that the updates weren't recorded.

My 2 Inkbird IBS-TH2 sensors are exhibiting similar behavior, with 1081 updates sent in a similar period, and few apparent recorded updates.

In the event that it is useful information, my Govee BLE sensors, of which I have > 10, are working perfectly.

I'd be happy to capture more debug information if you can tell me what you would like to see.

What version of Home Assistant Core has the issue?

2022.8.2+

What was the last working version of Home Assistant Core?

2022.8.1

What type of installation are you running?

Home Assistant OS

Integration causing the issue

SensorPush

Link to integration documentation on our website

https://www.home-assistant.io/integrations/sensorpush

Diagnostics information

No diagnostics file available, but here's a screenshot of the sensor graph.

.

Example YAML snippet

No response

Anything in the logs that might be useful for us?

An example of one SensorPush log entry from the bluetooth debug:

2022-08-13 10:15:42.611 DEBUG (MainThread) [homeassistant.components.bluetooth] Device detected: 58:93:D8:C8:92:41 with advertisement_data: AdvertisementData(local_name='SensorPush HTP.xw 241', manufacturer_data={17667: b'\xad\x00\x01\x00\x00', 24064: b'6[:\xbb\xb6', 1280: b'\xe7\xb4\xe1\xaf\xb6', 2304: b'7\xe1:\xb7\xb6', 55552: b'#\xc7$\xad\xb6', 58624: b';\x01%\x9d\xb6', 44288: b"\xa2|'x\xb6", 64000: b';\xad\xdc\xa7\xb6', 28672: b'\xdb\xe8\\\xa2\xb6', 7168: b'\xb5\xbd\xe0\xaf\xb6', 11264: b'\x00S}\xae\xb6', 4096: b'\xe9\xef\x8e\xba\xb6', 44800: b'\x85\xa2=\xb5\xb6', 32768: b'\x86b\xe9\xc1\xb6', 37376: b'\x8bS<\xc1\xb6', 25344: b'\xb4\xb2\xe7\xbb\xb6', 51200: b'\xae\xdc\xc8\x97\xb6', 49152: b'O\x80O\xc7\xb6', 17664: b'\x0e\xb7q\xa0\xb6', 34816: b'\x9a\xf6\xf8\xc3\xb6', 21760: b'G\xd9\xd6\xa7\xb6', 512: b'\xaa\x14M\xc5\xb6', 41984: b'\xfd\xb4\xd7\xa5\xb6', 16640: b'\x9b\xdd\xd9\xa5\xb6', 33024: b'\x99\xdbB\xb9\xb6', 25088: b'\xee\xec\xea\xbf\xb6', 24576: b'\xc3G\x16\x99\xb6', 50176: b'\x88Q\x9d\xc4\xb6', 57856: b'~\x1a\xb0\x87\xb6', 2816: b'08\xa2\xc4\xb6', 19712: b',\xf1u\x9a\xb6', 26880: b'\x8f\x0f(\xa9\xb6', 54528: b'U\xbe\x1c\x9b\xb6', 7936: b'\x01\x1e\x93\xbc\xb6', 52992: b'R\x19\xb9\x91\xb6', 9472: b'\x0f\xb9\x9a\x87\xb6', 47360: b'A\xe16\xb5\xb6', 14080: b'r\x82S\xc7\xb6', 60416: b'#A\xc5v\xb6', 19968: b'\xf5=\x80\xa0\xb6', 30976: b'\r\x99\x13\x91\xb6', 9216: b'\x08">\xbd\xb6', 16896: b'"\x94L\xc7\xb6', 54784: b'\xae\xce%\x9d\xb6', 21248: b'\xb9\xe9\xe1\xb9\xb6', 40960: b'\x15}\xda\xbb\xb6', 16128: b's\xe9\xf7\xc5\xb6', 36608: b'\xad\xd6\x8f\xc0\xb6', 1536: b'\x1a\xd1\x8c\xb0\xb6', 30720: b'\xf4`\x93\xb4\xb6', 17920: b'mIi\xae\xb6', 30464: b'\x8c}\x19\x99\xb6', 61952: b'\xb4{\xec\xbd\xb6', 30208: b'\xa8\xac"\x9b\xb6', 27904: b'D\xcb8\xb5\xb6', 45568: b'\xfc\xb5\xdf\xa9\xb6', 12288: b'\xe9\x11\xa7\x8f\xb6', 6400: b'\\\xcf\xe0\xb7\xb6', 10496: b'P_\xe1\xbb\xb6', 52736: b'fv\xd3\xa1\xb6', 37888: b"\xb1\x7f'\xaf\xb6", 6656: b'\x80Wh\x90\xb6', 15872: b'\xd7\x91\xe0\xb7\xb6', 28160: b'P<\xc5\x95\xb6', 37632: b'NN\xc7x\xb6', 11776: b'\x03z0\xab\xb6', 48896: b'B\x9e\xaa\xc8\xb6', 65280: b'w\xb1\xee\xb9\xb6', 56320: b'\xb1\xfa\x1f\x99\xb6', 59136: b'_\xd5\x1c\x97\xb6', 26368: b'\xbe\x82\xbd\x93\xb6', 7424: b'A\xc8\x19\x99\xb6', 49408: b'\xef\xda\x91\xb4\xb6', 24832: b'l\xc03\xbd\xb6', 48128: b'Vs4\xa9\xb6', 48384: b'\xack;\xbb\xb6', 20224: b'\xd8O\xe5\xb9\xb6', 35840: b'Nj\xe1\xbb\xb6', 51712: b'\x96\xba\xcc\x9b\xb6', 23296: b'\xda\\v\x9c\xb6', 39168: b'0j\xe3\xb3\xb6', 29440: b'\xf9\xc9J\xc3\xb6', 54016: b'\xe9\x1c\x88\xa6\xb6', 62208: b'\x1b\x0f\xe3\xbf\xb6', 33280: b'\xc2s\x83\xa2\xb6', 20480: b'\xa9\xc5\xc4\x95\xb6', 50688: b'\xd5O\xe5\xb7\xb6', 19456: b'T }\x9e\xb6', 27136: b'\xd3\n\xda\xbb\xb6', 34304: b'\x10\x164\xb7\xb6', 3328: b'"\xb1\x1c\x99\xb6', 50944: b'it\xbf\x91\xb6', 29952: b'\xd7\xc5\xb8\x93\xb6', 46592: b'\x14-\xbc\x95\xb6', 60928: b'|\xcd\xb8\x8d\xb6', 16384: b'4\x95\xce\x9b\xb6', 23040: b'\x99\xca\x9f\x8d\xb6', 58112: b'P\xcc;\xb7\xb6', 22784: b'\x8a4L\xc5\xb6', 12800: b'el\xe0\xad\xb6', 8960: b'xe\x8e\xb8\xb6', 13568: b'\xec2\x8f\xb8\xb6', 36864: b'\r\xde1\xb5\xb6', 64512: b'\xf7\xf8\x17n\xb6', 39424: b'?\xbc\x87\xa8\xb6', 8448: b'\xfa\x8c\xa6\x8f\xb6', 53760: b'\xf3\x92\xdd\xb3\xb6', 23552: b'A\xb5A\xc3\xb6', 51968: b'\xb6\xc9\xa5^\xb6', 9728: b'\xff\xa1\x7f\xa6\xb6', 18944: b'\xc0\xddI\xc1\xb6', 46848: b'\x05t\xea\xb9\xb6', 33792: b'\xdb\xa8\xd9\xa3\xb6', 6144: b'+\xcb?\xb9\xb6', 10752: b';\x93:\xb9\xb6', 40704: b'\x8e\x85p\x96\xb6', 58368: b'\x91\xf2\xd0\x9d\xb6', 32512: b'\xec\x80\x85\xa4\xb6', 55808: b'-\x98\x80\xb0\xb6', 25856: b'\x90\xd5\x85\xaa\xb6', 58880: b':J\x81\xba\xb6', 31232: b'\x80\xfe\xdd\xa5\xb6', 55040: b'o13\xa9\xb6', 50432: b't\xe5I\xc3\xb6', 37120: b'\xd3\x05\x89\xa6\xb6', 12544: b'\x06\x00<\xbd\xb6', 59904: b'\xddb\xbe\x93\xb6', 27392: b'OB\x0f\x8f\xb6', 61696: b'\x1d\xe8\x18\x97\xb6', 29696: b'#\xcc\xde\xbd\xb6', 32000: b'X}3\xb5\xb6', 44544: b'\xb8\xa1\x1e\x99\xb6', 7680: b'\xe7Qr\x98\xb6', 45312: b'\xfbI\x10\x8f\xb6', 63488: b'\xc6\xda(\xa5\xb6', 25600: b'O\xda\xe2\xb7\xb6', 24320: b'r\x14n\x98\xb6', 62464: b'\xb0\x87\xf4\xc1\xb6', 63744: b'\x96\xd6\x14\x95\xb6', 21504: b'[\x85\x0f\x93\xb6', 8192: b'\xb7\x84\xd3\xa5\xb6', 29184: b'\xbf\xdfg\x90\xb6', 64768: b'\xa2\x84\xe5\xbf\xb6', 57088: b'9\t\x8a\xb4\xb6', 22272: b'r~(\x9f\xb6', 55296: b'.\x03\xc6\x97\xb6', 34560: b'\xb5r\x7f\xa0\xb6', 52224: b'\xe2\xc3\x1c\xa3\xb6', 13824: b'8>\xe6\xb5\xb6', 46080: b'Y\x7f@\xb9\xb6', 34048: b'/_k\x96\xb6', 4608: b'9\x95K\xc3\xb6', 62720: b'K-;\x84\xb6', 44032: b'\xd5\xd0\xa7\xc6\xb6', 35584: b'?}D\xcd\xb6', 43008: b'2\x8f\x8a\xae\xb6', 47104: b'\xa1\xff\xe6\xbb\xb6', 61184: b'\xa3\x7f%\xa5\xb6', 59648: b'\xf1\xb8\x8d\xb4\xb6', 57344: b'\x88\xee2\xb7\xb6', 36096: b'\\\xd5\x9c\xc0\xb6', 38912: b'n\x12_\x90\xb6', 56832: b'$gG\xc3\xb6', 18176: b'\xf9\x96\xfc\xc5\xb6', 18432: b'b\xcdA\xbf\xb6', 57600: b':\x19@\xbf\xb6', 18688: b'$\xe2\xcb\x99\xb6', 38656: b'\x0cA-\xb9\xb6', 48640: b'V\x8c\xda\xab\xb6', 46336: b'"WL]\xb6', 9984: b'\xa8\xab\xa2\xd2\xb6', 42496: b'4\x0b\x1f\x9b\xb6', 41216: b')M%\xab\xb6', 49664: b'M%6\xa9\xb6', 42240: b',\x1e\x86\xb6\xb6', 20992: b'\xab\x052\xbd\xb6', 53504: b'\x8a\xf6\x84\xaa\xb6', 56064: b'\xda\xbf\xa4`\xb6', 53248: b'\x18:\xc8\x99\xb6', 19200: b'\xb1\xb4\x89\xbc\xb6', 38400: b'\xba=\x1f\x99\xb6', 41728: b'\xe4\xa3+\xb1\xb6', 5376: b'z\xd6\x94\xb2\xb6', 47616: b'\x88\x1f\xe3\xb9\xb6', 60672: b'\x9c\x85{\xb4\xb6', 3584: b'\xe7\xdc\xa8\xc6\xb6', 28416: b'\xdc\xddT\x90\xb6', 14336: b'\x87\xa6\xf2\xc5\xb6', 43776: b'9y\x8a\xae\xb6', 39936: b'\xe2\x8cSa\xb6', 5632: b'\xa5_0\xab\xb6', 14592: b'\xbf\xa9\x80\xae\xb6', 63232: b'\xd6A\xc5\x99\xb6', 13312: b'=\xcdL\xc3\xb6', 8704: b"\xf9\xd1'\xa1\xb6", 11008: b'\xdc\xed\xf6\xc5\xb6', 26624: b'\x9b\x81\xc2\x99\xb6', 13056: b'\x88@\xda\xab\xb6', 5888: b'p\xea\x85\xaa\xb6', 12032: b'L\xdb\xe9\xb9\xb6', 3072: b'$\x1e\x83\xac\xb6', 31744: b'\xcb\xe60\xad\xb6', 14848: b'\xee\x9d\xe8\xc9\xb6', 45824: b'Mo\x8e\xb2\xb6', 768: b'\xa6\x8d=\xb5\xb6', 56576: b'\x02\xba\x8d\xb0\xb6', 49920: b'\xa3\xac$\x9d\xb6', 41472: b'\x9dM\xe0\xab\xb6', 65024: b'\x89!\xf2\xbf\xb6', 1024: b'\x89\xbf\x8d\xb4\xb6', 0: b'\xb6\xc5\xc2\x97\xb6', 61440: b'\xad\xa8s\x98\xb6', 17408: b'\xc2\x99?\xbb\xb6', 42752: b'R\xf81\xa9\xb6', 38144: b'\x83\x89"\x9d\xb6', 43520: b"\xb7\xa2'\x9f\xb6", 35328: b';d\xa2\xd0\xb6', 51456: b'\xa4\x85h\x90\xb6', 35072: b'\xfb\x90@\xbf\xb6', 39680: b'\xf5\xcb\x04\xa1\xb6', 4352: b'j\xd0e\x92\xb6', 32256: b'\xcc\x99\xbf\x95\xb6', 3840: b'\xd0\xdd\xc7\x99\xb6', 45056: b'U\xf2\xf0\xc3\xb6', 47872: b'\xdc\x07"\x9b\xb6', 60160: b'0\x8a\xdf\xbb\xb6', 28928: b'\xe7\xa8\xdc\xaf\xb6', 54272: b'c\x15\x85\xb4\xb6', 17152: b'\xc0Q\x7f\xa0\xb6', 5120: b'B@w\x9a\xb6', 43264: b'rC\x85\xaa\xb6', 23808: b'[\xe3=\xb7\xb6', 256: b'\x9c\x9f\x90\xb2\xb6', 6912: b'\xf2\x18\xdc\xab\xb6', 15616: b'\x1b,/\xb5\xb6', 15104: b'{=\xbf\x91\xb6', 4864: b'g?\xe3\xb9\xb6', 36352: b'\x88\xac2\xad\xb6', 22016: b'O\x91\x18\x95\xb6', 52480: b'q\x8dS\xc9\xb6', 62976: b'ZX\x7f\xa2\xb6', 59392: b'\xef\xdc\xa5\xc4\xb6', 15360: b'\xd0\x9aD\xc1\xb6', 10240: b'a\x92\x92\xb0\xb6', 2048: b' /]\x9a\xb6', 20736: b'\x9d\xdek\x94\xb6', 2560: b'\xf5z=\xb3\xb6', 22528: b'j@\xe2\xad\xb6', 26112: b'\x18\x1f\xc5x\xb6', 40448: b'\xdf\xfe=\xbb\xb6', 11520: b'2\xf7<\xbb\xb6', 1792: b'$\n\x1c\x99\xb6', 40192: b'\xaa\x88\xff\xc9\xb6', 27648: b'\x87\xac\xb8\x8d\xb6', 33536: b'{:\x1b\x97\xb6', 64256: b'B\r.\xa9\xb6', 31488: b'\x98\xfa\xb6\x91\xb6'}, service_uuids=['ef090000-11d6-42ba-93b8-9dd7ec090ab0']) matched domains: {'sensorpush'}

An example of one InkBird log entry:
2022-08-13 10:18:10.341 DEBUG (MainThread) [homeassistant.components.bluetooth] Device detected: 49:22:05:17:03:8F with advertisement_data: AdvertisementData(local_name='tps', manufacturer_data={63556: b'\x00\x00\x00\xe5od\x06', 63566: b'\x00\x00\x00}nd\x06', 63576: b'\x00\x00\x004\xadd\x06', 63586: b'\x00\x00\x00\xec\xa8d\x06', 63596: b'\x00\x00\x00\x85i`\x06', 63606: b'\x00\x00\x00\xdc\xabd\x06', 63616: b'\x00\x00\x00\x14\xbed\x06', 63626: b'\x00\x00\x00\x8c\xbfd\x06', 63636: b'\x00\x00\x00$\xbdd\x06', 63526: b'\x00\x00\x00\x1c\xa7d\x06', 63646: b'\x00\x00\x00\xbc\xbcd\x06', 63506: b'\x00\x00\x00\xadcd\x06', 63496: b'\x00\x00\x00\xf4\xa1d\x06', 63486: b'\x00\x00\x00?\xa0d\x06', 63476: b'\x00\x00\x00\xa7\xa1d\x06', 63426: b'\x00\x00\x00o\xa5c\x06', 63286: b'\x00\x00\x00\xdep`\x06', 63406: b'\x00\x00\x00\xff\xacd\x06', 63396: b'\x00\x00\x00g\xadd\x06', 63386: b'\x00\x00\x00Nhd\x06', 63376: b'\x00\x00\x00\xd6id\x06', 63366: b'\x00\x00\x00\x9f\xaad\x06', 63316: b"\x00\x00\x00'\xb8d\x06", 63326: b'\x00\x00\x00\xbf\xb9d\x06', 63336: b'\x00\x00\x00w\xbdd\x06', 63346: b'\x00\x00\x00.\x7fd\x06', 63356: b'\x00\x00\x00G\xbed\x06', 63436: b'\x00\x00\x00\x06dd\x06', 63546: b'\x00\x00\x00\xcded\x06', 63676: b'\x00\x00\x00D\xbbd\x06', 63686: b'\x00\x00\x00\x9dqd\x06', 63696: b'\x00\x00\x00\xd4\xb2d\x06', 63746: b'\x00\x00\x00m\\d\x06', 63736: b'\x00\x00\x00\xb4\xb4d\x06', 63726: b'\x00\x00\x00\xfdwd\x06', 63666: b'\x00\x00\x00-zd\x06', 63656: b'\x00\x00\x00t\xb8d\x06', 63536: b'\x00\x00\x00Udd\x06', 63516: b'\x00\x00\x00\xc4\xa2d\x06', 63416: b'\x00\x00\x00\xb6od\x06', 63446: b'\x00\x00\x00_\xa6d\x06', 63276: b'\x00\x00\x00\x87\xb2`\x06', 63456: b'\x00\x00\x00\x97\xa2d\x06', 63466: b'\x00\x00\x00\x0f\xa3d\x06'}, service_uuids=['0000fff0-0000-1000-8000-00805f9b34fb']) matched domains: set()

Additional information

No response

[probot-home-assistant[bot]]

sensorpush documentation sensorpush source _{^{(message by IssueLinks)}}

[probot-home-assistant[bot]]

Hey there @bdraco, mind taking a look at this issue as it has been labeled with an integration (sensorpush) you are listed as a code owner for? Thanks! _{^{(message by CodeOwnersMention)}}

[bifurcated]

I should also mention that I've power cycled the SensorPush sensor, moved it closer to my Bluetooth modules, and reloaded the integration from the Integrations dashboard with no effect.

[bdraco]

Both inkbird and sensorpush rely on the insertion order of the mfr data on linux because both of these vendors abuse the mfr data fields.

On linux it probably fails once the same mfr id gets reused.

[bdraco]

It probably works fine on macos because bleak only has this problem with bluez

[bdraco]

We likely need a better way to detect new data on linux with these devices

[bdraco]

I'm not sure we can fix this since bleak doesn't give us access to the underlying packets.

[bdraco]

We will need all the Bluetooth debug log entries for the device from before and after the data goes wonky to see if we can come up with a fix

[bifurcated]

Unfortunately, I don't have logs from before the data went wonky. I could stand up another HA instance and try to replicate the problem with existing sensors.

[bdraco]

If you reboot the host it should normalize for a few hours before it gets into a bad state again. If you collect from startup you should have enough within a day

[bifurcated]

OK, I'll give that a shot.

[bifurcated]

Here's a log from last night to this morning, where both SensorPush and InkBird were working and at the end only SensorPush is working. Govee works throughout, best I can tell.

I included the Govee sensor log data in case it was useful, but grepped out a bunch of obvious noise from other random bluetooth devices in the air, erring on the side of inclusion.

Based on the sensor graph, the first InkBird stopped recording around 1:24:31 and began behaving again briefly between 2:38:06 and 2:49:32. The other InkBird began glitching around 2:09:07.

homeassistant.components.bluetooth.20220814.log.gz

[bdraco]

Thanks. I'll try to come up with a way to strip out the garbage data for sensorpush first. If that works out we can apply it to Inkbird as well

[bifurcated]

I'm updated to 2022.8.5 and things appear to be working well thus far after 12+ hours. Thanks for the quick fix! I'll update if the problem isn't ultimately resolved.