home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
73.65k stars 30.79k forks source link

Inkbird IBS-TH2 'noisy' data #85432

Closed Anto79-ops closed 1 year ago

Anto79-ops commented 1 year ago

The problem

Good day,

I recently acquired 2 Inkbird's, IBS-TH2, and placed them in my fridge and freezer. The devices almost immediately showed up on HA upon adding batteries to it. I have my HA Raspberry Pi instance and 7, ESP32 BT proxies (enabled for both active and passive scanning from ESPHome). I noticed however, the values that InkBird's (both devices) report is quite noisy and temperatures can jump up to 20oC sometimes, but its mostly only 5oC, for example.

image

When using the manufactures app (Engbird) the plots to not look like this, just your normal smooth line (its not clear if the manufacturer is applying smoothing).

I've seen similar reports about this, such as:

https://github.com/home-assistant/core/issues/76602

and

https://community.openmqttgateway.com/t/inkbird-ha-integration-values/1939

Not sure if this is something to do with my BT setup or some code issue, hence I'm just posting here.

Both my Inkbird sensors exhibit this behavior in HA. and it seems to be doing this with all entities as the same is true for humidity readings, and signal strength readings. Battery is at 100% so the line is straight.

thanks for your time

What version of Home Assistant Core has the issue?

2023.1.1

What was the last working version of Home Assistant Core?

not sure

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Inkbird

Link to integration documentation on our website

https://www.home-assistant.io/integrations/inkbird/

Diagnostics information

just noisy plots, no errors in regular logs

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

home-assistant[bot] commented 1 year ago

Hey there @bdraco, mind taking a look at this issue as it has been labeled with an integration (inkbird) you are listed as a code owner for? Thanks!

Code owner commands Code owners of `inkbird` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Change the title of the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign inkbird` Removes the current integration label and assignees on the issue, add the integration domain after the command.

(message by CodeOwnersMention)


inkbird documentation inkbird source (message by IssueLinks)

david0161 commented 1 year ago

I have the same issue with 20F+ spikes using ESP BT Proxy. It seems on ocassion my sensors will stop updating on a spike, unsure if a connection issue. 3 in garage (one in deep freezer, one in fridge, one in fridge freezer). BT Proxy is within 10 ft of any sensor. Have not checked the inkbird app yet, will try that and report back on that data.

Anto79-ops commented 1 year ago

@david0161 thanks for bringing your issue here. When a developer get a hold of this issue, I'm sure it will be fixed.

By the way, the app for IBS-TH2 is actually called Engbird (by InkBird), its pretty terrible but it works. You'll likely see smooth plots in the app as I did, as I'm starting this think the noise we are seeing could be related to the ESP32 proxies just needing some minor tweak.

Yonghui commented 1 year ago

I too have the noisy value problem. esphome's built-in inbird component doesn't have the problem with the same device.

esphome built-in inkbird component provide a separate sensor for external temperature. https://github.com/esphome/esphome/blob/dev/esphome/components/inkbird_ibsth1_mini/inkbird_ibsth1_mini.cpp

// sensor output encoding // data[5] is a battery level // data[0] and data[1] is humidity 100 (in pct) // uuid is a temperature 100 (in Celsius) // when data[2] == 0 temperature is from internal sensor (IBS-TH1 or IBS-TH1 Mini) // when data[2] == 1 temperature is from external sensor (IBS-TH1 only)

bdraco commented 1 year ago

That would explain the issue, does anybody with a sensor that's affected feel like opening a PR to the library?

Anto79-ops commented 1 year ago

@bdraco I'd be happy to open an issue...but to be clear as I don't understand the probllem...is this an ESPHome issue?

If so, I can post the issue here, and reference what @Yonghui is saying above

https://github.com/esphome/issues/issues

thanks

bdraco commented 1 year ago

Looking for someone with the device who has the issue to port the code from the esphome component to the lib. I don't have one of the devices with the issues so it's not something I can replicate or validate a fix.

Anto79-ops commented 1 year ago

If someone more technical than me would like to do this, that would be great otherwise I can do this with some guidance. Thanks.

Anto79-ops commented 1 year ago

Hi, been asking around to try and solve this, but have not been successful in figuring out how to port code. sorry about that.

bdraco commented 1 year ago

Since I only have the th1, I ordered the th2. Hopefully it shows up before I leave for a few weeks. If not I'll see if I can replicate the issue in March when I get back

Anto79-ops commented 1 year ago

oh, you ended up getting one...that's a dedicated developer!

bdraco commented 1 year ago

Just installed it and not seeing the spikes (yet)

Can you turn on debug logs for homeassistant.components.bluetooth and capture a good and bad AdvertisementData?

Jc2k commented 1 year ago

When I first put mine in the freezer I was seeing spikes quite a lot. However they seem much rarer now. I can only see a couple over a week. Which is going to make capturing a good and bad hard in prod.

Anto79-ops commented 1 year ago

Oh that is interesting maybe the fridge has something to do with it? RSSI ranges from -75 to -85 not the greatest signal, for sure.

It might take up to 2 hrs for the issue to appear, but you'll know soon enough if it happens.

To give you more information about my BT system, I have 7 ESP32 proxies, 1 Shelly Plus, and 1 HAOS RPi hcio :

For my ESP32's, here is the config files for ALL ESP32 (and the Shelly is active active also).

substitutions:
  name: esp32-bluetooth-proxy-xxxxxx
packages:
  esphome.bluetooth-proxy: github://esphome/bluetooth-proxies/esp32-generic.yaml@main
esphome:
  name: ${name}
  name_add_mac_suffix: false

esp32_ble_tracker:
  scan_parameters:
    interval: 1100ms
    window: 1100ms
    active: true

bluetooth_proxy:
  active: true

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

sensor:
  # Uptime sensor.
  - platform: uptime
    name: esp32-bluetooth-proxy-xxxxxx.uptime
  - platform: template
    name: esp32-bluetooth-proxy-xxxxxx.free_memory
    lambda: return heap_caps_get_free_size(MALLOC_CAP_INTERNAL);
    unit_of_measurement: 'B'
    state_class: measurement

I placed this in my configuration.yaml and restarted HA:

logger:                       
  default: info               
  logs:                      
    homeassistant.components.bluetooth: debug

The log file was getting huge, in 11 minutes it got up to 16,000 kB so I only collected for 11 minutes. For good and bad data, its a little more tricky, for example, here is a view for 1 of the sensors:

image

and here's while collecting logs in the 11 minute window (this should be in the logs):

image

In the 11 minute snippet, sure, there's some outliers, but not sure if that would be considered bad data?

@bdraco I have the log files, but, if you don't mind, can I send it to you, personally? I thought I saw some device keys in there, feel free to share the BT related issues in the forum.

Anto79-ops commented 1 year ago

Hi All,

Just wanted to provide and update. I took out a temperature sensor from my fridge and just left it on my counter to warm up to rule out anything to do with my fridge.

What is interesting about this, is its almost as if the there is a delay in reporting data, for example, because fridge temperature cycles on an off, therefore the temperature goes up and and down by a few degree, the noisy data is within the peaks and troughs of this, a shown here:

image

Notice in the above, the extra peaks are within the peak-trough hieght.

Now, in this test, where I took out the sensor from the my fridge below:

image

You'll see to massive outliers, but if those outliers were reported say 30 minutes ago, it would be valid.

Is it possible that data packets are just being held back, somehow, queued and then reported at a later (but wrong) time? Too much data, maybe?

bdraco commented 1 year ago

The esphome device does have a queue but it should always be processed the next loop so milliseconds lag would be the most expected.

I bought a second one of these devices and have had no luck replicating the issue

Anto79-ops commented 1 year ago

Thanks Nick for trying to figure this one out. Just curious if you have tried placing it in your freezer and then taking it out? the noise seems to present when the temps change, often, especially large temp swings.

thanks again

bdraco commented 1 year ago

My freezer is out of range. I'll build another proxy tomorrow and get it close enough

nsthompson commented 1 year ago

I put an Inkbird IBS-TH2+ in my garage freezer about a week ago or so, and I see a similar amount of noise.

inkbird-noise

I have another sensor I just pulled out of the package and will leave inside to see if it has a similar type of issue as well.

bdraco commented 1 year ago

Proxy installed, device moved to freezer... waiting for data

Screenshot 2023-01-31 at 3 46 26 PM
bdraco commented 1 year ago

Sadly it looks like my freezer is blocking the bluetooth signal as it went unavailable shortly after putting it in

Anto79-ops commented 1 year ago

Hey Nick thanks again for trying this.

Maybe you can try this: place the temperature sensor in the freezer anyways for a few hours let it cool down and then take it out and put it beside a Bluetooth proxy you'll get the warming trend that should show some spikes if yours is affected by the same issue. I know the test won't be exactly the same but if you see something then you'll know.

bdraco commented 1 year ago

I was able to get the proxy close enough to the freezer that it could read it even though the freezer seems quite good a blocking the signal.

Screenshot 2023-01-31 at 9 52 53 PM

Now I'm waiting for the problem to happen

david0161 commented 1 year ago

Don't want to foul any data but I have 4 of these running for about 3 weeks. BT Proxy hanging above garage deep freezer, assuming garage fridge, garage fridge freezer on the proxy. Connectivty/reporting wise they seem a lot better now. Still have an issue with one sensor 4 ft from a BT Proxy that won't update for 6-8hrs typically in the kitchen, should only be an inkbird and a switchbot. Still not sure how to see what is connected to what. Here are my graphs over the past week. garage fridge fridge fridge freezer deep freezer kitchen fridge

bowentz commented 1 year ago

I've been following this thread and see some interesting info in my set up with these devices. I have a bluetooth proxy reporting the sensor data as well as using ESPHome (no proxy) to collect the data from the same device. These are on different ESP32s, so I don't think there is any type of problem with bluetooth resources... The screenshots below are for the one I have on my back porch, but I have another in the freezer that has the same reporting pattern. I'm not sure if this helps, but here is what I see..

This one is from the proxy: porchbluetoothproxy

This one is from the ESPHome device: porchesphome

bdraco commented 1 year ago
Screenshot 2023-02-01 at 1 13 28 PM

Mine seems to still be going strong.

Can you post your esphome yaml?

bowentz commented 1 year ago

https://pastebin.com/nHzVqKm0

bdraco commented 1 year ago

Add this and reflash with a serial cable

esp32:
  board: YOUR_BOARD_HERE
  framework:
    type: esp-idf
    version: 4.4.3
    platform_version: 5.3.0
Anto79-ops commented 1 year ago

thanks! Should this be placed on all the proxies or just for testing I can do this one one proxy...closest to it, hopefully

bdraco commented 1 year ago

All of them

bowentz commented 1 year ago

The ESP32s that I use for the proxies already have the esp-isf info (well almost what you have) and have been flashed with a cable. Is it necessary to add the version lines?

https://pastebin.com/UDvww3ri

All of my proxies have: esp32: board: m5stack-atom framework: type: esp-idf

bdraco commented 1 year ago

There are some bluetooth fixes in the newer version

bowentz commented 1 year ago

FYI, I get this warning in the logs after flashing with versions using cable. Just ignore?

INFO Reading configuration /config/esphome/master-bluetooth-proxy.yaml... WARNING The selected ESP-IDF framework version is not the recommended one. If there are connectivity or build issues please remove the manual version. WARNING The selected ESP-IDF framework version is not the recommended one. If there are connectivity or build issues please remove the manual version.

https://pastebin.com/CjdzQzmP

bdraco commented 1 year ago

FYI, I get this warning in the logs after flashing with versions using cable. Just ignore?

INFO Reading configuration /config/esphome/master-bluetooth-proxy.yaml...

WARNING The selected ESP-IDF framework version is not the recommended one. If there are connectivity or build issues please remove the manual version.

WARNING The selected ESP-IDF framework version is not the recommended one. If there are connectivity or build issues please remove the manual version.

https://pastebin.com/CjdzQzmP

Yes good to ignore. It's just not updated in esphome yet

bowentz commented 1 year ago

ok.. I just realized that this sensor is about equal distance from my Pi (with bluetooth enabled) running HA and the proxy I just updated. I disabled the bluetooth in the Pi to see is that has any affect. Also moved the sensor closer to the proxy to help make sure the sensor is being picked up by this particular proxy.

Will check it tomorrow. Thanks for all your help on this!!

bowentz commented 1 year ago

Well now lookey here... After updating the yaml with the version info as @bdraco suggested I no longer have the problem with this sensor. Just as an FYI, I did remove all other proxies from my network (powered off) and disabled the bluetooth integration from HA, as well as, placing the sensor about 2 feet away from the proxy being tested just to isolate the sensor to the proxy.

Today I plan to update all proxies and re-enable the bluetooth integration on HA to see if the issue comes back (I don't expect it to)..

A couple of questions for @bdraco :

  1. Is there any action to take to update the bluetooth integration on the Pi?
  2. When will I be able to remove the version lines from the yaml?

image

nsthompson commented 1 year ago

Add this and reflash with a serial cable


esp32:

  board: YOUR_BOARD_HERE

  framework:

    type: esp-idf

    version: 4.4.3

    platform_version: 5.3.0

Can you do an OTA update or does it need to be done via serial?

bowentz commented 1 year ago

Add this and reflash with a serial cable


esp32:

  board: YOUR_BOARD_HERE

  framework:

    type: esp-idf

    version: 4.4.3

    platform_version: 5.3.0

Can you do an OTA update or does it need to be done via serial?

I used cable since @bdraco suggested that.. I think it has to do with changes to the file structure on the board.

Anto79-ops commented 1 year ago

Hey all thanks for doing some of the leg work. I have not had a chance to test this out some of my proxies are installed up high and I need to get a ladder to access them.

If it matters I did do a Serial update on a 12.x update recently. If this is good enough I'd be happy to do an OTA update otherwise I might get to it in a day or two as I have to go get my ladder in the shed... but it's -30° outside LOL

bowentz commented 1 year ago

Hey all thanks for doing some of the leg work. I have not had a chance to test this out some of my proxies are installed up high and I need to get a ladder to access them.

If it matters I did do a Serial update on a 12.x update recently. If this is good enough I'd be happy to do an OTA update otherwise I might get to it in a day or two as I have to go get my ladder in the shed... but it's -30° outside LOL

I did the update with cable in December too, but used the cable again anyway.. I'd say, based on your situation (-30), go ahead and try it OTA and see if that fixes the issue. Oh course unless @bdraco says otherwise......

Anto79-ops commented 1 year ago

Ok, sounds good! by the way I think you're pinging someone else....bdraco is the right one, but bdrako could be someone else.

bdraco commented 1 year ago

If you did a serial flash with esp-idf set you can OTA. If esp-idf was not set or you don't know if it was do the serial flash again

bdraco commented 1 year ago

The version numbers above should eventually get updated to be the default in an esphome production version via https://github.com/esphome/esphome/pull/4254

Anto79-ops commented 1 year ago

ok, thank you! I will do this tonight, and serial flash them.

can someone tell me what do I place in for the "board"? Its an esp32 so, do I just place esp32?

esp32:

  board: YOUR_BOARD_HERE

  framework:

    type: esp-idf

    version: 4.4.3

    platform_version: 5.3.0
bdraco commented 1 year ago

board

All the supported boards are here

https://registry.platformio.org/platforms/platformio/espressif32/boards

Click on the board you have to get the string

bowentz commented 1 year ago

@bdraco I may have spoken too soon. When I put the sensor back on the porch it goes back to doing the same thing as before. I'm thinking now that it must be a signal problem?? It's back inside now for a sanity check..

actingupagain

BUT. The strange thing is that the ESPHome (no proxy) device that is farther away from the sensor on the porch still does not show the same issue.

yaml for ESPHome no proxy device that is working: https://pastebin.com/WEd1Jx2w

image

Anto79-ops commented 1 year ago

all right! did the the updates via serial to all 7 of my ESP32 BT proxies. Here's my yaml for the proxies (all the same except proxy name):

substitutions:
  name: esp32-bluetooth-proxy-4cae58
packages:
  esphome.bluetooth-proxy: github://esphome/bluetooth-proxies/esp32-generic.yaml@main
esphome:
  name: ${name}
  name_add_mac_suffix: false

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

esp32_ble_tracker:
  scan_parameters:
    interval: 1100ms
    window: 1100ms
    active: true

bluetooth_proxy:
  active: true

sensor:
  # Uptime sensor.
  - platform: uptime
    name: esp32-bluetooth-proxy-4cae58.uptime
  - platform: template
    name: esp32-bluetooth-proxy-4cae58.free_memory
    lambda: return heap_caps_get_free_size(MALLOC_CAP_INTERNAL);
    unit_of_measurement: 'B'
    state_class: measurement

esp32:
  board: esp32dev
  framework:
    type: esp-idf
    version: 4.4.3
    platform_version: 5.3.0

just to confirm the update took, my proxies also give the same warning messages as here

last proxy was updated at 9:23 PM (MST) now just have to give it some time. Below are freezer and fridge compartment temps just after the last proxy was updated:

image

and

image

Some notes:

  1. I did not move or touch the inkbirds
  2. I would imagine the inkbirds are also within reach of my RPi BT and my Shelly Plus 1 UL which is also set up for active Bluetooth scanning. Coincidently, the Shelly got FW update today to 20230202-152410/0.13.0-beta3-g6e7ec69

Ok, fingers crossed and I'll post back in a bit.

Anto79-ops commented 1 year ago

Hi all, just an update, its been about 10 hrs now, not sure if there was an improvement. The fridge compartment (warmer) looks like there is, but freezer looks about the same. Red arrow indicates when I updated the last proxy.

image

image

bdraco commented 1 year ago
Screenshot 2023-02-04 at 2 29 54 PM

Still going strong after a few days. The drop outs are restarts from testing.

@Anto79-ops Can you send a link to the esp boards you purchased? I'm starting to think this is a board specific problem.

Anto79-ops commented 1 year ago

Glad to hear its working for you, still, that means a solution is possible!

For sure, I bought here:

https://www.aliexpress.com/item/1005002410521023.html?spm=a2g0o.order_list.order_list_main.23.cfe81802ZjQPKs

But here is the actual picture of the board:

DSC_0007