search

home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
71.41k stars 29.9k forks source link

Update of Mi Flora sensor is taking over 10 seconds #19362

Closed mlevit closed 5 years ago

mlevit commented 5 years ago

Home Assistant release with the issue: 0.84

Last working Home Assistant release (if known): < 0.83

Operating environment (Hass.io/Docker/Windows/etc.): Hass.io

Component/platform: https://www.home-assistant.io/components/sensor.miflora/

Description of problem: I have 4 Mi Flora sensors. 3 of them are working fine but the 4th is no longer connecting and updating Home Assistant. If I use the Android Flower Care app, the sensor is actually functioning, but Home Assistant is timing out trying to receive updates.

Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):

# sensor
  - platform: miflora
    mac: 'C4:7C:8D:67:2B:43'
    name: Master Bedroom Palm
    force_update: true
    # median: 3
    monitored_conditions:
      - moisture
      - light
      - temperature
      - conductivity
      - battery

# plant
  master_bedroom_palm:
    sensors:
      moisture: sensor.master_bedroom_palm_moisture
      battery: sensor.master_bedroom_palm_battery
      temperature: sensor.master_bedroom_palm_temperature
      conductivity: sensor.master_bedroom_palm_conductivity
      brightness: sensor.master_bedroom_palm_light_intensity
    min_moisture: 15
    max_moisture: 60
    min_battery: 15
    min_conductivity: 350
    max_conductivity: 2000
    min_temperature: 10
    max_temperature: 32
    max_brightness: 35000

Traceback (if applicable):

Update of sensor.master_bedroom_palm_temperature is taking over 10 seconds
8:54 AM util/async_.py (WARNING)
Update of sensor.master_bedroom_palm_battery is taking over 10 seconds
8:54 AM util/async_.py (WARNING)
Update of sensor.master_bedroom_palm_light_intensity is taking over 10 seconds
8:54 AM util/async_.py (WARNING)
Update of sensor.master_bedroom_palm_conductivity is taking over 10 seconds
8:54 AM util/async_.py (WARNING)
Update of sensor.master_bedroom_palm_moisture is taking over 10 seconds
8:54 AM util/async_.py (WARNING)

Additional information:

ronytomen commented 5 years ago

I struggled with timeouts as well (in previous versions), but I have been steadily adding more MiFlora sensors. I was wondering if the BT driver was having issues, because I sometimes could get it to function correctly if I restarted BT daemon.

I ended up setting up a Plant Gateway on a Raspberry Pi that is more centrally placed in my home.

Watching Plant Gateway working, I have noticed that it will fail to connect to 1 or 2 of the 5 sensors I currently have active. It will eventually connect, get the metrics from the sensor and send the results to Home Assistant via MQTT. I believe it's due to the distance between the Raspberry Pi and some sensors. So, knowing that, I have been wondering if the MiFlora component code is issuing this warning and not retrying? I have yet to inspect the component code.

raccettura commented 5 years ago

Something did get worse here.

Pre 0.84 Mi Flora threw some "over 10 seconds" errors but otherwise worked ok. Since 0.84 having Mi Flora sensors has resulted in HomeAssistant startup times taking in excess of 15-20 minutes, no UI (http not responding). Commenting out the Mi Flora sensors gets it working again.

This is all via Docker.

mlevit commented 5 years ago

@ChristianKuehnel since you've worked on Mi Flora before, would you be able to investigate this?

ChristianKuehnel commented 5 years ago

The general problem with these sensors is that they are not meant to be used over longer distances. In my apartment I get a range of about 1 dry wall or 6m direct line of sight. Everything beyond that is not really stable. And as it's a radio connection there can always be issues with the stability.

There weren't any recent changes to the miflora library. https://github.com/open-homeautomation/miflora/releases

There were some recent changes to the home assistant component by @PaulAnnekov . Could you please have a look?

Otherwise I don't really know what could cause the change...

Am Sa., 29. Dez. 2018, 22:42 hat Marat Levit notifications@github.com geschrieben:

@ChristianKuehnel https://github.com/ChristianKuehnel since you've worked on Mi Flora before, would you be able to investigate this?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/home-assistant/home-assistant/issues/19362#issuecomment-450525631, or mute the thread https://github.com/notifications/unsubscribe-auth/ATCWCH6aAyKpLbHCCiyAxdNqSc_iR_aqks5u9-_BgaJpZM4ZVe96 .

PaulAnnekov commented 5 years ago

The last release of miflora component was in 0.80.0. I don't see any significant changes after that time. Checking this issue will require a lot of debugging and it will be faster to debug on place, e.g. to change a distance between miflora sensor and bluetooth transmitter or to patch-restart HA. Also, I don't use miflora anymore.

raccettura commented 5 years ago

Just tried 0.88b and it's not working.

It seems to have broken between 0.83 and 0.84 best I can tell. I don't think it's necessarily a change in the mi flora library/component itself. This smells like a race condition or some other contention issue. It seems to be mainly (or only) impacting people with multiple sensors.

ChristianKuehnel commented 5 years ago

@raccettura Can you please turn on debug logging on miflora, btlewrap and bluepy? Maybe this will give us a better idea of what's going wrong...

You can do this by adding this to your configuration.yaml:

logger:
  default: info
  logs:
    miflora: debug
    btlewrap: debug
    bluepy: debug

Please append these debug log to the ticket.

raccettura commented 5 years ago

@ChristianKuehnel

Some snippets that seem relevant:

2019-02-18 19:03:08 DEBUG (SyncWorker_16) [miflora.miflora_poller] Received result for handle 56: 5B 15 32 2E 37 2E 30
2019-02-18 19:03:08 DEBUG (SyncWorker_3) [miflora.miflora_poller] Received result for handle 56: 5B 15 32 2E 37 2E 30
2019-02-18 19:03:11 DEBUG (SyncWorker_13) [btlewrap.bluepy] Call to <function BluepyBackend.connect at 0x7f0f36e6e048> failed, try 1 of 3
2019-02-18 19:03:16 DEBUG (SyncWorker_13) [btlewrap.bluepy] Call to <function BluepyBackend.connect at 0x7f0f36e6e048> failed, try 2 of 3
2019-02-18 19:03:18 DEBUG (SyncWorker_13) [miflora.miflora_poller] Received result for handle 56: 0B 15 32 2E 37 2E 30
2019-02-18 19:03:21 DEBUG (SyncWorker_19) [miflora.miflora_poller] Received result for handle 56: 0A 10 32 2E 36 2E 32
2019-02-18 19:03:21 DEBUG (SyncWorker_0) [btlewrap.bluepy] Call to <function BluepyBackend.connect at 0x7f0f36e6e048> failed, try 1 of 3
2019-02-18 19:03:24 DEBUG (SyncWorker_0) [miflora.miflora_poller] Received result for handle 56: 0B 15 32 2E 37 2E 30
2019-02-18 19:03:24 DEBUG (SyncWorker_0) [miflora.miflora_poller] Filling cache with new sensor data.
2019-02-18 19:04:59 DEBUG (SyncWorker_16) [btlewrap.bluepy] Call to <function BluepyBackend.connect at 0x7f0f36e6e048> failed, try 2 of 3
2019-02-18 19:05:40 DEBUG (SyncWorker_16) [btlewrap.bluepy] Call to <function BluepyBackend.connect at 0x7f0f36e6e048> failed, try 3 of 3
2019-02-18 19:05:40 INFO (SyncWorker_16) [homeassistant.components.sensor.miflora] Polling error

After 3rd attempt it actually loads. Took maybe 3-4 minutes. Interestingly without the debugger turned on I could never get it to start.

johnflorin commented 5 years ago

I had the same problem - it stopped working suddenly, had the 10 second warnings (though they were also present before, when it worked), tried several restarts (including of the entire VM) and when enabling the debugger it magically started working again!

daenny commented 5 years ago

This seems indeed the same problem as I am facing in https://github.com/home-assistant/home-assistant/issues/21774, I think enabling the debug log slows down the startup process in such a way that sometimes not all threads are assigned to the miflora sensor and then the system is able to boot.

sajikur commented 5 years ago

Hi all. I have the same problem with my MIFLORA sensors (10 seconds warnings). Moreover, I've tried to make a clean new installation of hassio on new sd card. Well, when rasp rebooted for the first time after upload last snapshot, UI stopped until I've wrote "#" on miflora lines in configurator.yaml. Could someone solve this issue? TY

Spirituss commented 5 years ago

Have similar problems with Mi Flora. It suddenly stops working but after HA restart could start working again. I turned on debug for Flora and bluetooth, have tons of the following messages in log: DEBUG (Thread-4) [btlewrap.bluepy] Call to <function BluepyBackend.write_handle at 0x61b44d20> failed, try 1 of 3 and DEBUG (Thread-3) [btlewrap.bluepy] Call to <function BluepyBackend.connect at 0x61d4e6a8> failed, try 1 of 3

Weezul commented 5 years ago

Any news on this?

I started experiencing the same problem (No values from sensor, with the BluepyBackend.connect failed call error) a week or two ago. Not sure what I changed :/ My config file history shows no changes on the MiFlora config, I have updated to a later Hassio version, not sure if that was the cause.

sajikur commented 5 years ago

I solved every problems with 3A power supply and ABSOLUTELY IMPORTANT 32 GB micro sd V30

In data 13 agosto 2019 11:45:06 Wessel du Plooy notifications@github.com ha scritto:

Any news on this? I started experiencing the same problem (No values from sensor, with the BluepyBackend.connect failed call error) a week or two ago. Not sure what I changed :/ My config file history shows no changes on the MiFlora config, I have updated to a later Hassio version, not sure if that was the cause. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

Weezul commented 5 years ago

Thanks, will give that a try! (And replace this remark with the results)

sajikur commented 5 years ago

Little tip: I upgraded my system to Intel nuc i3 with 8gb ram and ssd 128 GB. FABOLOUS! Another universe! Reboot in about 5 sec and cameras without stuck. If you can, upgrade to nuc Bye

In data 13 agosto 2019 13:03:19 Wessel du Plooy notifications@github.com ha scritto:

Thanks, will give that a try! — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

gilbert-grape commented 5 years ago

I have the same problem with 3 Miflora Sensors. It used to work until the end of year 2018, and since then I have problems. I am sure it can not be the distance between the sensor and the Raspberry Pi3. I tried the following with no effects:

What I see now:

bluetoothctl
[bluetooth]# scan on
bluetoothctl failed to connect org.bluez.error.failed
sajikur commented 5 years ago

Hi, I solved with esphome (esp32) and nuc Ciao!

In data 29 agosto 2019 10:35:22 gilbert-grape notifications@github.com ha scritto:

I have the same problem with 3 Miflora Sensors. It used to work until the end of year 2018, and since then I have problems. I am sure it can not be the distance between the sensor and the Raspberry Pi3. I tried the following with no effects:Reduce the distance of the sensor and Pi to 1m and less. Delete all sensors except one from configuration.yaml. Update Hass.io Reinstall / Clean new installation Restarting Hass.io and Home Assistant Server Change battery of the Miflora Sensors (the battery is almost full) What I see now: It looks like it is working worse for every month. First I thought a restart of Hass.io helps. Most time a restart helps after some time I get values, but very fast the values freeze. And then I need a new restart. Sometimes a restart helps and all sensors have new values. Sometimes 2 sensors have new values and 1 has only the battery value. It is like random. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

gilbert-grape commented 5 years ago

Little tip: I upgraded my system to Intel nuc i3 with 8gb ram and ssd 128 GB. FABOLOUS! Another universe! Reboot in about 5 sec and cameras without stuck. If you can, upgrade to nuc Bye In data 13 agosto 2019 13:03:19 Wessel du Plooy notifications@github.com ha scritto: … Thanks, will give that a try! — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

Nice for you. But this is not really helpful for me. How do you set up your ESP32? Do you have a manual?

sajikur commented 5 years ago

Buy esp32 wroom from ebay, install esphome in hassio (integration) and follow wizard when open esp page from hassio. If you don't see ota option to upload code in esp32 from hassio, you must connect only for first time esp32 with usb cable to pc with arduino ide.

https://esphome.io/devices/esp32.html

https://esphome.io/components/sensor/xiaomi_miflora.html

Try and let me know Bye Mario

In data 29 agosto 2019 11:00:45 gilbert-grape notifications@github.com ha scritto:

Little tip: I upgraded my system to Intel nuc i3 with 8gb ram and ssd 128 GB. FABOLOUS! Another universe! Reboot in about 5 sec and cameras without stuck. If you can, upgrade to nuc Bye In data 13 agosto 2019 13:03:19 Wessel du Plooy notifications@github.com ha scritto: … Thanks, will give that a try! — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.Nice for you. But this is not really helpful for me. How do you set up your ESP32? Do you have a manual?— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

ElGranLoky commented 5 years ago

I do the same like @sajikur, works fine.

And after you can use the esphome to update esp32 & ESP8266 from more devices:

https://blakadder.github.io/templates/

The templates from tasmota help you to find the correct pin use.

sajikur commented 5 years ago

It's true! you can use the same device for both miflora and mijia ble temperature and humidity or to update esp8266 and esp32

In data 30 agosto 2019 08:57:05 ElGranLoky notifications@github.com ha scritto:

I do the same like @sajikur, works fine. And after you can use the esphome to update esp32 & ESP8266 from more devices: https://blakadder.github.io/templates/ The templates from tasmota help you to find the correct pin use. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

gilbert-grape commented 5 years ago

In Home Assistant exists the attribute scan_interval, where I can define the read interval for the miflora sensor. But in esp32home does not exist this attribute. In the documenataion is written it depends on the sending of miflora. Esp42 does only listen.

Can you explain how it works exactly?

sajikur commented 5 years ago

In miflora over esp32 you can't control battery level and scan interval. So you use esp32 if you have errors in Rasp log, miflora is far from Rasp or simply to make Rasp's work easier. Ciao

In data 30 agosto 2019 16:47:57 gilbert-grape notifications@github.com ha scritto:

In Home Assistant exists the attribute scan_interval, where I can define the read interval for the miflora sensor. But in esp32home does not exist this attribute. In the documenataion is written it depends on the sending of miflora. Esp42 does only listen.Can you explain how it works exactly? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

ChristianKuehnel commented 5 years ago

duplicate of #24453

ElGranLoky commented 5 years ago

@gilbert-grape The ESPHome Xiaomi integration listens passively to packets the xiaomi device sends by itself. Battery_level is the most slow sensor of all, you can read about the component in this link:

https://esphome.io/components/sensor/xiaomi_miflora.html

@ChristianKuehnel Don't despair. Since version 0.84 this problem exists and not everyone knows how to program or search for errors. For my part I don't usually do many PR but I try to help whenever I have time.

And even if the missing library is added, the problem reappears over time as users comment.

Weezul commented 5 years ago

For me, the problem was solved by changing to a stronger 3A power supply. I can only assume that something on the RasPi started drawing more power, which underpowered it's BT radio. Maybe this helps someone.

gilbert-grape commented 5 years ago

Weezul: Thanks for the tip. I changes my power adapter to an Apple iPad adapter which has enough power. Result: Nothing changed, still only an update on miflora when i restart HA. And not all miflora have values. Sometimes One of 3 has values after a restart. Sometimes 2.

balloob commented 5 years ago

Closing this as it's a duplicate of #24453 and that has been resolved.