Wemo Insight Plug reporting sensor data every few seconds - filling database

[u8915055]

The problem

I have a series of wemo insight plugs used for power monitoring.. they are working with home assistant however im finding that they are reporting a LOT of data on the current consumption sensor. Im using them with a greenhouse heater as an example. When i look in the database im getting a reading about every 2-3 seconds.. given that i store 32 days worth of data that's a LOT of data.. orders of magnitude higher than any other sensor i have in home assistant.

Im wondering if there is any way to control the update time on these sensors. Not sure what controls this update but i assume it is not based on the scan interval for sensors in home assistant.

This one sensor is resulting in 192,000 records in my states database.

Any help is greatly appreciated.

What version of Home Assistant Core has the issue?

core-2021.12.7

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant Core

Integration causing the issue

wemo

Link to integration documentation on our website

https://www.home-assistant.io/integrations/wemo/

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[probot-home-assistant[bot]]

Hey there @esev, mind taking a look at this issue as it has been labeled with an integration (wemo) you are listed as a code owner for? Thanks! _{^{(message by CodeOwnersMention)}}

---

wemo documentation wemo source _{^{(message by IssueLinks)}}

[u8915055]

hi there @esev , remember me? :) Hope you are well. Not trying to create any work here.. just was hoping for some insight on the logic that the pywemo based sensor reports data. Just to add to this, i have another insight running on the lighting of the green house and for the same time period its only created 51000 or so records.. now the power consumption on the lighting is much more steady than the heater.. less fluctuation. Is there some sort of range of fluctuation that results in an update of the sensor? Ie. if its more than a 1/4 watt or something like that? Thanks again @esev(Eric)

[esev]

Hi @u8915055, happy new year!

I'm seeing the same behavior on my insight devices. There is actually 2x the state updates being recorded, as the energy usage is captured both as a state attribute of the insight switch entity as well as in the Current Power sensor entity.

I believe the WeMo Insight measures in milliwatts and samples the current about once per second. It seems to update the state any time it has changed by 5 milliwatts (or maybe that is just the resolution of its current sensor). So, yeah, it results in a lot of updates and makes the database very large.

I ended up adding this to my configuration.yaml to not record the data for the devices I wasn't interested in

recorder:
  exclude:
    entities:
      - switch.laundry_dryer
      - sensor.laundry_dryer_current_power
      - switch.laundry_washing_machine
      - sensor.laundry_washing_machine_current_power

That's not ideal though. You probably want to keep the data recorded, correct?

There are a few things we could try:

1. Disable recording for the devices that you don't need to keep history on. Use the yaml above as a starting point. (just mentioning the above, for completeness)
2. The frequent state updates are happening because Home Assistant is subscribed to state updates from the WeMo devices, and the Insight device seems to think a +- 5 milliwatt change is an "important" state update. PR (#56972) provides a global setting to disable the state subscriptions and control the polling interval. But it comes with some downsides, documented in PR (https://github.com/home-assistant/home-assistant.io/pull/20951) & here.
3. An improvement to the change above could be to make the options above be device specific and not global. You'd still have the downsides, but they'd be limited to just the devices that have subscriptions disabled. I plan to work on this after #56972 has been merged.
4. At some point we should remove the duplicate state attributes in the switch entity. That'll cut the storage cost of these updates in half. I have this on my radar, but since it's a breaking change I've been hesitant to do it. I probably should though.
5. We could also modify the home assistant sensor so it doesn't change until the current measured by the Insight device changes by some threshold (maybe 2 watts?). This might be the best solution, as it doesn't have any of the downsides of 2 & 3. I can work on this after PR (#63525) is merged. Assuming this works for you, do you have a preference for the threshold?

WDYT?

PS I definitely read this assuming the insight pun was intended :)

just was hoping for some insight on the logic that the pywemo based sensor reports data.

[u8915055]

Hey @esev! Thanks for that very very detailed response. That was super helpful!

I have already gone through most of my entities with recorder and have a pretty strict inclusion policy for data collection mainly because the database was getting so large :) However these insight sensors provide a few critical things for me especially since they are maintaining some heaters in hobby greenhouses and up here in vancouver its been quite cold.. so i use this info from them to make sure the heater is kicking on, and also how much its costing me monthly to run them.. so option 1 wouldn't work for me. Just reading through the other info that you provided to me.

So for #2, and in my case since i only use wemo insights and their newer non-power-management smart plugs, i dont see any downsides.. but that's just me, i dont use dimmers etc and likely will never. Although your #3 would be ideal whereby for users like me who only have smart plugs i would just poll the device instead.. and if we could have a scan_interval option even better!

Just to add a little more colour here, this is a set of data just taken from the database..

Notice the small power changes but these are happening every 2 seconds!

Thought this was interesting too.. these are state changes sorted by entity_id.. the top 4 are all related to two wemo devices.. two insights.

Anyway.. number 5 looks interesting too and would certainly be a viable solution but what threshold to pick? Everyone will likely have a different idea there depending on their use case.

To me number 3 with some sort of configurable scan_interval would be ideal. I dont need to know the current draw from the device every 2 seconds (sometimes less), i would likely poll these things about every 30 seconds and that would be enough.. it wouldnt make much of a diff with energy consumption calculations for sure.

Tell me though, if you changed things from subscribing to events on the wemo device to polling the wemo device, and i tell the plug to turn off through home assistant, i assume the power consumption state probably could take up to 30 seconds to reflect the fact that the switch is now off? Or would the event of telling the plug to turn off trigger a state update? (or could it?) Also, i assume the switch entity itself would reflect the state of being 'off' right away wouldn't it?

I might not be fully understanding how the pywemo works.. hopefully what im stating makes sense.

im happy to test anything you like. Good to work with you again. :)

By the way, I dont see duplicate states being recorded.. each successive state is different in the database.

Thanks again so much for help here.

[u8915055]

hey @esev , just thought id mention that im playing around with the filter component now.. it has a filter type called time_throttle that might just solve my problem. Ill report back.. Right now i have a filter sensor ive defined that is watching the wemo current power sensor and recording a value every 60 seconds.. boy if this works it will significantly cut my database size down, which has doubled in the past week. Ill let you know.. hope all's well.

[esev]

That's a nice find! I was unaware of the filter component.

Sorry I'm slow in responding to your previous message. I got sick early last week and was out for a few days. I'll still plan on implementing Option 3 (Once option 2 is done). There will be a control for the scan interval too (it'll look something like the image at the bottom of this PR).

Tell me though, if you changed things from subscribing to events on the wemo device to polling the wemo device, and i tell the plug to turn off through home assistant, i assume the power consumption state probably could take up to 30 seconds to reflect the fact that the switch is now off? Or would the event of telling the plug to turn off trigger a state update? (or could it?) Also, i assume the switch entity itself would reflect the state of being 'off' right away wouldn't it?

The switch would turn off right away. Currently the sensor would take 30 seconds (the scan/update interval) to update. I'll keep it on my to-do list to fix that though. Triggering a sensor state update, following the switch state update, sounds like a good way to handle this. Thanks for the suggestion.

[u8915055]

hey no problem at all.. so i have been running the filter sensor now for a little bit.. its definitely doing the job. In a short period of getting 17 updates from the wemo current power sensor, only 2 updates were recorded by the 'filter' sensor. I have the filter sensor configured to grab a sample every 60 seconds and its certainly not exactly, sometimes every two mins.. but sort of in the ball park which better than what im getting. For now im probably going to switch over to only holding history for the filter sensor.. The wemo current power sensor is now over 500000 records!!

Certainly your proposed changes will make this sort of thing much easier so im certainly looking forward to that.. i think those changes will be a great addition!

Hope you are feeling better. Thanks again for working on this.

[u8915055]

hey @esev, i think i found something else interesting. So i have another wemo insight on a plant stand controlling lights. Now with this specific light i dont care so much about the power consumption so the 'sensor.lean_to_greenhouse_current_power' insight sensor has been excluded from recorder. I checked the database and sure enough there are no entries for that sensor. So that works. However in this particular case I want to track the on and off time just so i know that the automation is doing its job and turning them on and off. So i do have the switch.lean_to_greenhouse enabled and included in recorder. However here's the weird part. Even though i have excluded the sensor, the SWITCH state is updating itself every two seconds.. just like the sensor was when i had it included. Even though the switch state itself from an on/off perspective only changes twice per day, im getting a state record in the state database every 2 seconds.. This doesnt make any sense at all.. Its sort of like any change in the current power sensor triggers a state update on the switch entity itself. Very weird.

On another front i did implement those two filter sensors for the lean_to insight above and another one that was really filling up the database, they really seem to be doign their jobs.. i was able to remove about 750,000 database entries just by doing that.. cut the database size down by about 60%. Nice. Anyway just wanted to update the situation.

[u8915055]

oh one more thing.. before i removed the history for the insight current power sensor that was updating every two seconds, i plotted that sensor against the filtered sensor that is only grabbing samples about every 60 seconds.. tracks pretty close .

[esev]

Its sort of like any change in the current power sensor triggers a state update on the switch entity itself.

Yes, this is exactly what's happening. The switch entity still has the current power embedded as an attribute. These attributes update at the same time as the sensor entities. This was the older way that the current was exposed in Home Assistant. I've been hesitant to remove them, in case someone was using them for an automation (I actually still am). But given this issue, it's probably best if I do remove them. .

As a work-around, there is another binary_sensor entity that you could record to get the same information. Look for binary_sensor.<name>_device. It is intended to track whether or not the device that is plugged into the Insight is powered on. I suspect the lights you have connected will be enough wattage to turn on/off that binary_sensor when the switch state changes. Example of what this is used for: One of my Insight sensors is connected to my clothes washer. I leave that one switched on all the time. The binary_sensor.laundry_washer_device entity turns on when the clothes washer is running, and off when it is not. That allows me to add automations based on the state of the connected device.

On another front i did implement those two filter sensors for the lean_to insight above and another one that was really filling up the database, they really seem to be doign their jobs.. i was able to remove about 750,000 database entries just by doing that.. cut the database size down by about 60%

Thanks for the update. That's great news!

[u8915055]

THank you for that update.. ok that makes sense now.. i will definitely look into that binary sensor.. sounds like exactly what im looking for..

on the note of the database size.. thought you'd get a kick out of this diagram.

That is what happened to the database size when i removed current_power sensors from those two insights and cleared their records out of the states database.. i did an optimize on the database as well. shrunk considerably. Definitely looks better.. when i use that binary sensor you're referring to that should take another big chunk out of that database.. thanks again for that recommendation.

[esev]

However in this particular case I want to track the on and off time just so i know that the automation is doing its job and turning them on and off.

One additional advantage of the binary_sensor is that you can know if the light actually turns on. That binary_sensor will only be "on" when there is at least 8 watts being drawn by the connected device (you can configure this threshold in the wemo app).

on the note of the database size.. thought you'd get a kick out of this diagram.

That's great! Mine is like 2.2Gb right now. I'm going to try the same thing.

[u8915055]

Actually that's a great point about the binary_sensor.. i just added it to recorder so ill use that goign forward..

Here's the config i used for the filter sensor in case its of use.

  - platform: filter
    name: "filter_greenhouse_current_power"
    entity_id: sensor.greenhouse_current_power
    filters:
      - filter: time_throttle
        window_size: 00:01

  - platform: filter
    name: "filter_lean_to_greenhouse_current_power"
    entity_id: sensor.lean_to_greenhouse_current_power
    filters:
      - filter: time_throttle
        window_size: 00:01

Im probably going to add them in for a tplink kasa power strip that also sends too much data.. not as much as the insights but getting up there.

[github-actions[bot]]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.