search

bramstroker / homeassistant-powercalc

Custom component to calculate estimated power consumption of lights and other appliances
MIT License
958 stars 252 forks source link

After every reboot integration automatically creates a None device #2281

Closed KlaraJa closed 3 months ago

KlaraJa commented 3 months ago

System Health details

System Information

version core-2024.6.0
installation_type Home Assistant OS
dev false
hassio true
docker true
user root
virtualenv false
python_version 3.12.2
os_name Linux
os_version 6.6.28-haos-raspi
arch aarch64
timezone Europe/Berlin
config_dir /config
Home Assistant Community Store GitHub API | ok -- | -- GitHub Content | ok GitHub Web | ok GitHub API Calls Remaining | 4489 Installed Version | 1.34.0 Stage | running Available Repositories | 1463 Downloaded Repositories | 28
Home Assistant Supervisor host_os | Home Assistant OS 12.3 -- | -- update_channel | stable supervisor_version | supervisor-2024.05.1 agent_version | 1.6.0 docker_version | 25.0.5 disk_total | 219.4 GB disk_used | 9.8 GB healthy | true supported | true host_connectivity | true supervisor_connectivity | true ntp_synchronized | true virtualization | board | rpi4-64 supervisor_api | ok version_api | ok installed_addons | Let's Encrypt (5.0.18), AirCast (4.2.1), Matter Server (6.1.0), Mosquitto broker (6.4.1), Studio Code Server (5.15.0), Advanced SSH & Web Terminal (18.0.0), Home Assistant Google Drive Backup (0.112.1), InfluxDB (5.0.0), Grafana (10.0.0), iSponsorBlockTV add-on (2.1.0), Whisper (2.1.0)
Dashboards dashboards | 7 -- | -- resources | 14 views | 15 mode | storage
Recorder oldest_recorder_run | May 28, 2024 at 20:08 -- | -- current_recorder_run | June 5, 2024 at 21:38 estimated_db_size | 642.38 MiB database_engine | sqlite database_version | 3.44.2

Checklist

Describe the issue

Everytime I reboot Homeassistant, the powercalc integration creates a None Device.

Untitled

Reproduction steps

  1. Reload homeassistant
  2. Integration should have created a None sensor

Debug logs

I put my log here because it was too long.
https://pastebin.com/SbATH8Jg

Diagnostics dump or YAML config

No response

k-korn commented 3 months ago

I also have this problem, every reboot adds 22 PowerCalc "None power + None energy" entities, 22 being close to the total amount of valid PowerCalc entities I have.

If there are any steps to assist in debugging - will be happy to help (Home Assistant Core 2024.6, installed via pip)

bramstroker commented 3 months ago

@k-korn @KlaraJa I was also reported by other user, but this issue was closed #2279.

I tried to reproduce quickly on my test instance but was not able to reproduce yet. I don't have much time on my hand coming days as I'm a long weekend on vacacion. Will have a more thorough look when I'm back.

Looking at the logs I notice the following:

2024-06-05 21:38:39.786 DEBUG (MainThread) [custom_components.powercalc.sensors.group] Creating grouped energy sensor: None energy (entity_id=sensor.none_energy)

So it's trying to create a group sensor. Not sure what causes it yet.

I also see it's creating an actual new configuration in integration section.

This makes me think it could be related to this PR: https://github.com/bramstroker/homeassistant-powercalc/pull/2246

Did you happen to create a daily fixed energy sensor and fill the group field?

bramstroker commented 3 months ago

@k-korn @KlaraJa I think I now what causes the problem. To verify my suspicion and make the actual fix I need diagnostics of all the powercalc integration configuration (either virtual power ones, or daily energy) you have added with the GUI. You can download these by clicking the three dots and than "Download diagnostics". Would be nice if you can provide these.

k-korn commented 3 months ago

@bramstroker , I've tried and it does not work. Three dots - download diagnostics does not produce any results for Powercalc. The following error appears in the browser console: image

Uncaught (in promise) 
Object 
code:  "invalid_format"
message:  "expected str for dictionary value @ data['path']. Got None"
KlaraJa commented 3 months ago

@bramstroker same issue with downloading diagnostics on my end

k-korn commented 3 months ago

On logging, debug logs for custom_components.powercalc show a lot of these lines:

2024-06-06 21:08:05.440 DEBUG (MainThread) [custom_components.powercalc.sensors.group] Creating grouped power sensor: None power (entity_id=sensor.none_power_16)
2024-06-06 21:08:05.440 DEBUG (MainThread) [custom_components.powercalc.sensors.group] Creating grouped energy sensor: None energy (entity_id=sensor.none_energy_16)
2024-06-06 21:08:05.441 DEBUG (MainThread) [custom_components.powercalc.sensors.group] Creating grouped power sensor: None power (entity_id=sensor.none_power_19)
2024-06-06 21:08:05.441 DEBUG (MainThread) [custom_components.powercalc.sensors.group] Creating grouped energy sensor: None energy (entity_id=sensor.none_energy_19)
2024-06-06 21:08:05.445 DEBUG (MainThread) [custom_components.powercalc.sensors.group] Creating grouped power sensor: None power (entity_id=sensor.none_power_21)
2024-06-06 21:08:05.445 DEBUG (MainThread) [custom_components.powercalc.sensors.group] Creating grouped energy sensor: None energy (entity_id=sensor.none_energy_21)
2024-06-06 21:08:05.446 DEBUG (MainThread) [custom_components.powercalc.sensors.group] Creating grouped power sensor: None power (entity_id=sensor.none_power_25)
2024-06-06 21:08:05.446 DEBUG (MainThread) [custom_components.powercalc.sensors.group] Creating grouped energy sensor: None energy (entity_id=sensor.none_energy_25)
bramstroker commented 3 months ago

You don't need to download the diagnostics of the "None" entries, but the ones you've setup correctly before. For example "Balcony light" from your screenshot.

k-korn commented 3 months ago

You don't need to download the diagnostics of the "None" entries, but the ones you've setup correctly before.

yes, it does not work for valid entries as well. And actually, for other integrations too on 2024.6 (for example, on https://www.home-assistant.io/integrations/google_assistant). However, I've not found any actual HA core bugs on it yet.

bramstroker commented 3 months ago

You don't need to download the diagnostics of the "None" entries, but the ones you've setup correctly before.

yes, it does not work for valid entries as well. And actually, for other integrations too on 2024.6 (for example, on https://www.home-assistant.io/integrations/google_assistant). However, I've not found any actual HA core bugs on it yet.

And if you remove the "None" entries first? without restarting? Otherwise you can find concerning json data in config/.storage/core.config_entries

k-korn commented 3 months ago

You don't need to download the diagnostics of the "None" entries, but the ones you've setup correctly before.

yes, it does not work for valid entries as well. And actually, for other integrations too on 2024.6 (for example, on https://www.home-assistant.io/integrations/google_assistant). However, I've not found any actual HA core bugs on it yet.

And if you remove the "None" entries first? without restarting? Otherwise you can find concerning json data in config/.storage/core.config_entries

I've got 400+ of these "none" entries, so would probably need to investigate some bulk removal (as I don't want to lose the history on actual Powercalc entries by reinstalling the integration)

bramstroker commented 3 months ago

You don't need to download the diagnostics of the "None" entries, but the ones you've setup correctly before.

yes, it does not work for valid entries as well. And actually, for other integrations too on 2024.6 (for example, on https://www.home-assistant.io/integrations/google_assistant). However, I've not found any actual HA core bugs on it yet.

And if you remove the "None" entries first? without restarting? Otherwise you can find concerning json data in config/.storage/core.config_entries

I've got 400+ of these "none" entries, so would probably need to investigate some bulk removal (as I don't want to lose the history on actual Powercalc entries by reinstalling the integration)

Sorry to hear you have this issue, and so many of them :-( You can have a look into editing that file I mentioned. Please make a backup of the file before you do that. I might have a look into adding some cleanup logic in upcoming powercalc version, as I'm afraid you are not the only one with this issue, and probably messed up some installations with this bug one of the latest Powercalc versions.

k-korn commented 3 months ago

Removed "None" entries via core.config_entries, core.entity_registry editing. However, this did not help to gather the debug data. Would dumping Powercalc-related part of core.config_entries help?

Also, here is something interesting from the log. See how "Creating grouped energy sensor: None energy" lines are only present for lights (e.g. for entity_id=light.night_light), but not for entity_id=climate.terneo:

2024-06-07 07:45:12.339 DEBUG (MainThread) [custom_components.powercalc.sensors.power] Creating power sensor (entity_id=climate.terneo entity_category=None, sensor_name=Terneo power strategy=fixed manufacturer= model= unique_id=pc_2C004D000150535658313720000159)
2024-06-07 07:45:12.340 DEBUG (MainThread) [custom_components.powercalc.sensors.energy] Creating energy sensor (entity_id=sensor.terneo_energy, source_entity=sensor.terneo_power, unit_prefix=k)
2024-06-07 07:45:12.347 DEBUG (MainThread) [custom_components.powercalc.sensors.power] Creating power sensor (entity_id=light.mini entity_category=None, sensor_name=Work Zone Light power strategy=fixed manufacturer= model= unique_id=powercalc.light.workzone)
2024-06-07 07:45:12.347 DEBUG (MainThread) [custom_components.powercalc.sensors.energy] Creating energy sensor (entity_id=sensor.work_zone_light_energy, source_entity=sensor.work_zone_light_power, unit_prefix=k)
2024-06-07 07:45:12.349 DEBUG (MainThread) [custom_components.powercalc.sensors.abstract] Binding sensor.work_zone_light_power to device cf0deafadc1510be7d21484ef333b116
2024-06-07 07:45:12.355 DEBUG (MainThread) [custom_components.powercalc.sensors.power] Creating power sensor (entity_id=switch.pc_wol entity_category=None, sensor_name=PC Power power strategy=fixed manufacturer= model= unique_id=powercalc.pc_wol)
2024-06-07 07:45:12.355 DEBUG (MainThread) [custom_components.powercalc.sensors.energy] Creating energy sensor (entity_id=sensor.pc_energy, source_entity=sensor.pc_power, unit_prefix=k)
2024-06-07 07:45:12.360 DEBUG (MainThread) [custom_components.powercalc.power_profile.loader.remote] Loading library.json from github
2024-06-07 07:45:12.362 DEBUG (MainThread) [custom_components.powercalc.sensors.power] Creating power sensor (entity_id=light.night_light entity_category=None, sensor_name=Night light power strategy=linear manufacturer= model= unique_id=pc_light.night_light)
2024-06-07 07:45:12.363 DEBUG (MainThread) [custom_components.powercalc.sensors.energy] Creating energy sensor (entity_id=sensor.night_light_energy, source_entity=sensor.night_light_power, unit_prefix=k)
2024-06-07 07:45:12.369 DEBUG (MainThread) [custom_components.powercalc.sensors.group] Creating grouped energy sensor: None energy (entity_id=sensor.none_energy)
2024-06-07 07:45:12.373 DEBUG (MainThread) [custom_components.powercalc.sensors.group] sensor.none_energy: Restoring state: 25.0701
2024-06-07 07:45:12.373 DEBUG (MainThread) [custom_components.powercalc.sensors.group] Load previous energy sensor states from store
2024-06-07 07:45:12.375 DEBUG (MainThread) [custom_components.powercalc.sensors.power] Creating power sensor (entity_id=light.kid_light entity_category=None, sensor_name=Kid light power strategy=linear manufacturer= model= unique_id=pc_light.kid_light)
2024-06-07 07:45:12.376 DEBUG (MainThread) [custom_components.powercalc.sensors.energy] Creating energy sensor (entity_id=sensor.kid_light_energy, source_entity=sensor.kid_light_power, unit_prefix=k)
2024-06-07 07:45:12.396 DEBUG (MainThread) [custom_components.powercalc.sensors.group] Creating grouped energy sensor: None energy (entity_id=sensor.none_energy_2)
2024-06-07 07:45:12.409 DEBUG (MainThread) [custom_components.powercalc.sensors.group] sensor.none_energy_2: Restoring state: 24.8915
2024-06-07 07:45:12.411 DEBUG (MainThread) [custom_components.powercalc.sensors.power] Creating power sensor (entity_id=light.toilet_light entity_category=None, sensor_name=Toilet light power strategy=linear manufacturer= model= unique_id=pc_light.toilet_light)
2024-06-07 07:45:12.412 DEBUG (MainThread) [custom_components.powercalc.sensors.energy] Creating energy sensor (entity_id=sensor.toilet_light_energy, source_entity=sensor.toilet_light_power, unit_prefix=k)
2024-06-07 07:45:12.422 DEBUG (MainThread) [custom_components.powercalc.sensors.group] Creating grouped energy sensor: None energy (entity_id=sensor.none_energy_3)
2024-06-07 07:45:12.424 DEBUG (MainThread) [custom_components.powercalc.sensors.group] sensor.none_energy_3: Restoring state: 25.2940
2024-06-07 07:45:12.446 DEBUG (MainThread) [custom_components.powercalc.power_profile.loader.remote] Loading library.json from github
2024-06-07 07:45:12.449 DEBUG (MainThread) [custom_components.powercalc.power_profile.loader.remote] Loading library.json from github
2024-06-07 07:45:12.456 DEBUG (MainThread) [custom_components.powercalc.power_profile.loader.remote] Loading library.json from github
2024-06-07 07:45:12.461 DEBUG (MainThread) [custom_components.powercalc.power_profile.loader.remote] Loading library.json from github
2024-06-07 07:45:12.463 DEBUG (MainThread) [custom_components.powercalc.sensors.power] Creating power sensor (entity_id=light.bathroom_light entity_category=None, sensor_name=Bathroom light power strategy=linear manufacturer= model= unique_id=pc_light.bathroom_light)
2024-06-07 07:45:12.463 DEBUG (MainThread) [custom_components.powercalc.sensors.energy] Creating energy sensor (entity_id=sensor.bathroom_light_energy, source_entity=sensor.bathroom_light_power, unit_prefix=k)
2024-06-07 07:45:12.469 DEBUG (MainThread) [custom_components.powercalc.sensors.group] Creating grouped energy sensor: None energy (entity_id=sensor.none_energy_4)
2024-06-07 07:45:12.471 DEBUG (MainThread) [custom_components.powercalc.sensors.group] sensor.none_energy_4: Restoring state: 62.5343
2024-06-07 07:45:12.475 DEBUG (MainThread) [custom_components.powercalc.sensors.power] Creating power sensor (entity_id=light.wardrobe_light entity_category=None, sensor_name=Wardrobe light power strategy=linear manufacturer= model= unique_id=pc_light.wardrobe_light)
2024-06-07 07:45:12.476 DEBUG (MainThread) [custom_components.powercalc.sensors.energy] Creating energy sensor (entity_id=sensor.wardrobe_light_energy, source_entity=sensor.wardrobe_light_power, unit_prefix=k)
2024-06-07 07:45:12.482 DEBUG (MainThread) [custom_components.powercalc.sensors.group] Creating grouped energy sensor: None energy (entity_id=sensor.none_energy_5)
bramstroker commented 3 months ago

@k-korn thanks. would definitely help if you could sent the powercalc related parts of the config_entries JSON. That would make me able to reproduce and probably fix.

k-korn commented 3 months ago

Here is the output of cat .storage/core.config_entries | jq '.data.entries[] | select(.domain=="powercalc")' powercalc-config.json It already some contains "None" entries.

bramstroker commented 3 months ago

Thanks a lot, this helps massively.

The problem is group being present as key, but with value null. This causes powercalc to create a group sensor on the fly. I have enough information now to fix the issue.

Screenshot 2024-06-07 at 08 05 22
k-korn commented 3 months ago

Thanks, looks like that is it. I've had exactly 11 such entries, and 22 new "none" entries were created on each restart.

For now, I've just removed "group": null, lines from the file and that's it.

bramstroker commented 3 months ago

Ah thanks for confirming this indeed fixes your issue. I have a fix with #2283. Which should prevent none entries to be created, and also remove all the none entities and config entries on startup. Will release soon.

bramstroker commented 3 months ago

Just released v1.12.7 with the fix.

KlaraJa commented 3 months ago

@bramstroker now the Integration is completely broken :/

Untitled 
Logger: homeassistant.setup
Source: setup.py:402
First occurred: 11:58:35 (1 occurrences)
Last logged: 11:58:35

Error during setup of component powercalc
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/setup.py", line 402, in _async_setup_component
    result = await task
             ^^^^^^^^^^
  File "/config/custom_components/powercalc/__init__.py", line 247, in async_setup
    await repair_none_config_entries_issue(hass)
  File "/config/custom_components/powercalc/__init__.py", line 450, in repair_none_config_entries_issue
    await hass.config_entries.async_remove(entry.entry_id)
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 1724, in async_remove
    del self._entries[entry.entry_id]
        ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 1541, in __delitem__
    self._unindex_entry(entry_id)
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 1535, in _unindex_entry
    del self._domain_unique_id_index[domain][unique_id]
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
KeyError: 'None' 
Logger: homeassistant.setup
Source: setup.py:483
First occurred: 11:58:52 (1 occurrences)
Last logged: 11:58:52

Unable to prepare setup for platform 'powercalc.sensor': Unable to set up component.
KlaraJa commented 3 months ago

I still have one None sensor, where I also have problems removing it via the GUI. I had problems with this specific sensor before the v1.12.7. But now it seems that the whole integration won't start because of this one sensor

KlaraJa commented 3 months ago

@bramstroker I've just fixed it like @k-korn did

Thanks, looks like that is it. I've had exactly 11 such entries, and 22 new "none" entries were created on each restart.

For now, I've just removed "group": null, lines from the file and that's it.

I don't know why, but my bathroom ceiling light still had the none group in their data. Removed that by hand, restarted HomeAssistant and integration works now.

While I was digging through some files, I found that in /config/.storage/powercalc_group are some of the none.sensor_energystill present, is that correct? In my case there are 9 in total, but iirc I also just had a maximum of 9 none sensors at one point.

Frankymuc commented 3 months ago

all destroy now after f...update...never before any problems bt since tousends of entrys none its horible and the update is repair nothing...now all is stoped and al data lost.thxy so much

bramstroker commented 3 months ago

@Frankymuc comments like this don't help at all solving the issue. And there should be no data lost.

Update:

I have issued new version which should catch all possible errors during repair routine, and at least startup HA and powercalc fully. The repair routine should be executed only when you had any None entities.

compoundbow83 commented 3 months ago

Version 1.12.8. now works again. But there are still "none" entities and I can't delete them.

bramstroker commented 3 months ago

@compoundbow83 Could you try again with 1.12.9. Should hopefully remove the None entities.

compoundbow83 commented 3 months ago

@compoundbow83 Could you try again with 1.12.9. Should hopefully remove the None entities.

@bramstroker I have installed 1.12.9.

It's a bit strange. When I go to powercal under integration, the none entities are still there.

Bildschirmfoto 2024-06-11 um 19 43 14

But when I go to entities they are not visible.

bramstroker commented 3 months ago

Hmm I see. I'm afraid I'm unable to automatically remove these, as HA gives an error when I try to. As they don't have a unique_id and HA flips on that.

Could you have a look into config/.storage/core.config_entries file? And look for the powercalc none entries. They should be there. Please share some of them here, I'll have a look if I can reproduce and find a workaround to be able to remove them.

You can manually edit the mentioned file and remove all the none entries. After that restart HA and your issue should be resolved.

WARNING Please backup the file before editing, because this file is not meant to be edited manually normally. When you make errors and cause invalid JSON this will prevent HA to start correctly.

compoundbow83 commented 3 months ago

@bramstroker hier one entry from config/.storage/core.config_entries

  {
    "data": {
      "sensor_type": "group",
      "name": "None",
      "group_member_sensors": [
        "78033d297ee68ab3fa227b2fc52cf0ec"
      ]
    },
    "disabled_by": null,
    "domain": "powercalc",
    "entry_id": "721c86c9e7e4a7ecb105a310597e7b1e",
    "minor_version": 1,
    "options": {},
    "pref_disable_new_entities": false,
    "pref_disable_polling": false,
    "source": "import",
    "title": "None",
    "unique_id": "None",
    "version": 3
  },

I am not sure if I will try to edit this file manuelle 🙈

bramstroker commented 3 months ago

I might have finally found a fix to actually remove all the None entries. With the current version you are running v1.12.9 eventually all the None entries will be removed after numerous HA restarts. It should remove one entry at a time, before running into an error raised by HA.

I have a new version which should hopefully remove them all in one go. https://github.com/bramstroker/homeassistant-powercalc/blob/833acde10356f652c1301d69d1593a49eda09cb9/custom_components/powercalc/__init__.py

Could you download this file and replace config/custom_components/powercalc/__init__.py with this one. Next restart HA and let me know if it is solved.

When it actually solves the issue I will release a new version with the fix.

compoundbow83 commented 3 months ago

Of corse, I have download the file and replace it and after this I have restart HA. And it works, the none entities are away.

Thank you for the fast fix

bramstroker commented 3 months ago

Of corse, I have download the file and replace it and after this I have restart HA. And it works, the none entities are away.

Thank you for the fast fix

Cool thanks for verifying, glad to hear issue is finally completely solved.

bramstroker commented 3 months ago

Two days ago I released v1.12.10, which should fully resolve this issue. Closing.