Support for new Home Assistant Energy Managment and Stats

[adam-olson]

Is your feature request related to a problem? Please describe. Would like to add support for new Home Assistant Energy Management functions. https://www.home-assistant.io/blog/2021/08/04/home-energy-management/

Describe the solution you'd like Support for the new statistics functions, seems like we would just need to add a few properties to the sensors. Not sure on the pricing and pricing projections can be worked in as of yet. https://developers.home-assistant.io/docs/core/entity/sensor/

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context https://www.home-assistant.io/docs/energy/

[dotKrad]

I'm currently figuring out how to start with this

[robtheone]

Hi there. I am wondering if my request is related to this one. I am trying to use the data provided by your FPL addon in my own energy dashboards. I have the Prometheus addon setup to scrape the data from Home Assistant, then the Grafana addon will put the data into nice looking graphs, but the way the data attributes are setup currently, the data cannot be read into Prometheus. The issues appears to be the attributes: "This data is provided by FPL. integration: FPL".

Other sensors that can be read by Prometheus say something similar to: state_class: measurement unit_of_measurement: kWh friendly_name: Energy Usage Total 1month device_class: energy

I would LOVE to be able to use the date fields from the FPL addon in Grafana as well to display current billing and next billing dates, but have struggled to get them into a format that can be read by Prometheus as well. In reading the docs for Prometheus, it appears the date field has to either be in a Unix format or Epoch (I don't know enough to be knowledgeable) how this needs to be formatted, but haven't been able to figure out how to template this data.

I have been using a custom template for converting the dollar amount into a readable field in the configuration.yaml:

`sensor:

• platform: template sensors: power_1516_monthly_usage_estimate: friendly_name: "1516 Current Monthly Usage Dollars Estimate" unit_of_measurement: '$' icon_template: mdi:currency-usd value_template: "{{ ((states('sensor.fpl_xxxxxxxxx_bill_to_date_kwh') |float) *0.1235) }}" `

and this has been working well for me. I will continue to see if I can re-format the data myself, in a custom template, but would be awesome if the data were available in a database readable format straight from the addon itself. Thank you for providing this tool! As is, It has been VERY helpful for me.

[adam-olson]

In case you had not seen this, https://developers.home-assistant.io/blog/2021/08/16/state_class_total/ , looks like it would help for this.

Adam

[ericober]

Love your work and add-on so far. To add to @adam-olson URL above, here is some additional Home Assistant documentation on the sensor entity updates for long term statistics capture.

https://www.home-assistant.io/more-info/statistics/

https://developers.home-assistant.io/docs/core/entity/sensor#long-term-statistics

Let me know if you need someone to help test this out if you ever get around to it.

[kzelnick]

the energy dashboard is looking for kWh and fpl comes in as KWH maybe the difference

[adamoutler]

I'm working on this. The system requires units and proper attributes assigned to each item. My solar system utilizes a grid return so this is important to me. I'll be done in a few days after I can finish up and double/triple check my work.

[adamoutler]

OK, I've got it all set up and operational. I'm going to give it a few days before I submit to upstream.

Additionally, there will be some changes to the sensors names and what not. It doesn't make sense to have "Bill To Date KWH 920 kWh"

[adamoutler]

@dotKrad I've made substantial changes to the sensors. I'd like to make sure we are aligned in this. Maybe you would want to try or evaluate the code before I submit? It's going to be about 2 more days as I've just made changes and I must wait for the values to level out due to the new last_reset which were just implemented in sensors relating to energy consumption. I'd prefer to make required changes sooner rather than later before I push patches.

https://github.com/adamoutler/hass-fpl/tree/master/custom_components/fpl

[kzelnick]

thank you for working on it i would be happy to look at it but still fairly new at this how do i add this to my ha, i added the original via hacs

[adamoutler]

@kzelnick for now you can remove the existing integration, then in a terminal, cd /config/custom_components; git clone https://github.com/adamoutler/hass-fpl/tree/master/custom_components/fpl Then restart HA core and add the FPL integration from Devices and Services menu.

It would be preferred to wait for official support as this is all up in the air right now, but you can do this if you'd like to try bleeding edge.

[randydgordon]

Can't seem to be able to copy the files to test? Getting a not found error. Something I'm doing wrong?

[adamoutler]

I committed so it should be available when/if the author merges it. It's a significant change so be patient.

[randydgordon]

Updated with new changes through HACS. Removed integration to get new sensors but authentication is failing.

[adamoutler]

Not sure what the problem is. It's working. Maybe get logs?

[randydgordon]

Nothing in the logs that I can see.

[Dakael]

I am having this issue too. Here is the error in the logs.

This error originated from a custom integration.

Logger: aiohttp.server Source: custom_components/fpl/fplapi.py:303 Integration: FPL (documentation, issues) First occurred: 12:28:41 PM (1 occurrences) Last logged: 12:28:41 PM

Error handling request Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/aiohttp/web_protocol.py", line 435, in _handle_request resp = await request_handler(request) File "/usr/local/lib/python3.9/site-packages/aiohttp/web_app.py", line 504, in _handle resp = await handler(request) File "/usr/local/lib/python3.9/site-packages/aiohttp/web_middlewares.py", line 117, in impl return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 60, in security_filter_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 220, in forwarded_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 28, in request_context_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 78, in ban_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 181, in auth_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 137, in handle result = await result File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 157, in post return await super().post(request, flow_id) File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 62, in wrapper result = await method(view, request, *args, **kwargs) File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 110, in post result = await self._flow_mgr.async_configure(flow_id, data) File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 252, in async_configure result = await self._async_handle_step(flow, cur_step["step_id"], user_input) File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 325, in _async_handle_step result: FlowResult = await getattr(flow, method)(user_input) File "/config/custom_components/fpl/config_flow.py", line 60, in async_step_user fplData = await api.async_get_data() File "/config/custom_components/fpl/fplapi.py", line 52, in async_get_data accountData = await self.async_get_data(account) File "/config/custom_components/fpl/fplapi.py", line 169, in async_get_data await self.getDataFromEnergyService(account, premise, currentBillDate) File "/config/custom_components/fpl/fplapi.py", line 303, in getDataFromEnergyService "netDeliveredKwh": daily["netDeliveredKwh"], KeyError: 'netDeliveredKwh'

[adamoutler]

Ok. Looks like only people with accounts configured to handle solar would have net delivered and net received. I'll add a try:except: there.

[dotKrad]

I'm working on it

[adamoutler]

Thanks @dotKrad . I am not able to handle this. Don't take my patch.

[adamoutler]

Oh, good, I didn't pull request it.

[flyboy013]

I'm having this problem too. @adamoutler mentioned that the issue is "for only people with accounts configured to handle solar". I don't have solar capability. Is there a way to reconfigure the FPL account to un-configure solar as a means to get around this error? When I first setup HA and this integration in June everything worked fine, so I'm assuming the problem was a change between then and now.

[randydgordon]

Wish I had the skills to help make this happen, but very willing to test. Working with the energy dashboard would be amazing.

[adamoutler]

I'm having this problem too. @adamoutler mentioned that the issue is "for only people with accounts configured to handle solar". I don't have solar capability. Is there a way to reconfigure the FPL account to un-configure solar as a means to get around this error? When I first setup HA and this integration in June everything worked fine, so I'm assuming the problem was a change between then and now.

That's my belief. I don't know @dotKrad says he's on it, dog gone it, don't rush perfection. The accounts which can return power to the grid apparently have additional values which are causing a null for those who don't. I was going to put something to handle it, but mr. kiloRad says he was going to handle it.

Should I handle it @dotKrad ?

[adamoutler]

From what I can see, it looks like fplApi.py needs the new data values wrapped in a try-except. The rest should be resilient.

[flyboy013]

@adamoutler Thanks. To get the integration working again, I have added two if statements before the block of code which produces the error. This will take care of the problem for the short term.

                            if ( "netDeliveredKwh" not in daily.keys()):
                                daily["netDeliveredKwh"] = 0
                            if ( "netReceivedKwh" not in daily.keys()):
                                daily["netReceivedKwh"] = 0

                            dailyUsage.append(
                                {
                                    "usage": daily["kwhUsed"],
                                    "cost": daily["billingCharge"],
                                    "date": daily["date"],
                                    "max_temperature": daily["averageHighTemperature"],
                                    "netDeliveredKwh": daily["netDeliveredKwh"],
                                    "netReceivedKwh": daily["netReceivedKwh"],
                                    "readTime": daily["readTime"],
                                }

[adamoutler]

Since you confirmed operation, I recommend submitting a pull request. kRad can take it or leave it. You can edit within the browser.

It just works for me so I can't really take your account type into account.

[Dakael]

If it helps, i added the code above to my install, works now.

[adam-olson]

Mine seems to be working as well with the new code and the extra IF statements, however I get this error in my log every so often.

2022-01-25 01:02:13 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 134, in _handle_refresh_interval
    await self._async_refresh(log_failures=True, scheduled=True)
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 265, in _async_refresh
    update_callback()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 325, in _handle_coordinator_update
    self.async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 505, in async_write_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 538, in _async_write_ha_state
    state = self._stringify_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 511, in _stringify_state
    if (state := self.state) is None:
  File "/config/custom_components/fpl/sensor_ProjectedBillSensor.py", line 10, in state
    budget = self.getData("budget_bill")
  File "/config/custom_components/fpl/fplEntity.py", line 46, in getData
    return self.coordinator.data.get(self.account).get(field)
AttributeError: 'NoneType' object has no attribute 'get'

[adamoutler]

Yeah. That's a weird problem I see as well. It's like sometimes fplapi is missing data.

[adamoutler]

This was working perfectly, but it broke during the last commit where the relevant _attr_state_class were all commented out. https://github.com/dotKrad/hass-fpl/commit/a9d343f1f5d7e214c87ae60c34885e51b35a1a75

[gouthamravee]

This was working perfectly, but it broke during the last commit where the relevant _attr_state_class were all commented out. a9d343f

had the same issue, removing the integration from HASS and HACS, restarting HASS, reinstalling and restarting fixed it.

[adamoutler]

This was working perfectly, but it broke during the last commit where the relevant _attr_state_class were all commented out. a9d343f

had the same issue, removing the integration from HASS and HACS, restarting HASS, reinstalling and restarting fixed it.

That doesn't change the fact that the code necessary for identifying the meter type was commented out.

I know, because I'm the one who added it and brought energy system support to this addon.

[adamoutler]

33 and #35 are direct results of the recent changes. The types of sensors were commented out. This issue and the other two should be closed after support is brought back for these energy total and daily sensors.

[gouthamravee]

This was working perfectly, but it broke during the last commit where the relevant _attr_state_class were all commented out. a9d343f

had the same issue, removing the integration from HASS and HACS, restarting HASS, reinstalling and restarting fixed it.

That doesn't change the fact that the code necessary for identifying the meter type was commented out.

I know, because I'm the one who added it and brought energy system support to this addon.

oh gotcha, I haven't had time to check/re-setup the energy monitoring stuff yet. I saw the sensors on my dashboard again and assumed it was working.

Thanks for your work on adding energy system support! Hope it works again soon.

[mcondren]

Checking in to see if there was any work being done to update this to work with the energy dashboard. I saw a comment that there was a refactor going on. @dotKrad

[adamoutler]

What's not working? Why is this still open?

[mcondren]

I’m getting last_reset errors in the log because it has been deprecated and is no longer valid to set it on this state type. It doesn’t happen until the end of the month, then the data is no longer valid and it starts throwing the error. The new type of ‘total_increasing’ sets its own last_reset. Or at least that’s my understanding from the error.

[mcondren]

`Logger: homeassistant.components.sensor Source: components/sensor/init.py:367 Integration: Sensor (documentation, issues) First occurred: 2:46:07 PM (1 occurrences) Last logged: 2:46:07 PM

Entity sensor.fpl_498081442_daily_usage_kwh (<class 'custom_components.fpl.sensor_DailyUsageSensor.FplDailyUsageKWHSensor'>) with state_class total_increasing has set last_reset. Setting last_reset for entities with state_class other than 'total' is not supported. Please update your configuration if state_class is manually configured, otherwise report it to the custom integration author. `

[jabetcha]

Created pull request #49 with changes that should make all the daily kWH and monthly stats usable in the energy dashboard. Please wait a few days before accepting in case it doesn't work like I expect.

[jabetcha]

My fork has been updated with code is working for reading the daily received reading and daily delivered reading (new sensors)

[GitittomeNow]

Thank you for all the work on this! Can’t wait to try it.