Issue with certain lifetime sensor values

[addd45]

The problem

Hi there,

it seems my enphase setup is either corrupt somehow or I am misunderstanding the docs

1. My lifetime net energy production just does not work and is always 'unavailable', however my lifetime net energy consumption appears to work just fine. In fact, there does not appear to be much consistency to which sensors work or not. As a reference, I can see all the details in the envoy application
2. I notice my lifetime consumption is not in sync with what appears on the envoy app either. I have a suspicion that it may be due to a stat correction I had the enphase team do, however, it is only affecting lifetime consumption and not production, so that does not really make sense either. I am attaching the diag report, and in there you will find that it has my lifetime consumption at around 25 MWh, (2502871..), but the app shows a much different 21 MWh:

I'm just hoping to get some answers to these issues as it is currently blocking my ability to use the energy dashboard. I've searched issues and read the docs and it appears to be issues specific to my setup.

thanks in advanced for any advice!

What version of Home Assistant Core has the issue?

core-2024.10.4

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant Container

Integration causing the issue

enphase envoy

Link to integration documentation on our website

https://www.home-assistant.io/integrations/enphase_envoy

Diagnostics information

config_entry-enphase_envoy-01J80P78K1QZQ457A7BDDQC8A5.json

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[home-assistant[bot]]

Hey there @bdraco, @cgarwood, @joostlek, @catsmanac, mind taking a look at this issue as it has been labeled with an integration (enphase_envoy) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `enphase_envoy` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign enphase_envoy` Removes the current integration label and assignees on the issue, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

---

enphase_envoy documentation enphase_envoy source _{^{(message by IssueLinks)}}

[catsmanac]

Hi @addd45, your diagnostic files shows that your consumption CT is total consumption which only measure the House load (Load only). So there is no direct measurement of the grid exchange and hence no lifetime net energy production available.

The Envoy calculates a net-consumption value which is available as (disabled) 'lifetime balanced net energy consumption'. This is grid import - grid export and will go up and down in time. Negative changes are export, positive changes are import. You would need to implement a template to split these changes into 2 new measurements.

As for the lifetime consumption, the value you mention comes from the Envoy, not sure what impact 'may be due to a stat correction I had the enphase team do' may have.

[addd45]

Thanks for the quick reply! I wondered if this was the case, but I think I was confused on why the entities were enabled/disabled by default. I thought the sensors were originally working and broke after some time, but perhaps I misremember that.

Regarding the issue with lifetime consumption values being mismatched, are we considering it may be a glitch with Enphase app / api? What is perplexing to me is how the data the app receives differs, because in theory the app should be using the same API (and thus data source) as this integration right? Just from a common sense check, the production value should definitely be higher than the consumption value, so in any sense the data is incorrect. If necessary I could compare the data with that from my utility but I think in any case there is still a clear discrepancy to be solved here.

Any suggestions?

[catsmanac]

What is perplexing to me is how the data the app receives differs, because in theory the app should be using the same API (and thus data source) as this integration right?

The App is getting data from the Enlighten cloud. The Enlighten cloud receives a dataset, typically every 15 min, pushed by the Envoy. We don't know what processing the cloud does with the data nor what is in the data set. HA pulls data locally from the Envoy API. So they don't use the same API. In the end, all data comes from the same sources, the inverters and CT if installed. So in theory values should match, or at least be close.

Looking at your diagnostics file I see that the CT endpoint and Envoy production endpoint values match for PV production and match the value in the App.

The consumption values don't match at all and the App shows yet another value. The consumption value reported is higher then the CT value. This is strange as one would expect same values.

	Wh
CT endpoint Lifetime production (actEnergyDlvd)	2.412.425
CT endpoint House/Load lifetime consumption (actEnergyDlvd)	2.350.787
Envoy production endpoint lifetime production (whLifetime)	2.412.425
Envoy production endpoint lifetime total-consumption (whLifetime)	2.502.871
Envoy production endpoint lifetime net-consumption (whLifetime)	90.445

The Envoy production endpoint lifetime net-consumption value is just the calculated difference between lifetime total-consumption and lifetime production.

Just from a common sense check, the production value should definitely be higher than the consumption value,

The CT report a slighter higher production versus consumption, so you have a lifetime net export of 2.4 - 2.3. Assuming that the raw CT values are correct.

I have a suspicion that it may be due to a stat correction I had the enphase team do

What was this?

These are the lifetime values. The HA energy dashboard uses changes in the lifetime values to calculate daily or period values. So even if the absolute numbers don't match, changes in them may be correct. So what you can check is like yesterdays numbers in both HA and App and see if these are close. And you can then also compare these to the utility data.

[addd45]

I suppose for my HA purposes I care less about the lifetime being in sync with the enphase app, however, the delta between production and consumption must be correct in order to get correct dashboard data, which I have doubts is the case. For one, the net lifetime balance is incorrect, so I'm not sure how to get this right.

What was this? So my enphase system was misconfiguration in terms of phase configuration, so the CT values were reading incorrect values, so I had enphase support reset the "start date" and throw away older values. I would think this would impact production and consumption values though?

Regarding the home assistant energy dashboard, I had updated it yesterday so it wasn't really reliable data, however I can already tell that today it isn't correct

At the time of this screen shot, I was producing 6.552 kw and only consuming 593W.

To validate my lifetime net sensors I am using for my energy dashboard, here is the template sensors I am using, does this appear correct? Since my net production is (incorrectly) configured to be 0, because my net consumption is >0 , I feel like something isn't right...

Here is my lifetime net energy consumed sensor template

{{ states('sensor.enphase_lifetime_net_energy_balanced') if states('sensor.enphase_lifetime_net_energy_balanced') | float(0) > 0 else 0 }}

and the opposite, here is net energy produced

{{ states('sensor.enphase_lifetime_net_energy_balanced') | float(0) | abs if states('sensor.enphase_lifetime_net_energy_balanced') | float(0) < 0 else 0 }}

[catsmanac]

so I had enphase support reset the "start date" and throw away older values. I would think this would impact production and consumption values though?

Generally yes, but guess the devil is in the details of what they changed.

I feel like something isn't right...

Your templates need to detect a change in value, if positive add to one template, if negative to another one. Currently you select negative or positive values, but not the changes.

Use a trigger template in your configuration.yaml or in templates.yaml if you include that in configuration.yaml. Below example should split positive and negative values in kWh. It should be about right, let alone typo's

- trigger:
    - platform: state
      entity_id: sensor.enphase_<sn>_lifetime_balanced_net_consumption
      not_to:
        - "unknown"
        - "unavailable"
      not_from:
        - "unknown"
        - "unavailable"
  sensor:
    - name: "envoy grid export"
      unique_id: envoy_grid_export
      unit_of_measurement: "kWh" 
      icon: mdi:transmission-tower 
      state: "{{ this.state | int(0) + ([0, (trigger.to_state.state | int(0) - trigger.from_state.state | int(0))] | min  | abs) }}" 
      device_class: energy 
      state_class: total_increasing 
    - name: "envoy grid import"
      unique_id: envoy_grid_import
      unit_of_measurement: "kWh" 
      icon: mdi:transmission-tower 
      state: "{{ this.state | int(0) + ([0, (trigger.to_state.state | int(0) - trigger.from_state.state | int(0))] | max | abs) }}" 
      device_class: energy 
      state_class: total_increasing

[addd45]

This has been really helpful and again I appreciate you taking the time to help me set this up.

At this point I just want to take a step back and evaluate the best way to configure my dashboard. I can certainly take the trigger sensor you share and I can be good to go, but what I still want to know (and the HA docs surely are lacking on) is what a "normal" setup is expecting for import/export. Is the dashboard expecting a constantly increasing value (indefinitely)? Or would daily / current sensors work better? E.g. today production / today consumption (calculate export and import from these). Sure it might be a bit more work to create a sensor from these values but a pro I would presume is these values would be more reliable and fine-grained.

I also have questions about how state would work for the trigger sensor, as both import and export essentially don't maintain state, but only increment based on the lifetime net sensor changes, meaning these sensors would be reset (leveled?) upon a Home Assistant restart or reload.

Nothing against your sensor idea by the way, it's brilliant! It just feels odd to have to hack a sensor together to get it working in the HA dashboard. Probably my gripe is with the energy dashboard :)

[catsmanac]

Is the dashboard expecting a constantly increasing value (indefinitely)?

No, it will accept also daily or period energy values and will handle the daily or period reset properly. For the Envoy in general we are recommending to use the lifetime value as we had reports that in some, but definitely not all, cases the daily values showed unexpected values. For your installation the reverse may be the case and daily values may work better.

It just feels odd to have to hack a sensor together to get it working in the HA dashboard

The fundamental issue you are running into is that the Energy Dashboard requires 2 separate entities for grid import and grid export while your consumption CT meter is installed as 'Load Only/Total consumption'. The latter results in the absence of split values for grid import/export as only a balanced value of these 2 is available in the Envoy data. Would the consumption CT have been installed as a 'net-consumption/Load with solar included CT' then the grid import and export values would be available as separate values in the Envoy data right away. (Not saying it should be installed as net-consumption, that is dependent on local situation, rules and alike, only sharing what data is available for each configuration.)

I also have questions about how state would work for the trigger sensor, as both import and export essentially don't maintain state, but only increment based on the lifetime net sensor changes, meaning these sensors would be reset (leveled?) upon a Home Assistant restart or reload.

The states of the calculated import and export sensors are permanent and are maintained over restart. Each time the trigger fires the states for the 2 are updated. After a restart, the states are available again. It may need some testing how it exactly behaves over restart, the 'not_to' and 'not_from' are intended to assist with handling restart as well.

In the end I'll expect you'll always have to split into 2 values in each option to use, but I'll be glad to learn about another option.

[addd45]

Just to update here, I decided to use the values from current net power to build sensors for "current net power imported" and "current net power exported"

example for imported:

{{ states('sensor.enphase_net_power_balanced') if states('sensor.enphase_net_power_balanced') | float(0) > 0 else 0 }}

this will give the power (in kw) using the balanced sensor. If its positive, it reflects current import. If it's negative, it reflects export.

With these 2 sensors (in kw), I created 2 additional integral sensors, which will convert the sensors from kW to kWh. This is what I added to my energy dashboard.

I did similar for production, take current power produced sensor (in kw) and create an integral sensor helper to convert to kWh.

One downside for this is that these integral sensors still continuously increase indefinitely and don't "reset". It still "works" but these integral sensors don't really reflect "live" values, which is still what I would want to have the energy dashboard be fully correct. My energy usage graph and production graph look correct (because these track changes). However, the "energy distribution" graph will never be correct because that appears to use the current state values which just continually increase.

When looking into why integral sensors seem to continually increase, it seems to be a known issue (expected behavior?). I don't really understand why, if an integral period uses 'hour' I would think it calculates always using these bounds. I did see that a potential solution is to use another helper, utility meter. It is definitely a bit frustrating though to have create almost 10 helpers to get everything working properly, and even using utility feels like a hack because there isn't really a "reset time" unless I decide to pick my own, which would be 12:00AM.

But this isn't an issue with this integration I guess, it's an issue with the massive lack of flexibility with the Energy Dashboard. All this being said, if I could add a suggestion, the enphase integration docs could be updated to explain more about setups like mine where net sensors are not natively available, and how to create the required template sensors. If I understand correctly, this is the standard setup for when a battery is not used, which would probably be more common?

Thanks again for all the help

[catsmanac]

I decided to use the values from current net power to build sensors for "current net power imported" and "current net power exported"

Definitely one of the options to use.

However, the "energy distribution" graph will never be correct because that appears to use the current state values which just continually increase.

I'm using ever increasing values since almost 2 years and it works fine. Make sure to add

      state_class: total_increasing

in your template definition.

if an integral period uses 'hour' I would think it calculates always using these bounds

The period is used to calculate the target UOM from the source UOM, i.e. to arrive at hour for kWh,, it's not the time it will get reset.

I did see that a potential solution is to use another helper, utility meter

Yes and the utility meter has a reset, so if you specify day you'll get daily numbers building up throughout the day. Although it's not required for the Energy dashboard as I mentioned.

if I could add a suggestion, the enphase integration docs could be updated to explain more about setups like mine where net sensors are not natively available, and how to create the required template sensors.

That is on the list to do as we're changing from getting it to work to make it simple to use as we learn about all those variants in use. Of course you are always welcome to contribute or propose changes to the docs