Open specter9mm opened 1 year ago
specter9mm
commented
1 year ago
The sensors for L1, L2, Today Total, and Last 7 days are all showing 0. This is on firmware 8. I have attached a screenshot of the consumption api path, so I know the data is there, but I do not see the consumption endpoint defined in envoy_reader.py. I am unfortunately not versed well enough to modify it myself.
catsmanac
commented
1 year ago Consumption data comes from the production.json endpoint, at least in fw d7 and if consumption CT are configured and connected.
The consumption endpoint you are sharing is which one exactly?
specter9mm
commented
1 year ago api/v1/consumption you mean?
catsmanac
commented
1 year ago Ok, I see. You say the firmware is 8, any more details on that number? I'm on D7.6.175.
Your running a metered Gateway / Envoy with CT for 2 phases on each production and consumption side looking at you screenshot?
Can you enable Debug on the Envoy, let it run for couples of minutes and disable debug again. The log file will download to your PC when disabling debug again. I'm most interested in the line where it fetched from /production and the resulting data. Feel free to post the file here as file, but best clean the tokens from it. (if you run v0.0.14-beta1 then tokens are cleaned)
specter9mm
commented
1 year ago From what I understand from the installers, I am one of the first customers to have the new controller. Here is a screenshot of the part and firmware. I have the debug running for envoy integration now.
specter9mm
commented
1 year ago catsmanac
commented
1 year ago From what I understand from the installers, I am one of the first customers to have the new controller
When 7 came out it was a bumpy ride, let's hope this is smoother. Thanks for the log, I'll have a look at it.
catsmanac
commented
1 year ago The envoy is indeed reporting 0 for consumption for whtoday and whlastsevendays (see below).
When you open /ivp/meters/readings on the envoy in your browser (may need to provide your token during that process) are you seeing consumption and production data?
The production.json report from the log:
{
"production": [{
"type": "inverters",
"activeCount": 44,
"readingTime": 1692373170,
"wNow": 8704,
"whLifetime": 305212
}, {
"type": "eim",
"activeCount": 1,
"measurementType": "production",
"readingTime": 1692373176,
"wNow": 8906.503,
"whLifetime": 1468263.802,
"varhLeadLifetime": 855.321,
"varhLagLifetime": 204551.001,
"vahLifetime": 1703599.145,
"rmsCurrent": 74.626,
"rmsVoltage": 247.102,
"reactPwr": 0.879,
"apprntPwr": 9221.107,
"pwrFactor": 0.97,
"whToday": 13102.802,
"whLastSevenDays": 1455834.802,
"vahToday": 21193.145,
"varhLeadToday": 31.321,
"varhLagToday": 8351.001,
"lines": [{
"wNow": 4458.917,
"whLifetime": 734376.431,
"varhLeadLifetime": 288.208,
"varhLagLifetime": 103581.477,
"vahLifetime": 852409.509,
"rmsCurrent": 37.309,
"rmsVoltage": 123.748,
"reactPwr": 7.588,
"apprntPwr": 4617.538,
"pwrFactor": 0.97,
"whToday": 6557.431,
"whLastSevenDays": 728156.431,
"vahToday": 10612.509,
"varhLeadToday": 10.208,
"varhLagToday": 4185.477
}, {
"wNow": 4447.586,
"whLifetime": 733887.371,
"varhLeadLifetime": 567.112,
"varhLagLifetime": 100969.524,
"vahLifetime": 851189.636,
"rmsCurrent": 37.317,
"rmsVoltage": 123.354,
"reactPwr": -6.709,
"apprntPwr": 4603.569,
"pwrFactor": 0.97,
"whToday": 6545.371,
"whLastSevenDays": 727678.371,
"vahToday": 10580.636,
"varhLeadToday": 21.112,
"varhLagToday": 4165.524
}
]
}
],
"consumption": [{
"type": "eim",
"activeCount": 1,
"measurementType": "total-consumption",
"readingTime": 1692373176,
"wNow": 8984.905,
"whLifetime": 1734930.993,
"varhLeadLifetime": 108613.238,
"varhLagLifetime": 25679.859,
"vahLifetime": 1584252.738,
"rmsCurrent": 76.484,
"rmsVoltage": 246.883,
"reactPwr": -93.923,
"apprntPwr": 9437.79,
"pwrFactor": 0.94,
"whToday": 0.0,
"whLastSevenDays": 0.0,
"vahToday": 1167916.738,
"varhLeadToday": 104656.238,
"varhLagToday": 13208.859,
"lines": [{
"wNow": 4255.948,
"whLifetime": 780975.015,
"varhLeadLifetime": 40098.095,
"varhLagLifetime": 52146.632,
"vahLifetime": 752802.237,
"rmsCurrent": 29.295,
"rmsVoltage": 123.637,
"reactPwr": 126.509,
"apprntPwr": 3621.966,
"pwrFactor": 1.0,
"whToday": 0.0,
"whLastSevenDays": 0.0,
"vahToday": 544504.237,
"varhLeadToday": 38119.095,
"varhLagToday": 45905.632
}, {
"wNow": 4728.957,
"whLifetime": 953955.978,
"varhLeadLifetime": 68515.144,
"varhLagLifetime": -26466.773,
"vahLifetime": 831450.501,
"rmsCurrent": 47.189,
"rmsVoltage": 123.246,
"reactPwr": -220.431,
"apprntPwr": 5815.824,
"pwrFactor": 0.81,
"whToday": 0.0,
"whLastSevenDays": 0.0,
"vahToday": 623412.501,
"varhLeadToday": 66537.144,
"varhLagToday": 149.0
}
]
}, {
"type": "eim",
"activeCount": 1,
"measurementType": "net-consumption",
"readingTime": 1692373176,
"wNow": 78.402,
"whLifetime": 266852.371,
"varhLeadLifetime": 109468.559,
"varhLagLifetime": 230230.86,
"vahLifetime": 1584252.738,
"rmsCurrent": 1.858,
"rmsVoltage": 246.883,
"reactPwr": -94.802,
"apprntPwr": 225.824,
"pwrFactor": -0.04,
"whToday": 0,
"whLastSevenDays": 0,
"vahToday": 0,
"varhLeadToday": 0,
"varhLagToday": 0,
"lines": [{
"wNow": -202.968,
"whLifetime": 46691.191,
"varhLeadLifetime": 40386.303,
"varhLagLifetime": 155728.109,
"vahLifetime": 752802.237,
"rmsCurrent": -8.014,
"rmsVoltage": 123.637,
"reactPwr": 118.921,
"apprntPwr": -990.788,
"pwrFactor": -1.0,
"whToday": 0,
"whLastSevenDays": 0,
"vahToday": 0,
"varhLeadToday": 0,
"varhLagToday": 0
}, {
"wNow": 281.371,
"whLifetime": 220161.18,
"varhLeadLifetime": 69082.256,
"varhLagLifetime": 74502.751,
"vahLifetime": 831450.501,
"rmsCurrent": 9.871,
"rmsVoltage": 123.246,
"reactPwr": -213.723,
"apprntPwr": 1216.612,
"pwrFactor": 0.23,
"whToday": 0,
"whLastSevenDays": 0,
"vahToday": 0,
"varhLeadToday": 0,
"varhLagToday": 0
}
]
}
],
"storage": [{
"type": "acb",
"activeCount": 0,
"readingTime": 0,
"wNow": 0,
"whNow": 0,
"state": "idle"
}
]
}[
{
"eid": x,
"timestamp": 1692293413,
"actEnergyDlvd": 1388678.209,
"actEnergyRcvd": 205.052,
"apparentEnergy": 1609111.767,
"reactEnergyLagg": 191301.817,
"reactEnergyLead": 824.062,
"instantaneousDemand": 12332.893,
"activePower": 12332.893,
"apparentPower": 12664.479,
"reactivePower": 67.527,
"pwrFactor": 0.974,
"voltage": 247.215,
"current": 102.451,
"freq": 60.000,
"channels": [
{
"eid": x,
"timestamp": 1692293413,
"actEnergyDlvd": 694573.233,
"actEnergyRcvd": 100.420,
"apparentEnergy": 805121.816,
"reactEnergyLagg": 96869.002,
"reactEnergyLead": 278.624,
"instantaneousDemand": 6154.442,
"activePower": 6154.442,
"apparentPower": 6322.906,
"reactivePower": 43.895,
"pwrFactor": 0.972,
"voltage": 123.450,
"current": 51.216,
"freq": 60.000
},
{
"eid": x,
"timestamp": 1692293413,
"actEnergyDlvd": 694104.976,
"actEnergyRcvd": 104.632,
"apparentEnergy": 803989.951,
"reactEnergyLagg": 94432.815,
"reactEnergyLead": 545.438,
"instantaneousDemand": 6178.451,
"activePower": 6178.451,
"apparentPower": 6341.572,
"reactivePower": 23.632,
"pwrFactor": 0.976,
"voltage": 123.765,
"current": 51.235,
"freq": 60.000
},
{
"eid": x,
"timestamp": 1692293413,
"actEnergyDlvd": 0.000,
"actEnergyRcvd": 0.000,
"apparentEnergy": 0.000,
"reactEnergyLagg": 0.000,
"reactEnergyLead": 0.000,
"instantaneousDemand": 0.000,
"activePower": 0.000,
"apparentPower": 0.000,
"reactivePower": 0.000,
"pwrFactor": 0.000,
"voltage": 0.000,
"current": 0.000,
"freq": 60.000
}
]
},
{
"eid": x,
"timestamp": 1692293413,
"actEnergyDlvd": 741099.134,
"actEnergyRcvd": 509862.380,
"apparentEnergy": 1490840.749,
"reactEnergyLagg": 214201.626,
"reactEnergyLead": 103003.810,
"instantaneousDemand": -7231.213,
"activePower": -7231.213,
"apparentPower": 7719.918,
"reactivePower": 670.704,
"pwrFactor": -0.935,
"voltage": 247.238,
"current": 62.398,
"freq": 60.000,
"channels": [
{
"eid": x,
"timestamp": 1692293413,
"actEnergyDlvd": 314224.143,
"actEnergyRcvd": 277938.999,
"apparentEnergy": 710294.227,
"reactEnergyLagg": 144869.888,
"reactEnergyLead": 38263.251,
"instantaneousDemand": -3336.928,
"activePower": -3336.928,
"apparentPower": 3571.307,
"reactivePower": 626.993,
"pwrFactor": -0.933,
"voltage": 123.442,
"current": 28.929,
"freq": 60.000
},
{
"eid": x,
"timestamp": 1692293413,
"actEnergyDlvd": 426874.991,
"actEnergyRcvd": 231923.380,
"apparentEnergy": 780546.522,
"reactEnergyLagg": 69331.739,
"reactEnergyLead": 64740.560,
"instantaneousDemand": -3894.284,
"activePower": -3894.284,
"apparentPower": 4148.611,
"reactivePower": 43.711,
"pwrFactor": -0.937,
"voltage": 123.797,
"current": 33.469,
"freq": 60.000
},
{
"eid": x,
"timestamp": 1692293413,
"actEnergyDlvd": 53312.306,
"actEnergyRcvd": 71374.037,
"apparentEnergy": 201581.859,
"reactEnergyLagg": 5395.541,
"reactEnergyLead": 1901.672,
"instantaneousDemand": -26.526,
"activePower": -26.526,
"apparentPower": 334.345,
"reactivePower": 1.826,
"pwrFactor": -0.133,
"voltage": 123.835,
"current": 2.699,
"freq": 60.000
}
]
},
{
"eid": x,
"timestamp": 1692293413,
"actEnergyDlvd": 106639.180,
"actEnergyRcvd": 142879.027,
"apparentEnergy": 403403.866,
"reactEnergyLagg": 10798.594,
"reactEnergyLead": 3803.596,
"instantaneousDemand": 241.614,
"activePower": 241.614,
"apparentPower": 533.442,
"reactivePower": 0.053,
"pwrFactor": -0.610,
"voltage": 247.277,
"current": 31.628,
"freq": 60.000,
"channels": [
{
"eid": x,
"timestamp": 1692293413,
"actEnergyDlvd": 53326.874,
"actEnergyRcvd": 71504.990,
"apparentEnergy": 201822.007,
"reactEnergyLagg": 5403.053,
"reactEnergyLead": 1901.924,
"instantaneousDemand": 268.140,
"activePower": 268.140,
"apparentPower": 199.097,
"reactivePower": -1.772,
"pwrFactor": -0.933,
"voltage": 123.442,
"current": 28.929,
"freq": 60.000
},
{
"eid": x,
"timestamp": 1692293413,
"actEnergyDlvd": 53312.306,
"actEnergyRcvd": 71374.037,
"apparentEnergy": 201581.859,
"reactEnergyLagg": 5395.541,
"reactEnergyLead": 1901.672,
"instantaneousDemand": -26.526,
"activePower": -26.526,
"apparentPower": 334.345,
"reactivePower": 1.826,
"pwrFactor": -0.133,
"voltage": 123.835,
"current": 2.699,
"freq": 60.000
},
{
"eid": x,
"timestamp": 1692293413,
"actEnergyDlvd": 0.000,
"actEnergyRcvd": 0.000,
"apparentEnergy": 0.000,
"reactEnergyLagg": 0.000,
"reactEnergyLead": 0.000,
"instantaneousDemand": 0.000,
"activePower": 0.000,
"apparentPower": 0.000,
"reactivePower": 0.000,
"pwrFactor": 0.000,
"voltage": 0.000,
"current": 0.000,
"freq": 0.000
}
]
}
]That one looks same as before.
When looking at the consumption numbers in production.json
"measurementType": "total-consumption",
"wNow": 8984.905,
"whLifetime": 1734930.993,
"measurementType": "net-consumption",
"wNow": 78.402,
"whLifetime": 266852.371,
and the picture of api/v1/consumption I see there ~200KWh difference for whLifetime for total consumption. Which one might be correct?
The api/v1/consumption is only overall and does not include data for l1 and l2, so these will miss for sure.
Any input from your installer to these missing numbers? All configured as it should?
specter9mm
commented
1 year ago I collected that info from the /ivp/meters/readings some time ago, and just copied it from my notepad, so it has old data in there. The screenshot is 2 or 3 days newer.
catsmanac
commented
1 year ago Are the numbers reported by /api/v1/consumption correct i.e. the same as what enphase site shows? The v1 pages reportedly have different numbers when CT are used, but maybe not always.
@Steve2017 and @madbrain76 you both have metered envoy on D7 firmware. Do you see data in 'Today's Energy consumption', 'last Seven Days Energy consumption', 'Today's Energy consumption L1/L2/L3' and/or last 'Last Seven Days Energy consumption L1/L2/L3' ?
Steve2017
commented
1 year ago Yes - although only L1 for me - on single phase.
This is off the integration page:
Plus the L1 that I have disabled - and all the micro-inverters.
EDIT: Sorry I did that late at night and for some reason deleted all the data readings. They all had readings, although current power production was of course reading 0.0kW.
madbrain76
commented
1 year ago @catsmanac , I'm running the 0.14 beta integration now, FYI.
Both my IQ gateways return the following for the /api/v1/consumption URI :
{ "wattHoursToday": 0, "wattHoursSevenDays": 0, "wattHoursLifetime": 0, "wattsNow": 0 }
The Envoy-S returns JSON and 503 error code.
As far as the sensors shown for my 3 Gateways , here they are : 1) for my 10/9/2022 IQ Envoy (Northwest panels) :
Obviously, a bunch of sensors are wrong there. There shouldn't be anything under consumption sensors, as there is no consumption CT on this gateway. Yet there are some non-zero values, some even quite high.
2) 7/24/2023 IQ Envoy PCB
3) 8/14/2023 Envoy-S standard :
(Yes, I know microinverter ending 8618 has a problem as it's showing 0W)
catsmanac
commented
1 year ago Thanks @Steve2017 and @madbrain76, comparing with @specter9mm D8 firmware data there seem to have changed something (again).
As for @madbrain76 's 10/9/2022 IQ Envoy (Northwest panels) returning data on non existing CT's, that has been reported in various chats as well where not used segments may contain any kind of data not making sense.
@specter9mm that is also the reason for me asking if the data in api/vi/consumption is valid data. In the past when using ct's those api/v1 pages could contain anything. Maybe that's fixed in this version and we can use it to report todays and last seven days consumption. Would be great if you can confirm the data is correct.
But it seems to imply that with update to D8 firmware todays and last seven days consumption for the phases is lost.
For this custom integration it will require some change to become firmware version is aware. When @briancmpbll started this it was sufficient to look at what the pages reported to know how to handle it. That seems no longer the case looking at @specter9mm's data set.
I see that the new HA Core Envoy integration which will become available in September just added support for the D8 firmware as well, but not sure if they are aware of this issue.
specter9mm
commented
1 year ago I did get a hold of the HA devs on their discord and sent them a bunch of info as well.
catsmanac
commented
1 year ago Ah ok, that's why the model data they had for D8 looked so familiar 😃 . In the long term that one should probably be the path forward. The 0.0.14 here was created to fix some issues around old legacy devices that HA Core will not support. For these devices and mixed environments with legacy (LCD fw< R3.9) and new this one may be around for a bit.
Still need to look into how big a change it will be to become fw aware or if there's an other way to determine from the data if api/v1/consumption should be used.
specter9mm
commented
1 year ago That being the case I will probably hold off reinstalling until the September release then. The whole reason I set it up was to track how much power I was generating back to the grid from my solar install. The Smart Meter Texas integrations is also lacking pulling the generation stats into a sensor, even though it's available via the api. I just don't have the time/knowledge to do a custom integration for it.
catsmanac
commented
1 year ago Well you can use the lifetime energy production and energy lifetime consumption and specify those in the energy dashboard. It will recognize it's an ever increasing number and show hourly and daily results from it. I have my envoy lifetime production in the energy dashboard, It's not required to use a daily value.
specter9mm
commented
1 year ago @catsmanac the problem I had there is it is a negative value right now, which the energy dashboard doesn't like, and I am not knowledgeable enough to figure out how to make that work. Here is the custom sensor I made to try to do that.
- sensor:
- unique_id: exporting
state_class: total_increasing
device_class: "energy"
state: >
{{ '%0.1f' | format(states('sensor.envoy_xxxxxxxxxxxx_lifetime_energy_production') | float -
states('sensor.envoy_xxxxxxxxxxxx_lifetime_energy_consumption') | float -
states('sensor.envoy_xxxxxxxxxxxx_battery_energy_charged') | float) }}
unit_of_measurement: 'Wh'I assume the issue is with the grid consumption and return to grid values. What you would have to do is each time a new value comes in the change in value is assigned to consumption or returned to grid. Below example is based of one I use and some googling:
- trigger:
- platform: state
entity_id:sensor.envoy_xxxxxxxxxxxx_lifetime_energy_consumption
sensor:
- name: "consumption from grid"
unique_id: "xyz"
unit_of_measurement: "Wh"
device_class: "energy"
state: >
{% set delta = trigger.to_state.state | float - trigger.from_state.state | float %}
{{ delta | float(0) }}
{% if delta > 0 %}
delta
{% else %}
0
{% endif %}
- trigger:
- platform: state
entity_id:sensor.envoy_xxxxxxxxxxxx_lifetime_energy_consumption
sensor:
- name: "production to grid"
unique_id: "xyz"
unit_of_measurement: "Wh"
device_class: "energy"
state: >
{% set delta = trigger.from_state.state - trigger.to_state.state | float | float %}
{{ delta | float(0) }}
{% if delta > 0 %}
delta
{% else %}
0
{% endif %}For battery you can use sensor.envoy_xxxxxxxxxxxx_battery_energy_charged and sensor.envoy_xxxxxxxxxxxx_battery_energy_discharged. And sensor.envoy_xxxxxxxxxxxx_lifetime_energy_production for the solarproduction parameter.
@cddu33 is working on adding import and export numbers and these may be in test phase in a couple of days as well. That uses the /ivp/meters/readings you tested before and has data on your firmware as well.
wittelw
commented
1 year ago @catsmanac > In the long term that one should probably be the path forward. The 0.0.14 here was created to fix some issues around old legacy devices that HA Core will not support.
I'm a relatively new user just monitoring one Envoy Firmware: D7.3.130, and it seems to be tracking the data in the Envoy Enlighten Android app fairly well.
I'm grateful I found this custom integration b/c I was unable to connect with D7 on the previous HA cores, however it sounds like Home Assistant 2023.9.0 now supports tokens.
Would I be of any assistance continuing on this custom integration to add additional coverage for the releases, or would you suggest moving over to HA core now?
catsmanac
commented
1 year ago Thanks for asking @wittelw, any tester is appreciated but switching over the HA Core if it meets your needs makes sense in my mind. It's new so it might not offer yet all features or even some different ones, that should be the driver.
As in your case, this one helped me out when hitting the token fence earlier this year and joining this community has been and still is worthwhile. An updated version with some new entities is coming here, but these are not in HA Core, so might mean a break in the future when wanting to switch. I hope to be able to propose those new features also to HA Core so compatibility between this and HA Core remains, but it remains to be seen how well that will work out.
wittelw
commented
1 year ago Thanks @catsmanac! Since this repo will be staying around for awhile, and sounds like an incubator for ideas that may eventually end up in HA Core, I'm content to stay here test new builds as they are released. I'm personally not too worried about breaking changes if I eventually want / have to move to HA Core. I likely will have a second Envoy in the next six to twelve months, so that may have newer firmware / hardware than what I'm running now so this may be the ideal spot to try it out.