search

briancmpbll / home_assistant_custom_envoy

177 stars 77 forks source link

Current Consumption no longer available in 0.0.19 #175

Open Indemnity83 opened 10 months ago

Indemnity83 commented 10 months ago

Before upgrading to 0.0.19, I had a current power consumption number that for all that I could tell was accurate, but I upgraded last night and now 'current power consumption' shows up as unavailable.

Is this a configuration thing I need to change/fix somewhere, or maybe a bug introduced in the latest version?

I have an Envoy controller, Firmware: D7.3.130, Hardware: 800-00695-r01 with CT's on both the solar backfeed and panel main.

Indemnity83 commented 10 months ago

So here's an interesting turn; I dug into the logs a bit and saw that it wasn't just the integration that stopped giving power consumption; the logs showed that the envoy controller wasn't providing it either. As I fiddled with it this morning, the sun came up and it just suddenly started working

Nevermind ... I need more sleep, its still not providing consumption. :P

catsmanac commented 10 months ago

Depending on which version you upgraded from, there are changes, but the Current Power Consumption should still work. So could be a bug or yet another firmware surprise.

To identify what is going on you best have a look at the diagnostic report. On the HA envoy device page there's a button download diagnostics. It will download the diagnostic report to your pc. The report contains all configured entities, the last value as well as the raw data received from the envoy in the last collection cycle.

Look for lines starting with endpoint like below. When using CT's the Endpoint-meters-reports and Endpoint-meters-readings should have data (the example doesn't). For current power consumption look for currW in a section that also contains the term total-consumption in the Endpoint-meters-reports line.

"Endpoint-meters": null,
"Endpoint-meters-readings": null,
"Endpoint-meters-reports": null,
"Endpoint-production_json": "{\"production\":[{\"type\":\"inverters\",\"activeCount\":24,\"readingTime\":1698217399,\"wNow\":11,\"whLifetime\":1032125}],\"storage\":[{\"type\":\"acb\",\"activeCount\":0,\"readingTime\":0,\"wNow\":0,\"whNow\":0,\"state\":\"idle\"}]}",

There is a change in 0.0.19 to handle differences in how data was reported in version 7.6 firmware. The change was made as if it was needed for all of version 7.x. As you are running 7.3 the change may not be with the start of 7.0 but only as of 7.6 and 7.3 still needs the old way. The 2 endpoint lines should be able to show if this is the case.

Indemnity83 commented 9 months ago

This is what I see, somewhat hard to parse with all the escapes but I do see 'total-consumption' in there.

"Endpoint-meters": "[\n    {\n        \"eid\": 704643328,\n        \"state\": \"disabled\",\n        \"measurementType\": \"production\",\n        \"phaseMode\": \"split\",\n        \"phaseCount\": 2,\n        \"meteringStatus\": \"normal\",\n        \"statusFlags\": []\n    },\n    {\n        \"eid\": 704643584,\n        \"state\": \"disabled\",\n        \"measurementType\": \"net-consumption\",\n        \"phaseMode\": \"split\",\n        \"phaseCount\": 2,\n        \"meteringStatus\": \"normal\",\n        \"statusFlags\": []\n    }\n]",
"Endpoint-meters-readings": "[\n    {\n        \"eid\": 704643328,\n        \"timestamp\": 1703362371,\n        \"actEnergyDlvd\": 0.000,\n        \"actEnergyRcvd\": 0.000,\n        \"apparentEnergy\": 0.000,\n        \"reactEnergyLagg\": 0.000,\n        \"reactEnergyLead\": 0.000,\n        \"instantaneousDemand\": 4583.833,\n        \"activePower\": 4583.833,\n        \"apparentPower\": 4610.292,\n        \"reactivePower\": 3.465,\n        \"pwrFactor\": 0.994,\n        \"voltage\": 246.633,\n        \"current\": 37.384,\n        \"freq\": 60.000,\n        \"channels\": [\n            {\n                \"eid\": 1778385169,\n                \"timestamp\": 1703362371,\n                \"actEnergyDlvd\": 0.000,\n                \"actEnergyRcvd\": 0.000,\n                \"apparentEnergy\": 0.000,\n                \"reactEnergyLagg\": 0.000,\n                \"reactEnergyLead\": 0.000,\n                \"instantaneousDemand\": 2289.035,\n                \"activePower\": 2289.035,\n                \"apparentPower\": 2300.896,\n                \"reactivePower\": 15.940,\n                \"pwrFactor\": 0.995,\n                \"voltage\": 123.106,\n                \"current\": 18.689,\n                \"freq\": 60.000\n            },\n            {\n                \"eid\": 1778385170,\n                \"timestamp\": 1703362371,\n                \"actEnergyDlvd\": 0.000,\n                \"actEnergyRcvd\": 0.000,\n                \"apparentEnergy\": 0.000,\n                \"reactEnergyLagg\": 0.000,\n                \"reactEnergyLead\": 0.000,\n                \"instantaneousDemand\": 2294.798,\n                \"activePower\": 2294.798,\n                \"apparentPower\": 2309.396,\n                \"reactivePower\": -12.475,\n                \"pwrFactor\": 0.993,\n                \"voltage\": 123.527,\n                \"current\": 18.695,\n                \"freq\": 60.000\n            },\n            {\n                \"eid\": 1778385171,\n                \"timestamp\": 1703362371,\n                \"actEnergyDlvd\": 0.000,\n                \"actEnergyRcvd\": 0.000,\n                \"apparentEnergy\": 0.000,\n                \"reactEnergyLagg\": 0.000,\n                \"reactEnergyLead\": 0.000,\n                \"instantaneousDemand\": 0.000,\n                \"activePower\": 0.000,\n                \"apparentPower\": 0.000,\n                \"reactivePower\": 0.000,\n                \"pwrFactor\": 0.000,\n                \"voltage\": 0.000,\n                \"current\": 0.000,\n                \"freq\": 60.000\n            }\n        ]\n    },\n    {\n        \"eid\": 704643584,\n        \"timestamp\": 1703362371,\n        \"actEnergyDlvd\": 0.000,\n        \"actEnergyRcvd\": 0.000,\n        \"apparentEnergy\": 0.000,\n        \"reactEnergyLagg\": 0.000,\n        \"reactEnergyLead\": 0.000,\n        \"instantaneousDemand\": -2658.794,\n        \"activePower\": -2658.794,\n        \"apparentPower\": 2774.206,\n        \"reactivePower\": 86.929,\n        \"pwrFactor\": -0.957,\n        \"voltage\": 246.803,\n        \"current\": 22.524,\n        \"freq\": 60.000,\n        \"channels\": [\n            {\n                \"eid\": 1778385425,\n                \"timestamp\": 1703362371,\n                \"actEnergyDlvd\": 0.000,\n                \"actEnergyRcvd\": 0.000,\n                \"apparentEnergy\": 0.000,\n                \"reactEnergyLagg\": 0.000,\n                \"reactEnergyLead\": 0.000,\n                \"instantaneousDemand\": -1275.379,\n                \"activePower\": -1275.379,\n                \"apparentPower\": 1330.887,\n                \"reactivePower\": -284.765,\n                \"pwrFactor\": -0.959,\n                \"voltage\": 123.202,\n                \"current\": 10.842,\n                \"freq\": 60.000\n            },\n            {\n                \"eid\": 1778385426,\n                \"timestamp\": 1703362371,\n                \"actEnergyDlvd\": 0.000,\n                \"actEnergyRcvd\": 0.000,\n                \"apparentEnergy\": 0.000,\n                \"reactEnergyLagg\": 0.000,\n                \"reactEnergyLead\": 0.000,\n                \"instantaneousDemand\": -1383.415,\n                \"activePower\": -1383.415,\n                \"apparentPower\": 1443.318,\n                \"reactivePower\": 371.694,\n                \"pwrFactor\": -0.955,\n                \"voltage\": 123.601,\n                \"current\": 11.681,\n                \"freq\": 60.000\n            },\n            {\n                \"eid\": 1778385427,\n                \"timestamp\": 1703362371,\n                \"actEnergyDlvd\": 0.000,\n                \"actEnergyRcvd\": 0.000,\n                \"apparentEnergy\": 0.000,\n                \"reactEnergyLagg\": 0.000,\n                \"reactEnergyLead\": 0.000,\n                \"instantaneousDemand\": 0.000,\n                \"activePower\": 0.000,\n                \"apparentPower\": 0.000,\n                \"reactivePower\": 0.000,\n                \"pwrFactor\": 0.000,\n                \"voltage\": 0.000,\n                \"current\": 0.000,\n                \"freq\": 60.000\n            }\n        ]\n    }\n]",
"Endpoint-meters-reports": "[\n    {\n        \"createdAt\": 1703362372,\n        \"reportType\": \"production\",\n        \"cumulative\": {\n            \"currW\": 4583.833,\n            \"actPower\": 4583.833,\n            \"apprntPwr\": 4610.292,\n            \"reactPwr\": 3.465,\n            \"whDlvdCum\": 0.000,\n            \"whRcvdCum\": 0.000,\n            \"varhLagCum\": 0.000,\n            \"varhLeadCum\": 0.000,\n            \"vahCum\": 0.000,\n            \"rmsVoltage\": 246.633,\n            \"rmsCurrent\": 37.384,\n            \"pwrFactor\": 0.99,\n            \"freqHz\": 60.00\n        },\n        \"lines\": [\n            {\n                \"currW\": 2289.035,\n                \"actPower\": 2289.035,\n                \"apprntPwr\": 2300.896,\n                \"reactPwr\": 15.940,\n                \"whDlvdCum\": 0.000,\n                \"whRcvdCum\": 0.000,\n                \"varhLagCum\": 0.000,\n                \"varhLeadCum\": 0.000,\n                \"vahCum\": 0.000,\n                \"rmsVoltage\": 123.106,\n                \"rmsCurrent\": 18.689,\n                \"pwrFactor\": 1.00,\n                \"freqHz\": 60.00\n            },\n            {\n                \"currW\": 2294.798,\n                \"actPower\": 2294.798,\n                \"apprntPwr\": 2309.396,\n                \"reactPwr\": -12.475,\n                \"whDlvdCum\": 0.000,\n                \"whRcvdCum\": 0.000,\n                \"varhLagCum\": 0.000,\n                \"varhLeadCum\": 0.000,\n                \"vahCum\": 0.000,\n                \"rmsVoltage\": 123.527,\n                \"rmsCurrent\": 18.695,\n                \"pwrFactor\": 0.99,\n                \"freqHz\": 60.00\n            }\n        ]\n    },\n    {\n        \"createdAt\": 1703362372,\n        \"reportType\": \"net-consumption\",\n        \"cumulative\": {\n            \"currW\": -2658.794,\n            \"actPower\": -2658.794,\n            \"apprntPwr\": 2774.206,\n            \"reactPwr\": 86.929,\n            \"whDlvdCum\": 0.000,\n            \"whRcvdCum\": 0.000,\n            \"varhLagCum\": 0.000,\n            \"varhLeadCum\": 0.000,\n            \"vahCum\": 0.000,\n            \"rmsVoltage\": 246.803,\n            \"rmsCurrent\": 22.524,\n            \"pwrFactor\": -0.96,\n            \"freqHz\": 60.00\n        },\n        \"lines\": [\n            {\n                \"currW\": -1275.379,\n                \"actPower\": -1275.379,\n                \"apprntPwr\": 1330.887,\n                \"reactPwr\": -284.765,\n                \"whDlvdCum\": 0.000,\n                \"whRcvdCum\": 0.000,\n                \"varhLagCum\": 0.000,\n                \"varhLeadCum\": 0.000,\n                \"vahCum\": 0.000,\n                \"rmsVoltage\": 123.202,\n                \"rmsCurrent\": 10.842,\n                \"pwrFactor\": -0.96,\n                \"freqHz\": 60.00\n            },\n            {\n                \"currW\": -1383.415,\n                \"actPower\": -1383.415,\n                \"apprntPwr\": 1443.318,\n                \"reactPwr\": 371.694,\n                \"whDlvdCum\": 0.000,\n                \"whRcvdCum\": 0.000,\n                \"varhLagCum\": 0.000,\n                \"varhLeadCum\": 0.000,\n                \"vahCum\": 0.000,\n                \"rmsVoltage\": 123.601,\n                \"rmsCurrent\": 11.681,\n                \"pwrFactor\": -0.95,\n                \"freqHz\": 60.00\n            }\n        ]\n    },\n    {\n        \"createdAt\": 1703362372,\n        \"reportType\": \"total-consumption\",\n        \"cumulative\": {\n            \"currW\": 1925.039,\n            \"actPower\": 1925.039,\n            \"apprntPwr\": 3666.355,\n            \"reactPwr\": 83.464,\n            \"whDlvdCum\": 0.000,\n            \"whRcvdCum\": 0.000,\n            \"varhLagCum\": 0.000,\n            \"varhLeadCum\": 0.000,\n            \"vahCum\": 0.000,\n            \"rmsVoltage\": 246.718,\n            \"rmsCurrent\": 14.861,\n            \"pwrFactor\": 0.53,\n            \"freqHz\": 60.00\n        },\n        \"lines\": [\n            {\n                \"currW\": 1013.656,\n                \"actPower\": 1013.656,\n                \"apprntPwr\": 966.393,\n                \"reactPwr\": -300.705,\n                \"whDlvdCum\": 0.000,\n                \"whRcvdCum\": 0.000,\n                \"varhLagCum\": 0.000,\n                \"varhLeadCum\": 0.000,\n                \"vahCum\": 0.000,\n                \"rmsVoltage\": 123.154,\n                \"rmsCurrent\": 7.847,\n                \"pwrFactor\": 1.00,\n                \"freqHz\": 60.00\n            },\n            {\n                \"currW\": 911.382,\n                \"actPower\": 911.382,\n                \"apprntPwr\": 866.613,\n                \"reactPwr\": 384.169,\n                \"whDlvdCum\": 0.000,\n                \"whRcvdCum\": 0.000,\n                \"varhLagCum\": 0.000,\n                \"varhLeadCum\": 0.000,\n                \"vahCum\": 0.000,\n                \"rmsVoltage\": 123.564,\n                \"rmsCurrent\": 7.013,\n                \"pwrFactor\": 1.00,\n                \"freqHz\": 60.00\n            }\n        ]\n    }\n]",
"Endpoint-production_json": "{\"production\":[{\"type\":\"inverters\",\"activeCount\":19,\"readingTime\":1703362348,\"wNow\":4645,\"whLifetime\":368953},{\"type\":\"eim\",\"activeCount\":0,\"measurementType\":\"production\",\"readingTime\":1703362374,\"wNow\":4581.794,\"whLifetime\":0.0,\"varhLeadLifetime\":0.0,\"varhLagLifetime\":0.0,\"vahLifetime\":0.0,\"rmsCurrent\":37.374,\"rmsVoltage\":246.616,\"reactPwr\":3.1,\"apprntPwr\":4608.853,\"pwrFactor\":0.99,\"whToday\":0.0,\"whLastSevenDays\":0.0,\"vahToday\":0.0,\"varhLeadToday\":0.0,\"varhLagToday\":0.0,\"lines\":[{\"wNow\":2287.442,\"whLifetime\":0.0,\"varhLeadLifetime\":0.0,\"varhLagLifetime\":0.0,\"vahLifetime\":0.0,\"rmsCurrent\":18.686,\"rmsVoltage\":123.091,\"reactPwr\":16.057,\"apprntPwr\":2300.151,\"pwrFactor\":0.99,\"whToday\":0.0,\"whLastSevenDays\":0.0,\"vahToday\":0.0,\"varhLeadToday\":0.0,\"varhLagToday\":0.0},{\"wNow\":2294.352,\"whLifetime\":0.0,\"varhLeadLifetime\":0.0,\"varhLagLifetime\":0.0,\"vahLifetime\":0.0,\"rmsCurrent\":18.689,\"rmsVoltage\":123.525,\"reactPwr\":-12.957,\"apprntPwr\":2308.702,\"pwrFactor\":0.99,\"whToday\":0.0,\"whLastSevenDays\":0.0,\"vahToday\":0.0,\"varhLeadToday\":0.0,\"varhLagToday\":0.0}]}],\"consumption\":[{\"type\":\"eim\",\"activeCount\":0,\"measurementType\":\"total-consumption\",\"readingTime\":1703362374,\"wNow\":1913.513,\"whLifetime\":0.0,\"varhLeadLifetime\":0.0,\"varhLagLifetime\":0.0,\"vahLifetime\":0.0,\"rmsCurrent\":14.836,\"rmsVoltage\":246.67,\"reactPwr\":87.114,\"apprntPwr\":3659.654,\"pwrFactor\":0.52,\"whToday\":0.0,\"whLastSevenDays\":0.0,\"vahToday\":0.0,\"varhLeadToday\":0.0,\"varhLagToday\":0.0,\"lines\":[{\"wNow\":998.728,\"whLifetime\":0.0,\"varhLeadLifetime\":0.0,\"varhLagLifetime\":0.0,\"vahLifetime\":0.0,\"rmsCurrent\":7.77,\"rmsVoltage\":123.122,\"reactPwr\":-297.225,\"apprntPwr\":956.71,\"pwrFactor\":1.0,\"whToday\":0.0,\"whLastSevenDays\":0.0,\"vahToday\":0.0,\"varhLeadToday\":0.0,\"varhLagToday\":0.0},{\"wNow\":914.785,\"whLifetime\":0.0,\"varhLeadLifetime\":0.0,\"varhLagLifetime\":0.0,\"vahLifetime\":0.0,\"rmsCurrent\":7.066,\"rmsVoltage\":123.548,\"reactPwr\":384.338,\"apprntPwr\":872.967,\"pwrFactor\":1.0,\"whToday\":0.0,\"whLastSevenDays\":0.0,\"vahToday\":0.0,\"varhLeadToday\":0.0,\"varhLagToday\":0.0}]},{\"type\":\"eim\",\"activeCount\":0,\"measurementType\":\"net-consumption\",\"readingTime\":1703362374,\"wNow\":-2668.281,\"whLifetime\":0.0,\"varhLeadLifetime\":0.0,\"varhLagLifetime\":0.0,\"vahLifetime\":0.0,\"rmsCurrent\":22.538,\"rmsVoltage\":246.724,\"reactPwr\":90.214,\"apprntPwr\":2780.568,\"pwrFactor\":-0.96,\"whToday\":0,\"whLastSevenDays\":0,\"vahToday\":0,\"varhLeadToday\":0,\"varhLagToday\":0,\"lines\":[{\"wNow\":-1288.714,\"whLifetime\":0.0,\"varhLeadLifetime\":0.0,\"varhLagLifetime\":0.0,\"vahLifetime\":0.0,\"rmsCurrent\":10.915,\"rmsVoltage\":123.152,\"reactPwr\":-281.168,\"apprntPwr\":1341.823,\"pwrFactor\":-0.96,\"whToday\":0,\"whLastSevenDays\":0,\"vahToday\":0,\"varhLeadToday\":0,\"varhLagToday\":0},{\"wNow\":-1379.567,\"whLifetime\":0.0,\"varhLeadLifetime\":0.0,\"varhLagLifetime\":0.0,\"vahLifetime\":0.0,\"rmsCurrent\":11.623,\"rmsVoltage\":123.572,\"reactPwr\":371.382,\"apprntPwr\":1438.745,\"pwrFactor\":-0.96,\"whToday\":0,\"whLastSevenDays\":0,\"vahToday\":0,\"varhLeadToday\":0,\"varhLagToday\":0}]}],\"storage\":[{\"type\":\"acb\",\"activeCount\":0,\"readingTime\":0,\"wNow\":0,\"whNow\":0,\"state\":\"idle\"}]}",
catsmanac commented 9 months ago

Looking at the data one can note that the Envoy reports it's CT as disabled. That is the reason the values don't show.

Power values are in the report (production: wNow ": 4581.794, net-consumtpion: " wNow ": -2668.281 and total-consumption: " wNow ": 1913.513). The Energy however is not integrated from the power, all Wh numbers are 0.

State: disabled in endpoint-meters and activeCount: 0 in Endpoint-production_json both signal CT's are not configured in the Envoy. Even though they are installed and seem to pickup the current (if correct), they are not enabled in the envoy setup and this version (and several versions before) of the integration does not trust the values for that reason.

You may want to check with your installer why the CT's are installed but not enabled in the Envoy.

endpoint meters has state: disabled:

"Endpoint-meters": [{
        "eid": 704643328,
        "state": "disabled",
        "measurementType": "production",
        "phaseMode": "split",
        "phaseCount": 2,
        "meteringStatus": "normal",
        "statusFlags": []
    }, {
        "eid": 704643584,
        "state": "disabled",
        "measurementType": "net-consumption",
        "phaseMode": "split",
        "phaseCount": 2,
        "meteringStatus": "normal",
        "statusFlags": []
    }
]

and the production report reports activeCount 0 for production, total-consumption and net-consumption 

"Endpoint-production_json": {
    " production ": [{
            " type ": " inverters ",
            " activeCount ": 19,
            " readingTime ": 1703362348,
            " wNow ": 4645,
            " whLifetime ": 368953
        }, {
            " type ": " eim ",
            " activeCount ": 0,
            " measurementType ": " production ",
            " readingTime ": 1703362374,
            " wNow ": 4581.794,
            " whLifetime ": 0.0,

    " consumption ": [{
            " type ": " eim ",
            " activeCount ": 0,
            " measurementType ": " total - consumption ",
            " readingTime ": 1703362374,
            " wNow ": 1913.513,
            " whLifetime ": 0.0,

       }, {
            " type ": " eim ",
            " activeCount ": 0,
            " measurementType ": " net - consumption ",
            " readingTime ": 1703362374,
            " wNow ": -2668.281,
            " whLifetime ": 0.0,

The full data you shared formatted in better readable format: split_phase_endpoints.json.txt

Indemnity83 commented 9 months ago

So, this is not really a question directed to you but how are there recent readings in production, total-consumption and net-consumption when the CTs are disabled.

I'll have to reach out to the installer on the CT's being disabled, but I suspect they've been that way the whole time and I'm guessing the older version of the HA plugin I was running before upgrading to 0.0.19 didn't check that and just took the wNow numbers which is why it was working.

Would you consider a PR that if the CT enabled check fails, then a check that the readingTime for production is within the last few minutes to also enable the consumption values?

Indemnity83 commented 9 months ago

Or, having looked at the way the checks are handled, a config option to skip checks (partial, untested code as example):

in conflig_flow.py

vol.Optional(
    "enable_all_features",
    default=self.config_entry.options.get(
        "enable_all_features", False
    ),
): bool,

in envoy_reader.py

# Assuming the option is passed into the constructor

self.isProductionMeteringEnabled = has_production_metering_setup(
    self.endpoint_meters_json_results.json()
 ) or self.enable_all_features
self.isConsumptionMeteringEnabled = has_consumption_metering_setup(
    self.endpoint_meters_json_results.json()
) or self.enable_all_features

Would need to have a warning on the config page that this could result in NaN and similar errors if the data is not actually available.

catsmanac commented 9 months ago

I can look into it, but that'll be after Christmas. Be aware that power can vary by sub-second while collecting data at 1 minute intervals only. When CT's are enabled you'll get the integration of power into energy which provides additional information on what's going on and allows you to configure the HA energy dashboard.

Looks like you figured out what could work, so changing the code on your system will allow you to test.

as for

So, this is not really a question directed to you but how are there recent readings in production, total-consumption and net-consumption when the CTs are disabled.

It seems that the envoy (in this firmware version at least) is reading the power values, calculation total-consumption from production and net-consumption and reporting them. But the Integration into Energy is not taking place. This could very well be the case for more firmware versions, but never a dull moment with Enphase, it's not always clear what is a feature or a bug. Hence the current implementation not trusting number if not enabled.

catsmanac commented 9 months ago

Not sure you got it going. I gave it a try and it would return grid exchange data from the consumption ct

As discussed, don't know if the numbers reported are correct or not. It would be an option to check after first configuration and then reload the integration.

If you have something running that's fine, than I won't take this any further. It's working around a non-standard configuration which isn't my preference.

Example configuration: afbeelding