Broken on Envoy firmware v7

[ZimbiX]

A few days ago, my Envoy's firmware was updated (without me doing anything) and it broke Zest and my telemetry collecting 😩

From https://envoy-ip/home#overview, the 'Software Version' is now: D7.6.175 (f79c8d) - previously it was: D5.0.55 (4f2662).

The Envoy requires HTTPS now, and the telemetry endpoint requires authentication. I have a branch that fixes the telemetry collection at least by addressing those: for-envoy-firmware-v7.

I haven't worked out how to fix Zest yet. I can no longer access the installer part of the Envoy's web server 😢 (needed to set the grid profile in order to enable/disable grid export).

Edit: My Envoy must have been upgraded sometime between 2023-07-07 08:07:34 (last successful grid profile change) and 2023-07-07 08:25:02 (first 401 Authorization Required error).

[brodykenrick]

First off thanks for this great tool/demonstration @ZimbiX .

To help get you back working again you can ask (demand) a DIY installer login from Enphase support.

It took me a couple of weeks and a few escalations to get them to decide between a downgrade, a DIY installer login or compensation, replacement or refund (under Australian consumer law them taking away this PEL functionality, which was a key determinant in me buying Enphase, would need to be redressed).

In the end I got the DIY permissions added to my login and can now access the installer sub pages on my local envoy (after creating the cloud token).

[brodykenrick]

It might be interesting to go down the path of adding the grid profile change into something like this in home assistant BTW.

https://github.com/vincentwolsink/home_assistant_enphase_envoy_installer

[brodykenrick]

I made some rather hacky changes to get this working with a DIY/installer login. https://github.com/brodykenrick/amber-enphase-zero-export-switcher-tool/tree/for-envoy-firmware-v7.

The main issue is that a DIY/installer token only has limited (12 hour) validity (as opposed to the old 12 months).

It is a bit messy as that is the first time I've ever touched ruby so it is no doubt horrible to a purist.

The enphase folks seem to really have made a mess of their tokens and what is allowed under what kind of token and when/what session might be needed also. I started by following the official docs and then veered off to scraping the Entrez website.

I haven't made it as a PR as it does not yet re-refresh the token (so it would need to be restarted each day until a timer or a retry-on-failure is implemented).

I mainly did these changes to expereiment and understand the tokens/bearers as I will put it in the the home assistant component longer term (which is the functinality I am keen on).

[ZimbiX]

Thanks @brodykenrick! Sorry for my delayed reply - I was pretty frustrated with Enphase support. I've just emailed them again, including:

Can you please escalate this issue, and offer me one of these options:

• Advise another way that I can enable/disable grid export (this is a very sensible feature to have, honestly)
• Grant me installer-level access to my system (I've seen that another customer who uses my app was granted this)
• Downgrade the Envoy to firmware version 5 and prevent automatic upgrades (less preferred option - in answer to your question, here is someone who was granted this)

I also wish to add my voice to the call for unexpiring auth tokens. I believe this would be necessary to be able to keep using my system if, god forbid, Enphase goes out of business, or even during a period of downtime.

I'd already given them these options (without the parenthesised parts), but hopefully an escalation you suggested helps.

It might be interesting to go down the path of adding the grid profile change into something like this in home assistant BTW.

Yeah, probably a good idea. I've been meaning to dive into Home Assistant, but I'm yet to do so.

I made some rather hacky changes to get this working with a DIY/installer login. https://github.com/brodykenrick/amber-enphase-zero-export-switcher-tool/tree/for-envoy-firmware-v7.

The main issue is that a DIY/installer token only has limited (12 hour) validity (as opposed to the old 12 months).

It is a bit messy as that is the first time I've ever touched ruby so it is no doubt horrible to a purist.

Wow, well done for giving Ruby a go and getting it working. That is definitely a big help.

The enphase folks seem to really have made a mess of their tokens and what is allowed under what kind of token and when/what session might be needed also. I started by following the official docs and then veered off to scraping the Entrez website.

Seems like it - fair enough.

I haven't made it as a PR as it does not yet re-refresh the token (so it would need to be restarted each day until a timer or a retry-on-failure is implemented).

No worries. I can implement that once I'm granted installer access so I can test.

I mainly did these changes to expereiment and understand the tokens/bearers as I will put it in the the home assistant component longer term (which is the functinality I am keen on).

Go for it! That'd be great.

[Delitants]

Hi, I need to get into installers page too to change to ZeroExport, but it says 401 auth required and no window to enter the credentials or anything. Any solution?

[Delitants]

I haven't worked out how to fix Zest yet. I can no longer access the installer part of the Envoy's web server 😢 (needed to set the grid profile in order to enable/disable grid export).

Did you find a way to bypass 401 error?

[brodykenrick]

Hi @ZimbiX ,

Yes, support was very frustrating. I just reviewed my comms with Enphase and in the end they handed me over to the API team with a separate email address. I just forwarded the whole support thread to the API team and they sorted me out quickly.

From support:

If still any issue persist as instructed by our superiors please send an email with screenshots to our API team (api@enphaseenergy.com)

Then I sent this to the API team:

I can generate a token and can access the system. However, the features that I use (and that were a reason I purchased the Enphase system) to change the power export limit via the grid profile are no longer present. I put the CURL command in a previous email. Please see attached again. This is using the locally available controls for my system (using the installer login) and this API call. I believe the best options available to you are to downgrade me back to my old firmware or to provide an installer/DIY login (or other way to control the grid profile programmatically).

Then the API team responded with:

We cannot downgrade the Gateway software. Even if we get a new Gateway it will have the new software sooner or later. So, I have given you DIY installer access. Please note that the DIY installer token will be valid only for 12 hours, so you might need to generate a token whenever you want to make changes to the grid profile.

[brodykenrick]

Hi, I need to get into installers page too to change to ZeroExport, but it says 401 auth required and no window to enter the credentials or anything. Any solution?

On the newer v7 firmware you can only access those installer pages/endpoints if you have a token (created at entrez) and ALSO you need a new set of permissions on your account to be an installer (either a "real installer" or a "DIY installer").

To get that DIY installer permission you will need to talk (probably a lot) to enphase support and their API team to have that added to your account. See my experience and Brendan's above.

[Delitants]

Hi, I need to get into installers page too to change to ZeroExport, but it says 401 auth required and no window to enter the credentials or anything. Any solution?

On the newer v7 firmware you can only access those installer pages/endpoints if you have a token (created at entrez) and ALSO you need a new set of permissions on your account to be an installer (either a "real installer" or a "DIY installer").

To get that DIY installer permission you will need to talk (probably a lot) to enphase support and their API team to have that added to your account. See my experience and Brendan's above.

I have a token but support ditched me. Actual installer threatens with fees per request to change something. Otrageous.

[brodykenrick]

I have a token but support ditched me. Actual installer threatens with fees per request to change something. Otrageous.

Yes, it is not a great experience with support. You can get the DIY installer permission for free - after jumping through enough hoops.

[Delitants]

I have a token but support ditched me. Actual installer threatens with fees per request to change something. Otrageous.

Yes, it is not a great experience with support. You can get the DIY installer permission for free - after jumping through enough hoops.

Thank you. Initially they told me to upgrade to Manager for one time fee of $200 or $10 a month. Crazy. What is their general support email except for api? Looks like they replied to me from noreply@ and I can’t reply back.

[brodykenrick]

Thank you. Initially they told me to upgrade to Manager for one time fee of $200 or $10 a month. Crazy. What is their general support email except for api? Looks like they replied to me from noreply@ and I can’t reply back.

support@enphaseenergy.com - I just started the support request from a chat when logged in to the app.

I don't think the paid-for manager login would even give you access to the installer API though. I think it just gives you more details in the app. Get that DIY installer permission and then you can use this nifty code here (or use home assistant; the current plugin/customer component supports turning off production entirely and eventually I'll add in export limit).

[upsuper]

You may not need installer permission to change the grid profile. I upgraded to Enlighten Manager (which costs A$349 but for live), and got the ability to disable power production completely, change grid profile, and limit power production arbitrary.

This thread has some information on limiting power production: https://support.enphase.com/s/question/0D53m00009YROWYCA5/is-there-a-way-to-limit-the-solar-exports-from-iq7a-microinverters-the-wholesale-electricity-price-is-often-negative-during-the-day-in-australia-0

I haven't tried to integrate those functionalities programmatically, though.

[Delitants]

Thank you. Initially they told me to upgrade to Manager for one time fee of $200 or $10 a month. Crazy. What is their general support email except for api? Looks like they replied to me from noreply@ and I can’t reply back.

support@enphaseenergy.com - I just started the support request from a chat when logged in to the app.

I don't think the paid-for manager login would even give you access to the installer API though. I think it just gives you more details in the app. Get that DIY installer permission and then you can use this nifty code here (or use home assistant; the current plugin/customer component supports turning off production entirely and eventually I'll add in export limit).

Thanks. Still no replies from them. My primary goal is to access the hidden profiles to limit Export. This ability is available in both installer and upgraded manager, which I think is ridiculous to pay for.

[brodykenrick]

I haven't tried to integrate those functionalities programmatically, though.

You can try the code in my repo and see if that works for you -- I suspect that the Manager paid-for permissions might be the same as the DIY-installer permissions. That would be interesting to know.

[brodykenrick]

@ZimbiX - I have implemented the refresh now (committed to my repo). Those -10c/kWh prices across the weekend in NSW were enough to want to set and forget.

I had to learn about that ruby conditional assignment ||= operator to get it working. My work here was also no doubt rather hacky there and will do better with a professional re-implementing rather than a PR. Let me know if you want a PR though.

For anyone else, watching and waiting for Brendan to get his installer DIY permissions, then my repo will work just like the original version used to but supporting v7 firmware and a DIY installer login (documentation is also updated).

https://github.com/brodykenrick/amber-enphase-zero-export-switcher-tool/tree/for-envoy-firmware-v7

[Delitants]

DIY-installer permissions now given only when passed online training and certification at Enphase University website. It's not just request and get it.

[brodykenrick]

I more demanded and got it but didn't need the training. In Australia (I assume you are in Oz because you are looking at this Amber tool) it is against a consumer's rights to have something that is defective (or removed) in a product that was an important part of the decision to buy that product. In my case I bought enphase knowing it had the ability to do a power export limit (as next year or the year after Ausgrid and many others will be making changes to generally have export charges -- i.e. negative pricing -- and of course wholesale pricing exposure already does the same).

When they took that important functionality away with the firmware upgrade then there is a problem they needed to resolve -- I presented to them a few possible solutions downgrade, which they said was impossible, uninstall and refund -- nobody wants that or "some other solution" (like a DIY installer permission). They gave me DIY installer permission.

[Delitants]

I more demanded and got it but didn't need the training. In Australia (I assume you are in Oz because you are looking at this Amber tool) it is against a consumer's rights to have something that is defective (or removed) in a product that was an important part of the decision to buy that product. In my case I bought enphase knowing it had the ability to do a power export limit (as next year or the year after Ausgrid and many others will be making changes to generally have export charges -- i.e. negative pricing -- and of course wholesale pricing exposure already does the same).

When they took that important functionality away with the firmware upgrade then there is a problem they needed to resolve -- I presented to them a few possible solutions downgrade, which they said was impossible, uninstall and refund -- nobody wants that or "some other solution" (like a DIY installer permission). They gave me DIY installer permission.

Good for you. No, I'm in the US. I don't know what's Amber, all I need is to control the export limit regardless of local electric company or country, this is why I'm looking here.

[brodykenrick]

Good for you. No, I'm in the US. I don't know what's Amber, all I need is to control the export limit regardless of local electric company or country, this is why I'm looking here.

Well this tool here is good to show you what to do if you have a installer token (or v6 firmware).

I'd watch over here for home assistant also - https://github.com/vincentwolsink/home_assistant_enphase_envoy_installer - it has the ability to turn off production entirely at the moments (again requiring an installer token) and sometime this year I expect I'll find the time to add the set_profile functionality also.

[upsuper]

You can try the code in my repo and see if that works for you -- I suspect that the Manager paid-for permissions might be the same as the DIY-installer permissions. That would be interesting to know.

Unfortunately it doesn't seem to be the case. I tried to manually follow what the script is doing: creating a token then posting the selected grid profile to set_profile.json via cURL, then I got 405 Method Not Allowed after a long pause.

I would probably try simulating going through the Enphase Manager website instead. I can't quite use the grid profile yet because the consumption CTs are not (yet) installed. Also I would need a bit more complicated automation since I do have a battery but it can't absorb all the energy the panels generate.

[brodykenrick]

Yeah, probably a good idea. I've been meaning to dive into Home Assistant, but I'm yet to do so. @ZimbiX it might be time to make the leap to HA -- everything is now waiting for you over there.....

The getting and setting of grid profiles is available in the home assistant enphase installer plugin and works rather nicely. See: https://github.com/vincentwolsink/home_assistant_enphase_envoy_installer/tree/0.3.5-beta2

It will be in a non-beta release sometime soon.

Connecting up with the amber integration/custom component (https://www.home-assistant.io/integrations/amberelectric/) I've been able to setup my system to now curtail if negative prices go less than 2c (I'm happy to help the grid with some extra green power at my expense but exporting 15kW at -0.10 for a couple of hours every sunny day through summer is not something I am keen on).

[ZimbiX]

Enphase finally gave me DIY installer permission! :tada: I'll update Zest for v7 support soon, basing it on @brodykenrick's changes

[Delitants]

Enphase finally gave me DIY installer permission! 🎉 I'll update Zest for v7 support soon, basing it on @brodykenrick's changes

How? Did you have to pass a training?

[ZimbiX]

@Delitants No, thankfully, but it doesn't seem like a DIY installer has access to change very much anyway.

My last email I shared here went unresponded to, so I sent this three weeks later:

Please respond. I'm getting to the point of frustration with this that I will be advising friends against buying Enphase.

Then a week later they responded:

Hi Brendan,

Thanks for your email. From the options provided, what we can only do is make you a self-installer, which I have gone ahead to do. You should be able to access your system as a self-installer.

Kind regards, Matti || APAC CS Enphase Enegry

[Delitants]

@Delitants No, thankfully, but it doesn't seem like a DIY installer has access to change very much anyway.

My last email I shared here went unresponded to, so I sent this three weeks later:

Please respond. I'm getting to the point of frustration with this that I will be advising friends against buying Enphase.

Then a week later they responded:

Hi Brendan, Thanks for your email. From the options provided, what we can only do is make you a self-installer, which I have gone ahead to do. You should be able to access your system as a self-installer. Kind regards, Matti || APAC CS Enphase Enegry

Are you able to disable export? Like enable export sero profile?

[ZimbiX]

Are you able to disable export? Like enable export sero profile?

I tried in the LAN UI last night, but with no sun, it didn't look like I had the option; but now I can :tada:

[brodykenrick]

Yeah, propagation to the UI is strange/slow (especially when the microinverters aren't powered). However, the export limiting effect on the site is very close to instantaneous.

[upsuper]

@brodykenrick How long does it take to propagate the grid profile for you? I tried using the Enlighten Manager to switch the grid profile, and while it takes just minutes to propagate to the relays, it seems to be taking forever to propagate to any microinverter even when they are powered.

[brodykenrick]

If you are watching the UI it is indeed slow to update and see it propagate.

The effect on the output of the microinverters is quick. A minute or less.

If you have an instantaneous monitor (I watch my Powerwall site export power) you can see when it is . limiting power export.

[ZimbiX]

Resolved in cb4a2edcc4b92dfbed81d5462b8702c5b920ffea =)