search

home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
70k stars 29.08k forks source link

Can't add Enphase Envoy Integration #110219

Open bruisehilda opened 5 months ago

bruisehilda commented 5 months ago

The problem

Same issue as in #103700 but I couldn't add a comment to it. I try to add the official integration and it spins for a moment before I get a "Cannot connect: (500, 'Timeout connecting to Envoy')" message. There are never any log files that I can find, either, even when I adjust the log level in my configuration.yaml file. The cloud and the app on my phone have all the current information for my system.

What version of Home Assistant Core has the issue?

core-2024.2.1

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Enphase Envoy

Link to integration documentation on our website

https://www.home-assistant.io/integrations/enphase_envoy

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

home-assistant[bot] commented 5 months ago

Hey there @bdraco, @cgarwood, @dgomes, @joostlek, @catsmanac, mind taking a look at this issue as it has been labeled with an integration (enphase_envoy) you are listed as a code owner for? Thanks!

Code owner commands Code owners of `enphase_envoy` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign enphase_envoy` Removes the current integration label and assignees on the issue, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

(message by CodeOwnersMention)

enphase_envoy documentation enphase_envoy source (message by IssueLinks)

cgarwood commented 5 months ago

Can you access your Envoy by going to https://envoy.local in a browser? Do you kmnow if the Envoy is connected to your network via WiFi or Ethernet?

bruisehilda commented 5 months ago

I have always used the IP address rather than envoy.local. I get a page cannot be found with envoy local. It is connected via Wi-Fi.

On Sat, Feb 10, 2024, 1:38 PM Charles Garwood @.***> wrote:

Can you access your Envoy by going to https://envoy.local in a browser? Do you kmnow if the Envoy is connected to your network via WiFi or Ethernet?

— Reply to this email directly, view it on GitHub https://github.com/home-assistant/core/issues/110219#issuecomment-1937192285, or unsubscribe https://github.com/notifications/unsubscribe-auth/A2YINARHZQ532ACIQJGH2YLYS7SFJAVCNFSM6AAAAABDC7XPPSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZXGE4TEMRYGU . You are receiving this because you authored the thread.Message ID: @.***>

catsmanac commented 5 months ago

To get a bit more info in the logfile you can extend the logging entry in configuration.yaml to:

logger:
  default: warn
  logs:
    homeassistant.components.enphase_envoy: debug
    httpx: debug
    httpcore: debug

When you added the official integration was it auto detected, or did you add it manually and specified the ip address, or did you run a custom one before and just removed that one?

Can you try opening https\://ip.address/info and/or http\://ip.adrres/info in your browser?

As for your reference to #103700, that was solved by switching to wired Ethernet. Although one would expect WIFI to work, it frequently pops-up with issues. The error is a timeout when connecting to the Envoy to read the info endpoint. If it spins just briefly then is has connection problems and times-out after 10 sec, tries once more for 10 sec and stops trying. If it spins for up to 60-120 seconds it times out on receiving the information from the Envoy after it could connect.

bruisehilda commented 5 months ago

I have tried it both ways, through autodetect and manual addition. I have never had a successful installation since I started trying last summer. At first I thought it was because of the firmware update that caused issues, so I waited until the September update and still had no luck. I have only had intermittent time available to try and chase it down until now.

There was no information in the log files related to the integration when I added the logger configuration requested, rebooted, and tried to re-add the integration.

When I go to ip.address/info, I get xml code that doesn't have style info associated with it, but it shows the document tree.

At this time I am unable to switch over to ethernet on my gateway. Everything is on the same subnet, and other wireless devices on that subnet connect to HASS without issue. No amount of waiting has yielded results.

On Sun, Feb 11, 2024 at 1:13 AM Arie Catsman @.***> wrote:

To get a bit more info in the logfile you can extend the logging entry in configuration.yaml to:

logger: default: warn logs: homeassistant.components.enphase_envoy: debug httpx: debug httpcore: debug

When you added the official integration was it auto detected, or did you add it manually and specified the ip address, or did you run a custom one before and just removed that one?

Can you try opening https://ip.address/info and/or http://ip.adrres/info in your browser?

As for your reference to #103700 https://github.com/home-assistant/core/issues/103700, that was solved by switching to wired Ethernet. Although one would expect WIFI to work, it frequently pops-up with issues. The error is a timeout when connecting to the Envoy to read the info endpoint. If it spins just briefly then is has connection problems and times-out after 10 sec, tries once more for 10 sec and stops trying. If it spins for up to 60-120 seconds it times out on receiving the information from the Envoy after it could connect.

— Reply to this email directly, view it on GitHub https://github.com/home-assistant/core/issues/110219#issuecomment-1937485596, or unsubscribe https://github.com/notifications/unsubscribe-auth/A2YINAVWTPXZEFSOROVVCWLYTCDTXAVCNFSM6AAAAABDC7XPPSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZXGQ4DKNJZGY . You are receiving this because you authored the thread.Message ID: @.***>

--

catsmanac commented 5 months ago

Getting the xml code in a tree without style info is the expected outcome. So the Envoy is responding to this request which is also used during the connection attempts.

As HA can not connect to the Envoy there's something in the network configuration that causes the issue. Multiple interface on HA and routing on the wrong one, firewall, ? Can you ping the Envoy from the HA?

bruisehilda commented 4 months ago

No it doesn't ping from HA, but HA pings from the Envoy. I've tried with no firewall rules to have it match every other WiFi device that works with HA, and I've tried it with firewall rules that are explicitly allowing all traffic between the two.

catsmanac commented 4 months ago

What platform is your HA running on? (Docker, linux, HA-OS,....)

Sounds as if the HA doesn't known where to find the Envoy, wrong interface, route, whatever.

Does traceroute envoy-ip-address result in some info?

bruisehilda commented 4 months ago

HA-OS on proxmox. No results for traceroute after the initial entry. Only thing I can think of is that the Envoy is wireless and not wired, but it's attached to the same wifi as all the other IoT devices on my network that HA can see without issue. I'm not sure I can get a cable into the box, based on where it's at, too.

On Mon, Feb 26, 2024 at 7:51 AM Arie Catsman @.***> wrote:

What platform is your HA running on? (Docker, linux, HA-OS,....)

Sounds as if the HA doesn't known where to find the Envoy, wrong interface, route, whatever.

Does traceroute envoy-ip-address result in some info?

— Reply to this email directly, view it on GitHub https://github.com/home-assistant/core/issues/110219#issuecomment-1964473238, or unsubscribe https://github.com/notifications/unsubscribe-auth/A2YINAW6RT3ZYMT62TYQXZ3YVSVRFAVCNFSM6AAAAABDC7XPPSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRUGQ3TGMRTHA . You are receiving this because you authored the thread.Message ID: @.***>

-- Lisa Campbell (She/Her)

catsmanac commented 4 months ago

It 'should' work as well on WiFi, but reality is often stubborn. Are the other devices also pulled or are they pushing to the HA? It seems that HA requesting is failing, at least for the Envoy. Is there multiple interfaces on the proxmox that may lead to confusion? I'm not into HA-OS, found this one: https://github.com/home-assistant/core/issues/66969, not exactly the same but it may trigger some ideas to try.

ShooterQ commented 4 months ago

I'm having the same issue with an enphase envoy, connected via Wi-Fi (I really wish we could run a cable out there but it's not easy to do right now) and the issue began when I updated to Core 2024.3.0. I can access the device's webpage via IP in the browser without issue though, but not using the "https://envoy.local/" address.

ETA: I also tried deleting my entire integration and reconnecting from scratch. This successfully pulls "something" from the IP because it automatically names itself, but then gets caught in the "Failed Setup" loop.

catsmanac commented 4 months ago

The name envoy.local is not actually used, just a convience, ah well if it works..... The ip it finds comes from autodiscovery, the Envoy tells it's ip and HA reacts by showing it in the found devices. But then it stalls in trying to connect. Sounds indeed the sameas what @bruisehilda reported, almost one-way traffic.

If you haven't tried debug yet can you add below to configuration.yaml and restart HA to see if it shows anything surfaces.

logger:
  default: warning
  logs:
    homeassistant.components.enphase_envoy: debug
    pyenphase: debug
    httpx: debug
    httpcore: debug
    homeassistant.components.zeroconf: debug
ShooterQ commented 3 months ago

I finally got around to checking the logs after adding the above to my config. I then re-enabled the Enphase integration, watched it fail, and then collecting the following form the logs:

This error originated from a custom integration.

Logger: custom_components.enphase_envoy Source: helpers/update_coordinator.py:318 integration: Enphase Envoy (DEV) (documentation) First occurred: 9:12:44 PM (1 occurrences) Last logged: 9:12:44 PM

Unexpected error fetching envoy Envoy 121629059124 data: 'NoneType' object has no attribute 'json' Traceback (most recent call last): File "/config/custom_components/enphase_envoy/init.py", line 66, in async_update_data await envoy_reader.getData() File "/config/custom_components/enphase_envoy/envoy_reader.py", line 625, in getData response.raise_for_status() File "/usr/local/lib/python3.12/site-packages/httpx/_models.py", line 761, in raise_for_status raise HTTPStatusError(message, request=request, response=self) httpx.HTTPStatusError: Client error '401 Unauthorized' for url 'http://10.0.0.36/api/v1/production/inverters' For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/401

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/config/custom_components/enphase_envoy/init.py", line 128, in async_setup_entry await coordinator.async_config_entry_first_refresh() File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 283, in async_config_entry_first_refresh await self._async_refresh( File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 318, in _async_refresh self.data = await self._async_update_data() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 274, in _async_update_data return await self.update_method() ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/enphase_envoy/init.py", line 68, in async_update_data raise ConfigEntryAuthFailed from err homeassistant.exceptions.ConfigEntryAuthFailed: Client error '401 Unauthorized' for url 'http://10.0.0.36/api/v1/production/inverters' For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/401

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 318, in _async_refresh self.data = await self._async_update_data() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 274, in _async_update_data return await self.update_method() ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/enphase_envoy/init.py", line 88, in async_update_data data[description.key] = await getattr( ^^^^^^^^^^^^^^ File "/config/custom_components/enphase_envoy/envoy_reader.py", line 826, in production raw_json = self.endpoint_meters_reports_json_results.json() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'json'

Honestly out of my depth when trying to interpret the issue. Does the failure to auth mean I should try to refresh my credentials?

catsmanac commented 3 months ago

Hi @ShooterQ, the log file shows you are running the Enphase Envoy (DEV) custom integration. Issues for that integration should be reported at https://github.com/briancmpbll/home_assistant_custom_envoy/issues.

If you intended to use the Home Assistant Core Enphase Envoy integration you should remove the custom integration and try again.

ShooterQ commented 3 months ago

Thank you, I knew I messed something up. I remember thinking to myself, "I'm going to try and use this custom integration if I can't get mine working" but forgot I went further than that. I'll get on that for the core integration.

catsmanac commented 2 months ago

Any luck with this @bruisehilda?

bruisehilda commented 2 months ago

Not yet. I haven't been able to run a cable, either. I ran wireshark but didn't see anything of interest. I reached out to Enphase but haven't heard back yet. Still same behavior.

On Fri, May 3, 2024 at 12:18 AM Arie Catsman @.***> wrote:

Any luck with this @bruisehilda https://github.com/bruisehilda?

— Reply to this email directly, view it on GitHub https://github.com/home-assistant/core/issues/110219#issuecomment-2092446454, or unsubscribe https://github.com/notifications/unsubscribe-auth/A2YINASKWAHC45ITTVZ6MFTZAM25BAVCNFSM6AAAAABDC7XPPSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJSGQ2DMNBVGQ . You are receiving this because you were mentioned.Message ID: @.***>

-- Lisa Campbell (She/Her) (509)845-4938 cell

catsmanac commented 2 months ago

Sorry to hear the trouble is still there, must be frustrating.

In the mean time we've changed logic around timeout. I'm not saying this will solve the issue and trigger for the change was something different. It's more like a far shot in the hope it may yield another tidbit of information that may help. I fully understand if you would pass on it, and I certainly don't want to build false hope for a quick solution.

If and when you feel like it:

You've done this all before I guess, so as said no issue if you want to pass on this and no hurry to get this done either.

bruisehilda commented 2 months ago

No luck yet. I can still ping HASS from the Envoy, but not vice versa. However the Envoy pings from my management network. I have firewall rules in place to allow that for that VLAN, but HASS and the Envoy are on the same VLAN, and I can ping other wireless devices on that VLAN from HASS. Just super weird. I've reached out to Enphase as well, but they haven't been helpful yet.

catsmanac commented 3 weeks ago

Hi @bruisehilda, I'm ok to keep this open for now, but don't think this is an Enphase Envoy integration issue. Sounds more like a network related issue. Maybe try with a docker container on another platform?

bruisehilda commented 3 weeks ago

Yeah I think you're right. I reached out to Enphase directly and beyond the "we're going to look into this" email, I haven't heard back, even when asked for updates.

On Sat, Jun 22, 2024 at 2:33 AM Arie Catsman @.***> wrote:

Hi @bruisehilda https://github.com/bruisehilda, I'm ok to keep this open for now, but don't think this is an Enphase Envoy integration issue. Sounds more like a network related issue. Maybe try with a docker container on another platform?

— Reply to this email directly, view it on GitHub https://github.com/home-assistant/core/issues/110219#issuecomment-2183960162, or unsubscribe https://github.com/notifications/unsubscribe-auth/A2YINAVN2FZP2UOPLV3PL2TZIVAGTAVCNFSM6AAAAABDC7XPPSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBTHE3DAMJWGI . You are receiving this because you were mentioned.Message ID: @.***>

-- Lisa Campbell (She/Her) (509)845-4938 cell

catsmanac commented 3 weeks ago

@bruisehilda , was going over all information again to find some leads, very few and what I can think off are really far shots...

"we're going to look into this" email

A response heard too often unfortunately.

bruisehilda commented 1 week ago

Support wrote back asking me to follow their "tech brief" that shows someone how to access the web ui and local api via python or curl. I didn't try python, but I can access the web ui and pull down info with curl,

They're on the same VLAN and literally nothing else on WiFi has a problem WiFi. Unless it's an issue that I'm accessing HAOS with HTTP and Envoy is HTTPS, but I'm authenticating through the platform, so I wouldn't think that should impact it, but who knows, since the only error I'm seeing is "Cannot connect: (500, 'Timeout connecting to Envoy')". I'll look into it, but I know I've tried enabling rules that specifically allow port 80 and 443 traffic between the two hosts in the past, with no luck.

I also was able to test another computer and can ping the Envoy from the same VLAN via wireless. I haven't plugged my computer in and tested if it still pings on the same VLAN via Ethernet yet, so I still have that on my list.

I just made sure everything was up to date and rebooted with the logging configurations. The integration recognizes the system and suggests I add it automatically, and I tried that before exporting. Here is the output: home-assistant_2024-07-07T23-42-58.437Z.log

I didn't see much that indicated what the problem is, but maybe you'll have a different perspective.

catsmanac commented 1 week ago

This remains a mystery for me as well I'm afraid. Like you noted, the Envoy is auto-discovered through the zeroconf method. But as soon as you added the configuration details it tries to connect to the envoy to read the /info page. First on https and when that fails on http, which also fails. When it repeats those step both fail again.

Both attempts to https or http fails with connection timeout after 10 seconds, which is as configured. If the Envoy is slow to build a connection and needs more time then this could be a lead. I guess you would note that slowness when trying the other connections?

The connection is going somewhere but is never accepted by where it ended up. If the Envoy / 192.168.50.49 would not be running you would see 500, Unable to connect to Envoy rather the 500, Timeout connecting to Envoy.

At the bottom I've included envoy_info.py.txt in case you want to test outside of HA in the HAOS console. Remove the .txt extension, copy it to the HA config folder. Then in the HAOS console shell run it from the config folder specifying the Envoy IP. Use the optional -d for debug output.

python /config/envoy_info.py 192.168.3.104  [-d]
Envoy: 192.168.3.104 fw: 7.6.175 sn: 123456789012

Only other item that I noted is that the envoy reported itself in zeroconf as envoy-2._enphase-envoy._tcp.local using the name envoy-2. That name is typically used by a second envoy on the network, though it should not be an issue as the specified IP 192.168.50.49 is used. Could be left-over from the past or installation.

Did you ran the curl from the HAOS console? If so, did GET http://192.168.50.49/info work?

Envoy log lines extract from your log file:

2024-07-07 16:36:26.750 DEBUG (MainThread) [homeassistant.components.zeroconf] Starting Zeroconf browser for: ['_Volumio._tcp.local.', ..., '_enphase-envoy._tcp.local.', ... , '_home-assistant._tcp.local.']
2024-07-07 16:36:26.940 DEBUG (MainThread) [homeassistant.components.zeroconf] service_update: type=_enphase-envoy._tcp.local. name=envoy-2._enphase-envoy._tcp.local. state_change=ServiceStateChange.Added
2024-07-07 16:36:26.941 DEBUG (MainThread) [homeassistant.components.zeroconf] Discovered new device envoy-2._enphase-envoy._tcp.local. ZeroconfServiceInfo(ip_address=ZeroconfIPv4Address('192.168.50.49'), ip_addresses=[ZeroconfIPv4Address('192.168.50.49'), ZeroconfIPv6Address('fd6c:99da:d085:6ba4:def3:1cff:fe2e:a790')], port=80, hostname='envoy-2.local.', type='_enphase-envoy._tcp.local.', name='envoy-2._enphase-envoy._tcp.local.', properties={'txtvers': '1', 'protovers': '7.3.621', 'serialnum': '<sn>'})
2024-07-07 16:36:53.139 DEBUG (MainThread) [homeassistant.components.enphase_envoy.config_flow] Zeroconf ip 4 processing 192.168.50.49, current hosts: set()
2024-07-07 16:36:53.139 DEBUG (MainThread) [homeassistant.components.enphase_envoy.config_flow] Zeroconf ip 192.168.50.49, fw 7.3.621, no existing entry with serial <sn>
2024-07-07 16:36:53.139 DEBUG (MainThread) [homeassistant.components.enphase_envoy.config_flow] Zeroconf ip 192.168.50.49 to step user
2024-07-07 16:38:15.977 DEBUG (MainThread) [pyenphase.firmware] Requesting https://192.168.50.49/info with timeout Timeout(connect=10.0, read=45.0, write=10.0, pool=10.0)
2024-07-07 16:38:15.977 DEBUG (MainThread) [httpcore.connection] connect_tcp.started host='192.168.50.49' port=443 local_address=None timeout=10.0 socket_options=None
2024-07-07 16:38:25.978 DEBUG (MainThread) [httpcore.connection] connect_tcp.failed exception=ConnectTimeout(TimeoutError())
2024-07-07 16:38:25.978 DEBUG (MainThread) [pyenphase.firmware] Retrying to http://192.168.50.49/info with timeout Timeout(connect=10.0, read=45.0, write=10.0, pool=10.0)
2024-07-07 16:38:25.979 DEBUG (MainThread) [httpcore.connection] connect_tcp.started host='192.168.50.49' port=80 local_address=None timeout=10.0 socket_options=None
2024-07-07 16:38:35.981 DEBUG (MainThread) [httpcore.connection] connect_tcp.failed exception=ConnectTimeout(TimeoutError())
2024-07-07 16:40:05.537 DEBUG (MainThread) [pyenphase.firmware] Requesting https://192.168.50.49/info with timeout Timeout(connect=10.0, read=45.0, write=10.0, pool=10.0)
2024-07-07 16:40:05.537 DEBUG (MainThread) [httpcore.connection] connect_tcp.started host='192.168.50.49' port=443 local_address=None timeout=10.0 socket_options=None
2024-07-07 16:40:15.539 DEBUG (MainThread) [httpcore.connection] connect_tcp.failed exception=ConnectTimeout(TimeoutError())
2024-07-07 16:40:15.539 DEBUG (MainThread) [pyenphase.firmware] Retrying to http://192.168.50.49/info with timeout Timeout(connect=10.0, read=45.0, write=10.0, pool=10.0)
2024-07-07 16:40:15.539 DEBUG (MainThread) [httpcore.connection] connect_tcp.started host='192.168.50.49' port=80 local_address=None timeout=10.0 socket_options=None
2024-07-07 16:40:25.541 DEBUG (MainThread) [httpcore.connection] connect_tcp.failed exception=ConnectTimeout(TimeoutError())

envoy_info.py.txt