New Weatherflow integration - Config Flow Failed

ChirpyTurnip commented 7 months ago

The problem

Have just uninstalled my old Weatherflow integration (which was via HACS), then I removed the HACS integration, then I restarted HA.

Now I try to add the Weatherflow integration via Services - it spins for a little bit and then it says:

Config flow could not be loaded: Unknown error

What version of Home Assistant Core has the issue?

2023.10.0

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Weatherflow

Link to integration documentation on our website

https://www.home-assistant.io/integrations/weatherflow

Diagnostics information

Starts with:

Please wait, starting configuration wizard for WeatherFlow Weather

Then after a short delay ends with:

Config flow could not be loaded: Unknown error

The log only shows:

2023-10-05 10:06:55.044 INFO (SyncWorker_13) [homeassistant.loader] Loaded weatherflow from homeassistant.components.weatherflow

Example YAML snippet

N/A

Anything in the logs that might be useful for us?

Not that I can find....

Additional information

No response

home-assistant[bot] commented 7 months ago

Hey there @natekspencer, @jeeftor, mind taking a look at this issue as it has been labeled with an integration (weatherflow) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `weatherflow` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign weatherflow` Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

weatherflow documentation weatherflow source _{^{(message by IssueLinks)}}

travissluka commented 7 months ago

I have the same issue. Logs show the following when this happens

Logger: aiohttp.server
Source: runner.py:186
First occurred: 3:02:34 PM (3 occurrences)
Last logged: 7:19:44 PM

Error handling request

jkaberg commented 7 months ago

Might be related or not but I'm not seeing the new "official" integration, only the custom one with 2023.10.0

tealc commented 7 months ago

Might be related or not but I'm not seeing the new "official" integration, only the custom one with 2023.10.0

Same here!

travissluka commented 7 months ago

@tealc @jkaberg silly question, but are you sure you removed the old integration from HACs and rebooted? The new built-in integration won't show up until you do that

joostlek commented 7 months ago

To use the core integration, please remove weatherflow from HACS or your custom_components folder. This way you use the core integration. Keep in mind that the core integration is built off of the custom one, and that the core one doesn't support every feature the custom does.

jkaberg commented 7 months ago

So I deleted that from integrations page and then HACS, then from custom_components/, and restarted HASS. Then I went to setup the new integration Weatherflow and when trying that I get:

Something else that needs to be removed?

joostlek commented 7 months ago

Did you remove the old configuration?

jkaberg commented 7 months ago

Where can I find that? Nothing in configuration.yaml. I guess I used the config flow when setting up the custom integration, but unsure where config that follows that resides

joostlek commented 7 months ago

Don't you have it on your integrations page?

jkaberg commented 7 months ago

Don't you have it on your integrations page?

I do have the "official" one

So I removed the old custom integration's config from .storage/core.config_entries (shutdown HASS before doing this) - now it doesn't complain about the integration being setup but throws (Translated to "Configuration flow can't be loaded: unknown error"). No errors in the logs about this

joostlek commented 7 months ago

There should be errors in the logs about it tho

joostlek commented 7 months ago

So if there isn't we should take a look in the code why not

jkaberg commented 7 months ago

There should be errors in the logs about it tho

Sorry, got missed at an oversight. I get the error 2023-10-05 14:27:32.371 ERROR (MainThread) [aiohttp.server] Error handling request when it's trying to set up the integration. So same as the first 2 guys

ChirpyTurnip commented 7 months ago

So I'm the same - no trace of the other Weatherflow HACS integration and the Weatherflow integration in the list is the 'official' one:

Weatherflow

Same error as @jkaberg, and same issue with nothing obvious in the logs. :-)

Glad I'm not the only one anymore....one's first reaction is always "Crap! What did I do? What did I dooooooo?" :-)

benquan commented 7 months ago

First I removed the old integration, then removed from HACS, I then added the new integration, but it kept saying I was using a custom component. So I erased it once again. restated home assistant, but now when I try to add the new integration [WeatherFlow] I get the following error:

And does not allow me to input the station ID or the key.

jkaberg commented 7 months ago

@benquan Just guessing here but the new integration uses UDP to communicate with the local hub which in turn communicates with the weather station. So there should be no need to insert station ID or key, if any I'd imagine an IP adress and port (but the hub broadcasts data? - that would mean no config I suppose)

benquan commented 7 months ago

Thanks @jkaberg, understood. But either way it was showing me that message, and now the message has changed to:

jeeftor commented 7 months ago

What platform are you on? If UDP isnt open you wont see anything?

If you have netcat you can run the following:

 nc -u -l 50222

And it should print out a packet similar to this:

{"serial_number":"HB-00068684","type":"hub_status","firmware_revision":"177","uptime":1288997,"rssi":-63,"timestamp":1696606467,"reset_flags":"BOR,PIN,POR","seq":128736,"radio_stats":[25,1,0,3,1016],"mqtt_stats":[124,40]}

If your HA machine can't receive UDP or your network is funky you might have to stick with the cloud version for now

benquan commented 7 months ago

It's running on a virtualbox machine running Home Assistant OS. UDP should be open. I haven't firewalled it other than the default Home Assistant OS settings.

The nc command never responds. Just hangs.

edit: Additional information, the connection from my vitual machine to the network is a bridged connection.

XinoGitHub commented 7 months ago

It's running on a virtualbox machine running Home Assistant OS. UDP should be open. I haven't firewalled it other than the default Home Assistant OS settings.

The nc command never responds. Just hangs.

Same here ...

mcnutter1 commented 7 months ago

Same here, new update of HA 2023.10.1 broke the HACS version of weatherflow. The UDP version in this new build doesn't work at all, and the previous REST API one now no longer works... ALSO, FWIW, why in the world would this have switched to UDP! Some people have weatherflows in multiple locations / networks and UDP doesn't work in these scenarios.

natekspencer commented 7 months ago

Same here, new update of HA 2023.10.1 broke the HACS version of weatherflow. The UDP version in this new build doesn't work at all, and the previous REST API one now no longer works... ALSO, FWIW, why in the world would this have switched to UDP! Some people have weatherflows in multiple locations / networks and UDP doesn't work in these scenarios.

If the HACS version of WeatherFlow is properly installed/updated, HA 2023.10.1 should not break anything with it. If you removed the HACS version of WeatherFlow without deleting the existing configuration tied to it, then yes it will appear broken since they use different code and you will need to re-download the HACS version of WeatherFlow to get things working again. As an FYI, if you have a custom component installed that uses the same domain as a core integration, the custom component will take precedence. The UDP version does work, though only locally as you've stated. And in those scenarios, you should still use the custom HACS version. Nothing was switched to UDP, it's just that the UDP local-only version now has an official core integration.

mcnutter1 commented 7 months ago

Both the HACS version and the Core version use the same domain, the problem is the config_flow only loads the core UI version making it impossible to configure the HACS version.

Tried re-installing the HACS version and restarting and it still loads the Core config_flow.

the CORE version doesn’t find anything because my tempest hub is on a different network.

stuck here, probably like many others.

Should have ported the REST version into the core, and made the UDP local only version optional.

natekspencer commented 7 months ago

Both the HACS version and the Core version use the same domain, the problem is the config_flow only loads the core UI version making it impossible to configure the HACS version.

Which HACS repo are you using? If it has a config flow file and uses the same domain, it will load appropriately. I've got it working on a machine that way myself.

Should have ported the REST version into the core, and made the UDP local only version optional.

Everyone has there preferences on which should come first, but both can exist in core. It just happens to be that the developers that worked on adding to core started with the UDP version.

meanpandamedia commented 7 months ago

I’m sharing this in case it helps. I had the same issues trying to add the WeatherFlow integration (same error message, same entry in the log). As it turned out my Tempest and HA Yellow were on different VLAN. Once I moved the Tempest to the same VLAN as the HA Yellow, the integration worked as expected.

ChirpyTurnip commented 7 months ago

My HA and tempest are definitely in separate subnets and firewalled off from each other. I would expect that the config flow will allow me specify the target IP address off the base station.... Otherwise things get crazy hard as discovery won't work well in a network which isn't a simple flat structure.

On Sun, 8 Oct 2023, at 11:06, meanpandamedia wrote:

I’m sharing this in case it helps. I had the same issues trying to add the WeatherFlow integration (same error message, same entry in the log). As it turned out my Tempest and HA Yellow were on different VLAN. Once I moved the Tempest to the same VLAN as the HA Yellow, the integration worked as expected.

— Reply to this email directly, view it on GitHub https://github.com/home-assistant/core/issues/101423#issuecomment-1751828232, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUIP7ZT2VMCZHDXXKJMWI23X6HG7DAVCNFSM6AAAAAA5THTNXCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJRHAZDQMRTGI. You are receiving this because you authored the thread.Message ID: @.***>

travissluka commented 7 months ago

I’m sharing this in case it helps. I had the same issues trying to add the WeatherFlow integration (same error message, same entry in the log). As it turned out my Tempest and HA Yellow were on different VLAN. Once I moved the Tempest to the same VLAN as the HA Yellow, the integration worked as expected.

That makes sense, though the error was of course cryptic. I'm running on a docker container so hass is on a different subnet than the tempest. I'm not sure what an easy fix would be other than having the ability to manually specify the tempest IP as @ChirpyTurnip mentions

jkaberg commented 7 months ago

My HA and tempest are definitely in separate subnets and firewalled off from each other. I would expect that the config flow will allow me specify the target IP address off the base station.... Otherwise things get crazy hard as discovery won't work well in a network which isn't a simple flat structure. … On Sun, 8 Oct 2023, at 11:06, meanpandamedia wrote: I’m sharing this in case it helps. I had the same issues trying to add the WeatherFlow integration (same error message, same entry in the log). As it turned out my Tempest and HA Yellow were on different VLAN. Once I moved the Tempest to the same VLAN as the HA Yellow, the integration worked as expected. — Reply to this email directly, view it on GitHub <#101423 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUIP7ZT2VMCZHDXXKJMWI23X6HG7DAVCNFSM6AAAAAA5THTNXCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJRHAZDQMRTGI. You are receiving this because you authored the thread.Message ID: @.***>

This is the case for me aswell, my Tempest base station is located within an IoT vlan which firewalled tightly. Altho the vlan HASS sits in does have unrestricted access to the IoT vlan, and the router has helper services in place (igmpproxy, avhai brigde etc). HASS in my case sits in an docker with an macvlan interface which has it's own dedicated IP within said subnet.

Never had issues with UPNP/avahi/igmp etc before with this setup, so this would be the first. However seeing as the Tempest basestation does broadcasts to any IP? I can understand why this then fails. Communication from IoT vlan to "hass vlan" is for now not allowed at my place.

meanpandamedia commented 7 months ago

@ChirpyTurnip I agree that being able to specify an IP address during the config flow would be ideal. I also have my IoT devices on their own VLAN which is heavily locked down. I had tried allowing bi-directional communication from the Tempest to the HA Yellow and even tried allowing the Tempest to communicate with the entire VLAN that the Yellow is on and both didn’t work. The only way to get it discovered was to put it on the same VLAN as the Yellow. It’s not ideal but I can still lock down the Tempest.

ChirpyTurnip commented 7 months ago

It's just bad practice to move devices around the network in order to make them work. I have everything banged up in an IoT LAN and the old HACS-based integration worked fine with this set up. Sometimes I think the easiest solution is to also treat HA as hostile and condemn it to the IoT LAN as well....

Still, for any integration I don't think it is unreasonable to expect the ability to specify the target IP rather than relying on auto-discovery. Furthermore, using a target IP is also much quicker as there is no need to scan for the end devices or to sit there waiting for some broadcast traffic to appear.....

On Mon, 9 Oct 2023, at 04:15, meanpandamedia wrote:

@ChirpyTurnip https://github.com/ChirpyTurnip I agree that being able to specify an IP address during the config flow would be ideal. I also have my IoT devices on their own VLAN which is heavily locked down. I had tried allowing bi-directional communication from the Tempest to the HA Yellow and even tried allowing the Tempest to communicate with the entire VLAN that the Yellow is on and both didn’t work. The only way to get it discovered was to put it on the same VLAN as the Yellow. It’s not ideal but I can still lock down the Tempest.

— Reply to this email directly, view it on GitHub https://github.com/home-assistant/core/issues/101423#issuecomment-1752051977, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUIP7ZUTUALPUUZDEQX6USLX6K7RBAVCNFSM6AAAAAA5THTNXCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJSGA2TCOJXG4. You are receiving this because you were mentioned.Message ID: @.***>

benquan commented 7 months ago

As I mentioned above I am using a virtualbox with HAOS running on it. I tested the nc -u -l 50222 command and got no response inside the HAOS terminal. Doing the same on the host computer of the virtualbox (a macbook) I do get a response: {"serial_number":"ST-00089889","type":"rapid_wind","hub_sn":"HB-00103896","ob":[1696952452,0.63,349]}.

What caught my attention is that I do get a response inside HAOS using a netstat | grep 50222:

➜  ~ netstat | grep 50222
udp        0      0 5c53de3b-esphome.:38213 67f4b1af-weatherf:50222 ESTABLISHED

not sure if that helps anyone with more knowledge of the inner workings...

ChirpyTurnip commented 7 months ago

My failed tempest has been replaced and I'm back on the HACS version of the integration....it just works. Will try the native Weatherflow one again in the future once there is an update....

On Wed, 11 Oct 2023, at 04:44, Ben wrote:

As I mentioned above I am using a virtualbox with HAOS running on it. I tested the nc -u -l 50222 command and got no response inside the HAOS terminal. Doing the same on the host computer of the virtualbox (a macbook) I do get a response: {"serial_number":"ST-00089889","type":"rapid_wind","hub_sn":"HB-00103896","ob":[1696952452,0.63,349]}.

What caught my attention is that I do get a response inside HAOS using a netstat | grep 50222:

➜ ~ netstat | grep 50222 udp 0 0 5c53de3b-esphome.:38213 67f4b1af-weatherf:50222 ESTABLISHED not sure if that helps anyone with more knowledge of the inner workings...

— Reply to this email directly, view it on GitHub https://github.com/home-assistant/core/issues/101423#issuecomment-1755719507, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUIP7ZQCLAPLKLOTNIKI3KLX6VUONAVCNFSM6AAAAAA5THTNXCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJVG4YTSNJQG4. You are receiving this because you were mentioned.Message ID: @.***>

dhover commented 7 months ago

In the beginning I also had UDP errors because the WeatherFlow2MQTT AddOn was still running. After removing the AddOn the Weatherflow core integration is working fine.

ChirpyTurnip commented 7 months ago

Can't say I've ever come across the Weatherflow2MQTT add on so that's not likely to be the case for me, unless it was bundled in there somehow....

On Thu, 12 Oct 2023, at 07:44, dhover wrote:

In the beginning I also had UDP errors because the WeatherFlow2MQTT AddOn was still running. After removing the AddOn the Weatherflow core integration is working fine.

— Reply to this email directly, view it on GitHub https://github.com/home-assistant/core/issues/101423#issuecomment-1758292341, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUIP7ZVORIX4GTSRITN4AGLX63SH7ANCNFSM6AAAAAA5THTNXA. You are receiving this because you were mentioned.Message ID: @.***>

jeeftor commented 7 months ago

In the past we did have an option to specify an IP -> but i took it out since it was much simpler to get the integration into core with a simpler flow.

If you are on separate subents or vlans then UDP traffic obliviously isn't going to traverse easily. I'm not sure, however, how having a specific IP would help out.

This integration is UDP based - so all the IP is serving as is a host filter to the best of my knowledge. By default we're using a 0.0.0.0 ip filter. Adding an IP would just mean I only want to receive UDP traffic from this specific ip... it probably won't solve the subnet UDP issue...

If you're using the main HACS addon that's going out to cloud and is TCP based - this is the fully UDP based integration: https://weatherflow.github.io/Tempest/api/udp/v171/

jeeftor commented 7 months ago

Maybe I can make some updates to the docs...

jeeftor commented 7 months ago

Is this helpful to add to the docs?

ChirpyTurnip commented 7 months ago

Yes - useful. To make this work I'll need to set up a "UDP Broadcast Relay" package/addon on my firewall.....then I can snaffle traffic UDP traffic destined for UDP 50222 and forward it to the HA vLAN/Subnet.

Such addons are available for OPNsense, pfSense, and presumably other FWs...though many will not have this option.

It's a pain, but at the same time it opens the door to letting Sonos speakers (and other such devices) work across vLANs too......so not necessarily all bad.

On Fri, 13 Oct 2023, at 01:09, Jeef wrote:

image https://user-images.githubusercontent.com/6491743/274593686-60401368-4cc8-4ec1-b536-622e8b387c89.png

Is this helpful to add to the docs?

— Reply to this email directly, view it on GitHub https://github.com/home-assistant/core/issues/101423#issuecomment-1759481556, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUIP7ZRS6QENXNY7O5RSTLTX67MY5ANCNFSM6AAAAAA5THTNXA. You are receiving this because you were mentioned.Message ID: @.***>

mrhelder commented 7 months ago

I'm also running into this issue trying to setup my Tempest with the native integration

benquan commented 7 months ago

Has anyone managed to install it in a virtualbox HAOS installation?

rrubin0 commented 7 months ago

I have the same problem as described above. I'm running HA Core 2023.10.3 in a docker container using host mode but receive:

xtego commented 7 months ago

For anyone else as dumb as I am... make sure your weatherflow base station is plugged in. After about a half an hour of pulling my hair out I checked the base station only to find my toddler had unplugged it and I didn't notice. As soon as I plugged it in, this error went away without any manual settings needing to be added. As others have mentioned, this configless setup would require HA and weatherflow to be on the same subnet. That is kind of lame.

alvinchen1 commented 7 months ago

In the beginning I also had UDP errors because the WeatherFlow2MQTT AddOn was still running. After removing the AddOn the Weatherflow core integration is working fine.

Yup, this is what fixed it for me. When attempting to add the WeatherFlow Integration I got

But I had WeatherFlow2MQTT add-on as well, and it was still running. Once I shut it off, adding the native WeatherFlow integration worked immediately.

I have found that with adding https://github.com/briis/weatherflow_forecast too via HACS has added most of the missing sensors that the native integration lacks that I need. It also added the new weather forecast service that was introduced in 2023.9. Still a active development in progress (shoutout to developer @briis for all their great work), just fixed a few issues in the last few hours.

Right now, after some creative entity_id renaming, I have converted from WeatherFlow2MQTT to Native WeatherFlow + WeatherFlow forecast and am only have a couple things missing or wrong, none of which critical to me. Doing some digging before I submit bugs through.

Right now (for those thinking of taking the plunge from WeatherFlow2MQTT to Native WeatherFlow + WeatherFlow forecast here are the list of entities that I am currently missing that were in WeatherFlow2MQTT but are not in either native WeatherFlow integration or WeatherFlow forecast integration:

sensor.st_000xxxxx_battery_mode
sensor.st_000xxxxx_current_conditions
sensor.st_000xxxxx_delta_t (not missing, but seems to be off/calculating wrong, still researching)
sensor.st_000xxxxx_dewpoint_comfort_level
sensor.st_000xxxxx_fog_probability
sensor.st_000xxxxx_last_lightning_strike (might be new sensor.st_000xxxxx_time_of_last_lightning_strike)
sensor.st_000xxxxx_last_rain_start
sensor.st_000xxxxx_lightning_count_today
sensor.st_000xxxxx_lightning_distance
sensor.st_000xxxxx_lightning_energy
sensor.st_000xxxxx_rain_intensity
sensor.st_000xxxxx_snow_probability
sensor.st_000xxxxx_solar_elevation
sensor.st_000xxxxx_solar_insolation
sensor.st_000xxxxx_temperature_level
sensor.st_000xxxxx_wind_bearing
sensor.st_000xxxxx_wind_bearing_avg
sensor.st_000xxxxx_zambretti_number
sensor.st_000xxxxx_zambretti_text

Would say, if none of these are important to you, you can go ahead and do this. A tip, use developer tools to get an output of all your WeatherFlow2MQTT entities using this template:

{{ expand(integration_entities('Mosquitto broker')) | selectattr("attributes.attribution", 'defined') | selectattr('attributes.attribution', 'eq', 'Powered by WeatherFlow2MQTT') | map(attribute='entity_id') | list }}

rrubin0 commented 7 months ago

I figured it out. After some research, I discovered that the updated weatherflow uses UDP to communicate and my tempest was on my IoT network and therefore not routable to communicate. After moving the weather station to the same LAN as my Home Assistant, it connected immediately.

ronniemullinsjr commented 7 months ago

I figured it out. After some research, I discovered that the updated weatherflow uses UDP to communicate and my tempest was on my IoT network and therefore not routable to communicate. After moving the weather station to the same LAN as my Home Assistant, it connected immediately.

Same issue here. Scratched my head for days trying to figure this out, but it turned out I had my WeatherFlow Hub on a different WiFi network (assigned to a different VLAN) than HomeAssistant was on. After switching the WeatherFlow Hub over to the same VLAN, the native integration started immediately.

jeeftor commented 7 months ago

I’m happy to modify error messages in any way people think could be useful. I’m open to suggestions. Generally the devs like to keep them simple for translation support but obviously it seems a lot of people are getting suck on subnet / vlan broadcast issues

rrubin0 commented 7 months ago

A clear note in the front of the documentation would suffice. Many of us are migrating from the cloud, and so it simply didn’t occur to me that it’s required to be on the same LAN due to UDP because the previous integration was working.

On Sun, Oct 15, 2023 at 9:33 AM Jeef @.***> wrote:

I’m happy to modify error messages in any way people think could be useful. I’m open to suggestions. Generally the devs like to keep them simple for translation support but obviously it seems a lot of people are getting suck on subnet / vlan broadcast issues

— Reply to this email directly, view it on GitHub https://github.com/home-assistant/core/issues/101423#issuecomment-1763441282, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFCPURETOKXBTHAI4Y63GVLX7QF5XANCNFSM6AAAAAA5THTNXA . You are receiving this because you commented.Message ID: @.***>

jkaberg commented 7 months ago

I just wanted to chime in and say I got it working a few days ago - here's how;

removed the old cloud based integration from HACS and then HASS
restarted HASS
setup UDP relay (in pfsense) to relay port 50222 between the two VLAN's (since I got the Weatherstation hub sitting in an different VLAN vs HASS) - this step isn't necessary if you got them in the same LAN/VLAN
install the "official" integration as per normal
works 🎆

an side note; I'm running HASS in docker using macvlan which has it's own IP facing VLAN X, while the Weatherstation hub is sitting in VLAN Y

jbdesilet commented 7 months ago

Both the HACS version and the Core version use the same domain, the problem is the config_flow only loads the core UI version making it impossible to configure the HACS version.

Which HACS repo are you using? If it has a config flow file and uses the same domain, it will load appropriately. I've got it working on a machine that way myself.

Should have ported the REST version into the core, and made the UDP local only version optional.

Everyone has there preferences on which should come first, but both can exist in core. It just happens to be that the developers that worked on adding to core started with the UDP version.

Does this mean that there are plans to integrate the REST version into core? It would really be ideal to have the choice of which to use. I'm another one of the many using an IoT VLAN, and would love to be able to keep my Tempest on that VLAN and not have to mess with UDP forwarding in pfSense. Thank you for your work on this!

home-assistant / core