GIOS integration is not initialising

[GSzabados]

The problem

Since the 18th of April the integration is not loading. It stuck at the initialisation stage. Turned on debug logging and it has not revealed anything, just it has timed out after 30 second. So increased timeout to 60, but it still times out.

API endpoints are there. Checked them manually from a browser. Sometimes response fails in the browser on the API and then loads quickly, like nothing happened, but the response comes through at the end.

2024-04-24 20:58:06.290 DEBUG (MainThread) [homeassistant.components.gios] Finished fetching gios data in 60.001 seconds (success: False)

What version of Home Assistant Core has the issue?

Core-2024.4.4

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

GIOS

Link to integration documentation on our website

https://www.home-assistant.io/integrations/gios/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[home-assistant[bot]]

Hey there @bieniu, mind taking a look at this issue as it has been labeled with an integration (gios) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `gios` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign gios` Removes the current integration label and assignees on the issue, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

---

gios documentation gios source _{^{(message by IssueLinks)}}

[bieniu]

Please attach a debug log file for GIOS integration and diagnostics file.

[GSzabados]

The debug log is completely useless with the 30 seconds timeout there wasn't any logs. I have increased finally the timeout to 600 seconds, and disabled the integration for 1 hour.

The final result is:

2024-04-24 22:39:55.396 DEBUG (MainThread) [homeassistant.components.gios] Finished fetching gios data in 179.799 seconds (success: True)

Even before disabling the integration I had failures after 270 seconds. Maybe the API is playing up, or the 1 minute retry interval throws everything away and the API just reacts with a ban.

[bieniu]

Please do not modify the code (timeout) but attach the files I ask for. I am not able to help without diagnostic data.

[GSzabados]

All of these sensors are updating on a 1 hour interval, why the retry interval has to be 10 seconds? I tried the API though a VPN and have not seen any lag.

2024-04-24 21:18:49.118 DEBUG (MainThread) [homeassistant.components.gios] Finished fetching gios data in 134.770 seconds (success: False)
2024-04-24 21:18:59.468 DEBUG (MainThread) [homeassistant.components.gios] Using station_id: XXX
2024-04-24 21:18:59.469 INFO (MainThread) [gios] Initializing GIOS for station ID: XXX

Seriously? 10 seconds? I though I have seen 1 minute before in the previous logs.

Do you recognise, that this website is operated by a governmental entity and it can be considered a DDoS attack?

[GSzabados]

I am not able to help without diagnostic data.

What diagnostic data? Status: 200? Or the missing lines of the API endpoints, because no reply for 30 seconds from the findAll endpoint at all, due to pinging the server with an off-beat 30 second then 10 second?

[bieniu]

What diagnostic data?

[GSzabados]

I am not. I am from a small city, and I am not willing to share that.

[bieniu]

In that case I’m not able to help you 🤷‍♂️

[GSzabados]

Could you tell me please then what you would look for in a log like this? Because I did not see anything, but that the integration failed to start with 30 second timeout.

 DEBUG (MainThread) [homeassistant.components.gios] Using station_id: XXX
 INFO (MainThread) [gios] Initializing GIOS for station ID: XXX
 DEBUG (MainThread) [homeassistant.components.gios] Finished fetching gios data in 30.180 seconds (success: False)

In reality it started to work with the following timing.

2024-04-24 21:18:59.468 DEBUG (MainThread) [homeassistant.components.gios] Using station_id: XXX
2024-04-24 21:18:59.469 INFO (MainThread) [gios] Initializing GIOS for station ID: XXX
2024-04-24 21:19:35.439 DEBUG (MainThread) [gios] Data retrieved from http://api.gios.gov.pl/pjp-api/rest/station/findAll, status: 200

Do you recognise that between this 2024-04-24 21:18:59.469 and this 2024-04-24 21:19:35.439 there are more than 30 seconds? And no, I am not running HA on an RPi3. (Processor usage is about 8-12%, and has about 6GB free memory.)

And no, I don't have any other integration which would use dacite, but it is on the latest version 1.8.1 also aiohttp uses 3.9.5 which is the latest as well.....

Most likely the main problem is that the integration when it fails it start request data from the API in an abusive way and most likely the API implies a ban, or delayed service to the assigned IP address. If you are using one of the mobile operator's networks then you will most likely never see it, because you would have a public IP which is shared by hundreds of users. And reasonable they would never get banned, but if you have a fixed IP from a local internet service provided, then you would hit it straight.

Anyhow that 1 minute (or 10 second) retry interval is seriously abusive. Try to reload your integration a few times, maybe 5 to 10 times just one after the other, and check it will it still work, and how long will you get banned?

And I would be not surprised if together with my next monthly bill I would receive a notice why am I doing DDoS attack on public sites.

[bieniu]

The integration polls the API for new data every 30 minutes. In case of failure, another attempt to get data is made after 30 minutes. You can see it in this log

2024-04-25 09:59:02.142 ERROR (MainThread) [homeassistant.components.gios] Timeout fetching gios data
2024-04-25 09:59:02.142 DEBUG (MainThread) [homeassistant.components.gios] Finished fetching gios data in 30.001 seconds (success: False)
2024-04-25 10:29:32.142 DEBUG (MainThread) [homeassistant.components.gios] Finished fetching gios data in 30.001 seconds (success: False)

This behavior is far from a DDoS attack.

If the integration is unable to get data during HA startup, Home Assistant tries to initialize it again and there is a pause after each attempt. Such behavior can also hardly be considered a DDoS attack.

2024-04-25 10:55:36.199 DEBUG (MainThread) [homeassistant.components.gios] Using station_id: 401
2024-04-25 10:55:36.199 INFO (MainThread) [gios] Initializing GIOS for station ID: 401
2024-04-25 10:56:06.200 DEBUG (MainThread) [homeassistant.components.gios] Finished fetching gios data in 30.001 seconds (success: False)
2024-04-25 10:56:26.583 DEBUG (MainThread) [homeassistant.components.gios] Using station_id: 401
2024-04-25 10:56:26.584 INFO (MainThread) [gios] Initializing GIOS for station ID: 401
2024-04-25 10:56:56.586 DEBUG (MainThread) [homeassistant.components.gios] Finished fetching gios data in 30.002 seconds (success: False)
2024-04-25 10:57:36.809 DEBUG (MainThread) [homeassistant.components.gios] Using station_id: 401
2024-04-25 10:57:36.809 INFO (MainThread) [gios] Initializing GIOS for station ID: 401
2024-04-25 10:57:42.071 DEBUG (MainThread) [homeassistant.components.gios] Finished fetching gios data in 5.261 seconds (success: False)
2024-04-25 10:59:02.374 DEBUG (MainThread) [homeassistant.components.gios] Using station_id: 401
2024-04-25 10:59:02.374 INFO (MainThread) [gios] Initializing GIOS for station ID: 401
2024-04-25 10:59:32.376 DEBUG (MainThread) [homeassistant.components.gios] Finished fetching gios data in 30.002 seconds (success: False)
2024-04-25 11:00:52.801 DEBUG (MainThread) [homeassistant.components.gios] Using station_id: 401
2024-04-25 11:00:52.801 INFO (MainThread) [gios] Initializing GIOS for station ID: 401
2024-04-25 11:01:22.803 DEBUG (MainThread) [homeassistant.components.gios] Finished fetching gios data in 30.001 seconds (success: False)
2024-04-25 11:02:42.907 DEBUG (MainThread) [homeassistant.components.gios] Using station_id: 401
2024-04-25 11:02:42.908 INFO (MainThread) [gios] Initializing GIOS for station ID: 401
2024-04-25 11:03:12.909 DEBUG (MainThread) [homeassistant.components.gios] Finished fetching gios data in 30.002 seconds (success: False)
2024-04-25 11:04:33.344 DEBUG (MainThread) [homeassistant.components.gios] Using station_id: 401
2024-04-25 11:04:33.344 INFO (MainThread) [gios] Initializing GIOS for station ID: 401
2024-04-25 11:05:03.346 DEBUG (MainThread) [homeassistant.components.gios] Finished fetching gios data in 30.002 seconds (success: False)
2024-04-25 11:06:23.741 DEBUG (MainThread) [homeassistant.components.gios] Using station_id: 401
2024-04-25 11:06:23.741 INFO (MainThread) [gios] Initializing GIOS for station ID: 401
2024-04-25 11:06:53.742 DEBUG (MainThread) [homeassistant.components.gios] Finished fetching gios data in 30.001 seconds (success: False)
2024-04-25 11:08:13.806 DEBUG (MainThread) [homeassistant.components.gios] Using station_id: 401
2024-04-25 11:08:13.806 INFO (MainThread) [gios] Initializing GIOS for station ID: 401
2024-04-25 11:08:43.808 DEBUG (MainThread) [homeassistant.components.gios] Finished fetching gios data in 30.002 seconds (success: False)

If your integration behaves differently, maybe it is due to the modifications you made to the code? I don't know that.

Why do I need diagnostic data and debug logs? To analyze what it looks like for you. Check how many GIOS instances you have configured, for which measuring stations, and which custom components you use.

Please remember that Home Assistant is committed to protecting user privacy. Neither the diagnostic file nor the debug log for GIOS contains the coordinates of your HA instance, it only contains the ID of the GIOS measuring station, which is publicly available information.

[GSzabados]

If the integration is unable to get data during HA startup, Home Assistant tries to initialize it again and there is a pause after each attempt.

You just managed to reproduce what could be considered a DDoS attack, especially that the station information is updated every hour. Look at the frequency of those calls. And it just goes on and on and on without a stop or an extensive delay. And in my case for days already. Considering that is a public entity's server, with most likely limited funding, that is really putting the server under load, especially that the API has this really awkward setup for the graphs, and the old website of GIOS has been decommissioned in 2022, most likely sitting on the same servers without proper maintenance too.

It only does the update every half hour, again, the station data updates every hour, if the integration initialised at the first place. If not, then it just rams the server with continuous requests.

If your integration behaves differently, maybe it is due to the modifications you made to the code?

Updated the timeout to get passed the 30 seconds without any log entry issue. You can see that is not useful at all. You have not get even a clue has it made a call or not, because no feedback on the aiohttp's timeout, because it has not timed out yet. Only the integration implied the 30 seconds, but no log entry that it has made a call to any API endpoint.

By increasing to 60 seconds and later on to 600 seconds, I could finally see that the integration indeed does API calls, and how the API replies.

ID of the GIOS measuring station.

It just puts you straight on a map. Of course if you live in Warsaw or Krakow there are a half dozen to choose fro, but for obvious reasons, you would most likely choose the closest one.

I came across a custom integrations which has not deducted information properly and left PIN codes, GPS coordinates in the diagnostic file, so not a fun of this at all anymore.

[GSzabados]

I disabled the integration for a day, just now I turned it on again and it works immediately:

2024-04-26 15:45:41.634 DEBUG (MainThread) [homeassistant.components.gios] Finished fetching gios data in 0.886 seconds (success: True)

Maybe that initialisation should be re-considered.