entur_public_transport http error 429 rate limit exceeded

Aviatorpaal commented 1 year ago

The problem

Same as #66141

ERROR (MainThread) [enturclient.api] Error connecting to Entur, response http status code: 429 shows up frequently in my logs. In the documentation I am unable to rate limit the API calls from the integration, even if the Note states:

Note that the underlying API is rate limited and to avoid getting your instance blocked from entur the sensor is only fetching new information every 45 seconds. It's recommended to not schedule updates more often than this.

Suggested fix: allow users to rate limit the calls from the sensor in configuration.yaml or sensor.yaml

What version of Home Assistant Core has the issue?

2023.1.7

What was the last working version of Home Assistant Core?

NA

What type of installation are you running?

Home Assistant OS

Integration causing the issue

entur_public_transport

Link to integration documentation on our website

https://www.home-assistant.io/integrations/entur_public_transport#faq---troubleshooting

Diagnostics information

2023-01-24 17:26:31.032 ERROR (MainThread) [enturclient.api] Error connecting to Entur, response http status code: 429 2023-01-24 17:29:31.118 ERROR (MainThread) [enturclient.api] Error connecting to Entur, response http status code: 429 2023-01-24 17:32:31.011 ERROR (MainThread) [enturclient.api] Error connecting to Entur, response http status code: 429 2023-01-24 17:35:31.012 ERROR (MainThread) [enturclient.api] Error connecting to Entur, response http status code: 429 2023-01-24 17:38:31.017 ERROR (MainThread) [enturclient.api] Error connecting to Entur, response http status code: 429 2023-01-24 17:41:31.024 ERROR (MainThread) [enturclient.api] Error connecting to Entur, response http status code: 429 2023-01-24 17:44:31.040 ERROR (MainThread) [enturclient.api] Error connecting to Entur, response http status code: 429 2023-01-24 17:45:32.090 ERROR (MainThread) [frontend.js.latest.202301100] :0:0 Script error. 2023-01-24 17:47:31.061 ERROR (MainThread) [enturclient.api] Error connecting to Entur, response http status code: 429 2023-01-24 17:50:31.062 ERROR (MainThread) [enturclient.api] Error connecting to Entur, response http status code: 429 2023-01-24 17:53:31.067 ERROR (MainThread) [enturclient.api] Error connecting to Entur, response http status code: 429 2023-01-24 17:56:31.069 ERROR (MainThread) [enturclient.api] Error connecting to Entur, response http status code: 429 2023-01-24 17:59:31.081 ERROR (MainThread) [enturclient.api] Error connecting to Entur, response http status code: 429 2023-01-24 18:05:31.085 ERROR (MainThread) [enturclient.api] Error connecting to Entur, response http status code: 429

Example YAML snippet

# Entur
- platform: entur_public_transport
  name: Bussavganger
  show_on_map: true
  stop_ids:
    - "NSR:StopPlace:53916" #Stop 1
    - "NSR:StopPlace:53099" #Stop 2

Anything in the logs that might be useful for us?

Infoscreen running on an iPad H24 with Home Assistant Companion with sensors displaying bus departures from this integration

Additional information

NA

home-assistant[bot] commented 1 year ago

Hey there @hfurubotten, mind taking a look at this issue as it has been labeled with an integration (entur_public_transport) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `entur_public_transport` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Change the title of the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign entur_public_transport` Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

entur_public_transport documentation entur_public_transport source _{^{(message by IssueLinks)}}

Chr1stian commented 1 year ago

This is because the client-id is the same for every Home Assistant user: https://github.com/home-assistant/core/blob/dev/homeassistant/components/entur_public_transport/sensor.py#L25

I can try to implement a fix, but I dont know the proper way to identify a client in HA. Optimally this would be something like API_CLIENT_NAME = "homeassistant-unique-id-per-install"

Any easy imports that I can do that fixes this? @hfurubotten

pvarlien commented 1 year ago

Christian says that he doesn't really know how to uniquely identify an instance of Home Assistant. I'm not sure there is. An idea that occurs to me is that the integration has an optional parameter "name" (defaults to "Entur" IIRC), that at least some users are configuring. I have mine set to "Reise" ("Travel"). Using this value as part of the API client name should help a little.

It may also be worth double checking the code to verify that API_CLIENT_NAME is going into the correct HTTP header (ET-Client-Name). The Entur docs say that unidentified clients will be subjected to strict rate limitation. (https://developer.entur.org/pages-intro-authentication)

home-assistant / core