Tomorrow.io integration not working anymore

[Remko76]

The problem

I have the Tomorrow.io integration set up and it has been working fine for weeks now. Since about 3 days it stopped working, all sensors show as unavailable and I have the corresponding messages about templates not being able to render. I tried deleting the integration and setting it up again, but that fails. After I enter my API key and press the "submit" button, it just sits there for a couple of minutes and then returns an "unknown error".

What version of Home Assistant Core has the issue?

2022.7.5

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Tomorrow.io

Link to integration documentation on our website

https://www.home-assistant.io/integrations/tomorrowio

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[probot-home-assistant[bot]]

tomorrowio documentation tomorrowio source _{^{(message by IssueLinks)}}

[probot-home-assistant[bot]]

Hey there @raman325, @lymanepp, mind taking a look at this issue as it has been labeled with an integration (tomorrowio) you are listed as a code owner for? Thanks! _{^{(message by CodeOwnersMention)}}

[Altycoder]

Same here

[MHerweijer]

Same here

[apetryk2]

Same here

[mvanlijden]

Also here all entities are unavailable for a few days now. Using the Free API, of which the provided URL in the dev console does give output. https://api.tomorrow.io/v4/timelines?location=-73.98529171943665,40.75872069597532&fields=temperature&timesteps=1h&units=metric&apikey=<>

Removing the integration and using the API key to set up again results in "Unknown error occurred".

[gbry71]

Same Issue

[apetryk2]

I added debug logging and got the following entry:

2022-07-20 07:13:07 DEBUG (MainThread) [homeassistant.components.tomorrowio] Finished fetching tomorrowio_ data in 0.933 seconds (success: False)

Note the "API KEY DELETED" was me just deleting my API key so I didn't post it on github as the log entry contains the api key itself.

It looks like this issue might be being addressed over at pytomorrowio:

https://github.com/raman325/pytomorrowio/pull/24

[apetryk2]

Also looks like it is addressed in: https://github.com/home-assistant/core/issues/75465

But it might be worth keeping that issue open instead of closed until it makes its way into a HA core update?

[GSzabados]

I've just bumped manually pytomorrowio to 0.3.4 and it does not resolve the issue:

2022-07-20 15:40:53 INFO (SyncWorker_2) [homeassistant.util.package] Attempting install of pytomorrowio==0.3.4
2022-07-20 15:41:16 INFO (MainThread) [homeassistant.setup] Setting up tomorrowio
2022-07-20 15:41:16 INFO (MainThread) [homeassistant.setup] Setup of domain tomorrowio took 0.0 seconds
2022-07-20 15:41:21 DEBUG (MainThread) [homeassistant.components.tomorrowio] Finished fetching Tomorrow.io - Home data in 4.775 seconds (success: False)
2022-07-20 15:41:21 WARNING (MainThread) [homeassistant.config_entries] Config entry 'Tomorrow.io - Home' for tomorrowio integration not ready yet; Retrying in background
2022-07-20 15:42:38 DEBUG (MainThread) [homeassistant.components.tomorrowio] Finished fetching Tomorrow.io - Home data in 16.358 seconds (success: False)

[GSzabados]

Restarted once again HA with 0.3.4 already installed after 5 minutes it started to work again randomly. My guess that something in the API changed again...

2022-07-20 15:55:30 INFO (MainThread) [homeassistant.setup] Setting up tomorrowio
2022-07-20 15:55:30 INFO (MainThread) [homeassistant.setup] Setup of domain tomorrowio took 0.0 seconds
2022-07-20 15:55:50 DEBUG (MainThread) [homeassistant.components.tomorrowio] Finished fetching Tomorrow.io - Home data in 18.572 seconds (success: False)
2022-07-20 15:55:50 WARNING (MainThread) [homeassistant.config_entries] Config entry 'Tomorrow.io - Home' for tomorrowio integration not ready yet; Retrying in background
2022-07-20 15:57:22 DEBUG (MainThread) [homeassistant.components.tomorrowio] Finished fetching Tomorrow.io - Home data in 4.025 seconds (success: False)
2022-07-20 15:57:42 DEBUG (MainThread) [homeassistant.components.tomorrowio] Finished fetching Tomorrow.io - Home data in 4.420 seconds (success: False)
2022-07-20 15:58:16 DEBUG (MainThread) [homeassistant.components.tomorrowio] Finished fetching Tomorrow.io - Home data in 1.282 seconds (success: False)
2022-07-20 15:58:57 DEBUG (MainThread) [homeassistant.components.tomorrowio] Finished fetching Tomorrow.io - Home data in 0.634 seconds (success: False)
2022-07-20 16:00:18 DEBUG (MainThread) [homeassistant.components.tomorrowio] Finished fetching Tomorrow.io - Home data in 0.702 seconds (success: True)
2022-07-20 16:00:18 INFO (MainThread) [homeassistant.components.sensor] Setting up sensor.tomorrowio
2022-07-20 16:00:18 INFO (MainThread) [homeassistant.components.weather] Setting up weather.tomorrowio

[lymanepp]

curl shows that they fixed the API to return 200 again. But 0.3.4 was working for me yesterday. You were likely rate-limited due to this issue and that takes up to an hour to clear.

[ulic75]

I've just bumped manually pytomorrowio to 0.3.4

How does one manually bump pytomorrowio?

[lymanepp]

How does one manually bump pytomorrowio?

No need, it looks like tomorrow.io has fixed the root cause.

[GSzabados]

But 0.3.4 was working for me yesterday.

Tomorrow.io was working for me until 17:30 CET yesterday, and I haven't had any issue before that. But since until now it was not working. I was using 0.3.3 and then 0.3.4 and both had the same result. (success: False)

I did check twice, that the pytomorrowio has been updated to 0.3.4, before I posted the first comment. Unfortunately DEBUG doesn't give any further information, just the False flag.

[lymanepp]

Did you restart HA after updating to 0.3.4? Did you check if you were still rate-limited?

[GSzabados]

How does one manually bump pytomorrowio?

You manually enter the container and edit the component's manifest file.

[GSzabados]

Did you check if you were still rate-limited?

As I see, I haven't hit the rate limit for a while.

But the daily number suggests I should be limited: 834 / 500 of available calls, but my guess, that is calculated wrongly and adds there the 4XX responses as well and calculated on a rolling 24 hour window maybe.

And yes, I restarted HA after installing the 0.3.4.

[lymanepp]

Add -i parameter to curl to see the rate-limit headers.

[robin-vandaele]

curl shows that they fixed the API to return 200 again. But 0.3.4 was working for me yesterday. You were likely rate-limited due to this issue and that takes up to an hour to clear.

Even when I see it returning 200, I'm still not getting any data from tomorrow.io integration (didn't bump to 0.3.4)

[GSzabados]

When surpassing your rate-limiting you will get a response with a 429 error code.

https://docs.tomorrow.io/reference/api-usage-limiting#rate-limiting

I think the graph tells it clearly, that there wasn't an active rate limit.

But I cannot see that it has hit any daily limitation, because it reports there are 122 left for today.

[bigcookie]

in const.py, the rate limit for the component is set to 100 by default. I always have to increase it manually by editing const.py and restart HA. Not sure, if this is the case for you, but it made me hit the rate limit (form the component - NOT the tomorrow.io account) after HA updates/Tomorrow.io updates...

[raman325]

we no longer use that constant. Instead we read the headers and adjust the time interval dynamically. So you shouldn't hit the rate limit

[JonathanTroyer]

Running into this issue as well. I attempted reinstalling the integration. With pytomorrowio 0.3.3, it hangs after putting in the API key. With pytomorrowio 0.3.4, it accepts the API key and finishes setup, but then does not expose any entities. I made sure that I was not rate limited.

[lymanepp]

does not expose any entities

Screenshot please.

[lymanepp]

Even when I see it returning 200, I'm still not getting any data from tomorrow.io integration (didn't bump to 0.3.4)

The API change caused the integration to repeatedly retry the calls. Which likely caused the account to be rate-limited for some period of time. Try using curl with -i option to see what your response looks like. Paste the status line and headers in your response.

[JonathanTroyer]

Setting up

Post setup

Nothing in the logs about a setup failure, but I don't have debug levels enabled.

[lymanepp]

@JonathanTroyer, the post setup image is definitely odd. It should say "28 entities" next to "1 service". Can you please try curl with the -i option and paste the status line and headers in a response?

[JonathanTroyer]

HTTP/2 200
date: Wed, 20 Jul 2022 18:53:53 GMT
content-type: application/json; charset=utf-8
x-ratelimit-limit-second: 3
x-ratelimit-limit-hour: 25
x-ratelimit-limit-day: 500
ratelimit-limit: 3
ratelimit-remaining: 2
x-ratelimit-remaining-second: 2
ratelimit-reset: 1
x-ratelimit-remaining-hour: 16
x-ratelimit-remaining-day: 364
x-request-id: 003d2171-21ed-48cf-a99f-6f41e4dabcc2
vary: Origin
access-control-allow-origin: *
x-moesif-transaction-id: c39f5c1e-4f0c-4e22-8cb3-674970b929cd
x-kong-upstream-latency: 15
x-kong-proxy-latency: 1
via: 1.1 google
cf-cache-status: DYNAMIC
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
server: cloudflare
cf-ray: 72dde6f88a596761-DFW

[lymanepp]

Well, that is normal and there's no rate-limiting in effect. Can you try restarting HA and see if it's still in that state?

[robin-vandaele]

Are you sure you correctly changed to 0.3.4 for the pytomorrowio integration? For me it started working when I changed it. Before, it wouldn't work even if I got 200 responses.

[lymanepp]

Now that the API is returning status code 200, upgrading to 0.3.4 is not necessary.

[robin-vandaele]

Now that the API is returning status code 200, upgrading to 0.3.4 is not necessary.

But it didn't work anyway... Even if I restated 10 times.. It only started working once I updated. Maybe a coincidence.

[JonathanTroyer]

First restart: integration was no longer active, I had to re-add it.

Second restart: integration was active after HomeAssistant finished

I can keep restarting and it always comes back to the same state as the second screenshot.

[lymanepp]

But it didn't work anyway... Even if I restated 10 times.. It only started working once I updated. Maybe a coincidence.

The only change in 0.3.4 was to accept 206 as successful. The API is no longer returning 206.

Due to retries, you were probably rate-limited and the rate limit happened to expire at the same time.

[lymanepp]

I can keep restarting and it always comes back to the same state as the second screenshot.

I haven't seen it do that before. Can you humor me and try curl with -i again to make sure that you didn't get rate-limited again?

[JonathanTroyer]

I can keep restarting and it always comes back to the same state as the second screenshot.

I haven't seen it do that before. Can you humor me and try curl with -i again to make sure that you didn't get rate-limited again?

Absolutely:

HTTP/2 200
date: Wed, 20 Jul 2022 19:06:39 GMT
content-type: application/json; charset=utf-8
x-ratelimit-limit-second: 3
ratelimit-remaining: 2
x-ratelimit-limit-hour: 25
x-ratelimit-remaining-hour: 23
x-ratelimit-remaining-day: 360
ratelimit-reset: 1
x-ratelimit-limit-day: 500
ratelimit-limit: 3
x-ratelimit-remaining-second: 2
x-request-id: 11d1c41c-092c-4306-89ac-9675170357d2
vary: Origin
access-control-allow-origin: *
x-moesif-transaction-id: dd142116-b8a1-4ee6-a446-74818ed99363
x-kong-upstream-latency: 26
x-kong-proxy-latency: 1
via: 1.1 google
cf-cache-status: DYNAMIC
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
server: cloudflare
cf-ray: 72ddf9ae0862aa33-DFW

[lymanepp]

@raman325 we need to add additional logging to make troubleshooting possible.

[lymanepp]

x-ratelimit-remaining-day: 360

That is 4 less than your previous headers from 12 minutes earlier, so assuming those weren't all from curl, it looks like there have been some successful API calls.

[GSzabados]

I would like to highlight a really important piece of information, what you are not recognizing.

The OP said:

Since about 3 days it stopped working

My Tomorrow.io setup stopped working yesterday, and was working normally prior that. If the issue started for some 3 days ago, then it means Tomorrow.io has shards for different geo locations which ran different states of their API. SmartThings does the same and the changes on the cloud side are deployed at different times to different geo locations.

Meanwhile @JonathanTroyer could you have a look at your logs and search for any other tomorrowio messages?

There is another issue open, which might have an impact on your situation, as it might be another change in the Tomorrow.io API.

https://github.com/home-assistant/core/issues/74644

[JonathanTroyer]

I would like to highlight a really important piece of information, what you are not recognizing.

The OP said:

Since about 3 days it stopped working

My Tomorrow.io setup stopped working yesterday, and was working normally prior that. If the issue started for some 3 days ago, then it means Tomorrow.io has shards for different geo locations which ran different states of their API. SmartThings does the same and the changes on the cloud side are deployed at different times to different geo locations.

Meanwhile @JonathanTroyer could you have a look at your logs and search for any other tomorrowio messages?

There is another issue open, which might have an impact on your situation, as it might be another change in the Tomorrow.io API.

74644

I see no other errors or warnings, unfortunately. For me, the integration stopped functioning about 20 hours ago (memory is fuzzy, don't take that as certain).

[GSzabados]

Mine stopped working at 17:30 CET 2022.07.19, which was about 28 hours ago. I get it from the history of the entity, when it went unavailable.

[ScottESanDiego]

It became unavailable for me 2022-07-19 08:52 PST, and remains that way as of this writing.

[lymanepp]

Besides the 206 status code and rate limits, I don't have anything to go on. Could one of you with the problem use curl with -i option and save the response to a file and attach it here?

[Remko76]

Unfortunately I do not have the entities anymore, since I uninstalled the integration. If you would provide some more guidance on the curl command, I could try it out. I run HASS OS on an RPi4 and have the terminal add-on installed...

[gbry71]

Looks like mine dropped out the same as @ScottESanDiego Except I'm on EST time. If that helps anyone.

[kreene1987]

Here is mine, CST:

I've duplicated the integration into custom_components and updated the dependencies per latest dev, but mine still isn't working either. Logs don't show much

`Logger: homeassistant.config_entries Source: config_entries.py:384 First occurred: 1:44:22 PM (1 occurrences) Last logged: 1:44:22 PM

Config entry 'ClimaCell' for tomorrowio integration not ready yet; Retrying in background`

[zacs]

Doubtful that it is useful, but I am in the same boat, mine stopped about 8am yesterday (Pacific time) and I get the "integration is not ready" error.

[d3sbo]

yes I to get integration is not ready" error.

[ScottESanDiego]

Here's what I get:

scotte@ServicePi ~ $ curl -i  --request GET --url 'https://api.tomorrow.io/v4/timelines?location=-73.98529171943665,40.75872069597532&fields=temperature&timesteps=1h&units=metric&apikey=[redacted]'
HTTP/2 200
date: Wed, 20 Jul 2022 21:04:49 GMT
content-type: application/json; charset=utf-8
x-ratelimit-limit-hour: 100
x-ratelimit-limit-day: 1000
ratelimit-remaining: 2
ratelimit-reset: 1
x-ratelimit-remaining-second: 2
x-ratelimit-remaining-hour: 98
x-ratelimit-remaining-day: 960
ratelimit-limit: 3
x-ratelimit-limit-second: 3
x-request-id: 3eaada78-40fe-4888-92e1-8b223e7be4f3
vary: Origin
access-control-allow-origin: *
x-moesif-transaction-id: 76dc0618-718f-49c3-9876-8fb8c64134dc
x-kong-upstream-latency: 22
x-kong-proxy-latency: 54
via: 1.1 google
cf-cache-status: DYNAMIC
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
server: cloudflare
cf-ray: 72dea6c6f8757cfd-LAX

{"data":{"timelines":[{"timestep":"1h","

[snip]