search

home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
73.48k stars 30.7k forks source link

Openweathermap daily mode not working #15105

Closed jckoester closed 5 years ago

jckoester commented 6 years ago

Home Assistant release with the issue: 0.72.0

Last working Home Assistant release (if known): none (new feature)

Operating environment (Hass.io/Docker/Windows/etc.): hass.io in virtual machine

Component/platform: weather.openweathermap

Description of problem: Adding the new mode option to the openweathermap configuration leads to an error when set to 'daily', using the (default) mode 'hourly' works as intended. The reason cited in the error log "Reason: Invalid API key" (full log see below) can't be true as

Though it looks like an api call is made that is only allowed on a paid subscription. If this is intended, it should be made clear in the documentation.

Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant): Not working

  - platform: openweathermap
    mode: daily
    api_key: !secret openweathermap_key

Working

  - platform: openweathermap
    mode: hourly
    api_key: !secret openweathermap_key

Also working:

  - platform: openweathermap
    mode: houry
    api_key: !secret openweathermap_key

Traceback (if applicable):

openweathermap: Error on device update!

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/pyowm/commons/weather_client.py", line 55, in _lookup_cache_or_invoke_API
    response = urlopen(API_full_url, None, timeout)
  File "/usr/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.6/urllib/request.py", line 532, in open
    response = meth(req, response)
  File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.6/urllib/request.py", line 570, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 401: Unauthorized

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/homeassistant/helpers/entity_platform.py", line 248, in _async_add_entity
    await entity.async_device_update(warning=False)
  File "/usr/lib/python3.6/site-packages/homeassistant/helpers/entity.py", line 320, in async_device_update
    yield from self.hass.async_add_job(self.update)
  File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/lib/python3.6/site-packages/homeassistant/components/weather/openweathermap.py", line 187, in update
    self._owm.update_forecast()
  File "/usr/lib/python3.6/site-packages/homeassistant/util/__init__.py", line 319, in wrapper
    result = method(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/homeassistant/components/weather/openweathermap.py", line 226, in update_forecast
    self.latitude, self.longitude, 15
  File "/usr/lib/python3.6/site-packages/pyowm/webapi25/owm25.py", line 653, in daily_forecast_at_coords
    json_data = self._api.call_API(DAILY_FORECAST_URL, params)
  File "/usr/lib/python3.6/site-packages/pyowm/commons/weather_client.py", line 94, in call_API
    return self._lookup_cache_or_invoke_API(self._cache, url, timeout)
  File "/usr/lib/python3.6/site-packages/pyowm/commons/weather_client.py", line 58, in _lookup_cache_or_invoke_API
    raise unauthorized_error.UnauthorizedError('Invalid API key')
pyowm.exceptions.unauthorized_error.UnauthorizedError: Your API subscription level does not allow to perform this operation
Reason: Invalid API key

Additional information:

fabaff commented 6 years ago

Works fine on dev with mode: hourly and mode: daily.

Also working:

 - platform: openweathermap
   mode: houry
   api_key: !secret openweathermap_key

This is definitely not working. Configuration validation will catch this.

jckoester commented 6 years ago

This is definitely not working. Configuration validation will catch this.

That for sure is a typo I introduced when copying the config snipped and changed mode: dailyto mode: hourly.

I'm still getting the error reported above in 0.72.1 with mode: daily but not with mode: hourly. Is there anything I can do to track down the error?

rdbahm commented 6 years ago

I can confirm dasmaeh's issue - works in "hourly," fails in "daily." Precisely the same errors as shown above. Seems like a documentation issue, primarily - I think we could just specify that "daily" requires a paid subscription.

mihalski commented 6 years ago

Potentially the daily mode could be derived from the hourly data if the component was modified to support it (not volunteering, still at the early stages of learning python)

balloobbot commented 6 years ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.

Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment :+1:

ad8mustanggt commented 5 years ago

I found this issue while troubleshooting this exact problem. Using daily kicks out the "Your API subscription level does not allow to perform this operation" error but using hourly works fine. Openweathermap haven't changed their website so it appears that daily should still be included in the free subscription but I've had no luck getting it to work.

FWIW - I'm running the 0.83.3 docker on unRAID.

e-alfred commented 5 years ago

I also use the free API key, but for me only "daily" works. I am using 0.86.2 in a Python virtual environment. Using "hourly" I am getting the "Your API subscription level does not allow to perform this operation" error.

tmechen commented 5 years ago

confirming this issue, default works (hourly) but mode: daily produces the 

unauthorized_error.UnauthorizedError('Invalid API key')
pyowm.exceptions.unauthorized_error.UnauthorizedError: Your API subscription level does not allow to perform this operation
Reason: Invalid API key

using hassio as a docker container on a raspberrypi 3b+

tmechen commented 5 years ago

some digging: openweathermap.py is calling (in update_forecast) the daily_forecast_at_coords() method from pyowm with parameter 15. this parameter is defined as limit in owm25.py, therefore this call tries to get 15 days forecast.
according to the pricing (https://openweathermap.org/price) free users only get 5 days forecast.
therefore this failing.
proposed fix: change parameter from 15 to 5 to allow free users to use this mode.

edit okay, i just tried, and calling the method with 5 results in the same error :/ 2nd edit found corresponding issue in pyowm: long story short: the 5days/3h forecast for free plans is just this. you wont get access to the daily forecast, you can just access the 3h forecast for the next 5 days :( one could now build the daily mode for free users to return just 5 values from the 3h forecast, with every day one value...

IanAdd commented 5 years ago

freedaily mode fills a lovelace custom:weather-card with erroneous data. The card shows correct current temp, humidity, wind etc at the specified lat/long, but then gives sunrise/set time of the current timezone - not that of the lat long specified timezone, gives the temp of the next few hours not the next 5 days. Any suggestions? Or shall I find another resource for weather in another place to my location? If so any recommendations?

tmechen commented 5 years ago

i am currently using entity: weather.met_home for my custom:weather-card because it is there anyways, therefore i didnt realize. will evaluate this evening thx

IanAdd commented 5 years ago

Thx, yr doesn't let me specify where I want the weather from. It uses my lat,long. So I use it for local weather but need weather from another location as well.

tmechen commented 5 years ago

hm, cannot confirm this bug with my installation? Bildschirmfoto 2019-08-08 um 09 58 31

top weather_card is with entity: weather.met_home while bottom one is entity: weather.openweathermap. while they do show different temperatures because of different sources the sunrise/set data is the same and the data is actually for 5 days. can you give more specific information about your setup? edit ah sorry, you did specify some special lat/long coords for your second weather card, did i get this right? sry, will try this too edit2 yes, can confirm the sunrise/set data does not work for different locations, but the actual weather data is for 5 days for me and looks valid

tmechen commented 5 years ago

okay, i checked the sources for the custom:weather-card and they use sun.sun for the times. (see https://github.com/bramkragten/custom-ui/blob/master/weather-card/weather-card.js#L137) therefore you cannot change the actual sunrise data for this custom weather card by changing the weatherprovider

IanAdd commented 5 years ago

Hi, I have found the issue is a little more subtle than I thought it would be.

The 2nd location is in a different time zone. I think you are selecting the anticipated temp on each day in the middle of the night at the 2nd location.

Note that the card is showing the time at my location against each day. Narrabeen is in Australia.

Ian

On Thu, 8 Aug. 2019, 09:06 tmechen, notifications@github.com wrote:

hm, cannot confirm this bug with my installation? [image: Bildschirmfoto 2019-08-08 um 09 58 31] https://user-images.githubusercontent.com/9123876/62685527-23174680-b9c3-11e9-8416-f6638182e71a.png

top weather_card is with entity: weather.met_home while bottom one is entity: weather.openweathermap. while they do show different temperatures because of different sources the times, the sunrise/set data is the same and the data is actually for 5 days see also next images

[image: Bildschirmfoto 2019-08-08 um 10 00 52] https://user-images.githubusercontent.com/9123876/62685703-78535800-b9c3-11e9-90b4-4f878f7ae0ba.png [image: Bildschirmfoto 2019-08-08 um 10 00 59] https://user-images.githubusercontent.com/9123876/62685705-79848500-b9c3-11e9-8ce6-8337083979d6.png

can you give more specific information about your setup?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/home-assistant/home-assistant/issues/15105?email_source=notifications&email_token=ACQBSNI5WMTDGY3YLB33XPTQDPHW3A5CNFSM4FGTDQ22YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD322CHY#issuecomment-519414047, or mute the thread https://github.com/notifications/unsubscribe-auth/ACQBSNLF63FZKORJHAPOYETQDPHW3ANCNFSM4FGTDQ2Q .

tmechen commented 5 years ago

the anticipated temp for each of the next five days is not a average temp for this day, but the temp at the current time for this day.
what freedaily mode is doing is:

but: current time is always the actual current time. therefore: when current time in your timezone is 1400 and in the second its 2300, both will take the datapoints from now on. and display the temperature for the following 5 days at the current time, so you will see for your time zone the data at 1400 while in the other it will display the temp data for every day at 2300, but displaying the wrong timestamp (your timezone)

would you like to, either:

IanAdd commented 5 years ago

Hi

I think most people would expect on a 5 day daily chart to see the peak temp for that day.

With time zone difference you'd need to consider that whilst it is Thursday here in the UK, it is already Friday in Sydney, Hawaii can be the opposite.

Thanks for your efforts.

Ian

On Thu, 8 Aug. 2019, 11:52 tmechen, notifications@github.com wrote:

the anticipated temp for each of the next five days is not a average temp for this day, but the temp at the current time for this day. what freedaily mode is doing is:

  • asking openweathermap for all the weather data for the next 5 days in 3 hour forecast mode, which is part of the free api key (see https://openweathermap.org/price)
  • they will return all datapoints (which are 5*(24/3)=35) from now on
  • every 8th datapoint is the datapoint for the current time on the corresponding day

but: current time is always the actual current time. therefore: when current time in your timezone is 1400 and in the second its 2300, both will take the datapoints from now on. and display the temperature for the following 5 days at the current time, so you will see for your time zone the data at 1400 while in the other it will display the temp data for every day at 2300, but displaying the wrong timestamp (your timezone)

would you like to, either:

  • display the temperature for the current time in the different timezone but with the correct timestamp, org
  • display the temperature for the corresponding time in the different timezone, in the example also 1400 but for this location?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/home-assistant/home-assistant/issues/15105?email_source=notifications&email_token=ACQBSNIEBBUIBD37J2EO3LTQDP3F7A5CNFSM4FGTDQ22YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD33HSRY#issuecomment-519469383, or mute the thread https://github.com/notifications/unsubscribe-auth/ACQBSNMJKZ6ZCMEMMOE4MSDQDP3F7ANCNFSM4FGTDQ2Q .

tmechen commented 5 years ago

yeah, thought about the peak possibility too, but: given the api posibilities every request delivers a different number of results. while a request at 1200 results in values for 1200, 1500, 1800, 2100 for this day, a request at 2000 would only result in the value for 2100. therefore, depending on the request time, the results would differ for peak values and this is why i decided to pick the corresponding value for the current time but for the next 5 days.

regarding the timezone problematic... yeah. like i said, the values are actually correct, so your request at the current time will show you the values for the current time over there too, but with the wrong timestamp, as openweathermap returns the results with the requested timestamp, which is your current timestamp for your hassio installation. the only possibility i see to correct this behavior would be some further time calculations to change the results.

tmechen commented 5 years ago

Bildschirmfoto 2019-08-08 um 14 24 21

tried some incredible hacky solution to rewrite the forecast object and its weather entrys to fix the timestamps... but i am not really sure if this should be included because of hacky hackness. i think the correct way to fix this would be changing the pyowm lib and return the correct timestamps for the corresponding long/lat.

edit
oh. https://github.com/csparpa/pyowm/issues/179 pyowm is returning UTC timestamps all the time. so the fix would actually be something like my hack

IanAdd commented 5 years ago

Since the only data needed in the 5 day fields is for the now/future of today, then daily highs for days after today they should be available . So getting 6pm temp for today at 4pm should be fine. Current time zone.

Now if i bend my brain over the distant time zone I think the same would be fine.

Ian

On Thu, 8 Aug. 2019, 13:28 tmechen, notifications@github.com wrote:

[image: Bildschirmfoto 2019-08-08 um 14 24 21] https://user-images.githubusercontent.com/9123876/62702969-689a3a80-b9e8-11e9-987b-9eb1686cf611.png

tried some incredible hacky solution to rewrite the forecast object and its weather entrys to fix the timestamps... but i am not really sure if this should be included because of hacky hackness.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/home-assistant/home-assistant/issues/15105?email_source=notifications&email_token=ACQBSNLQUDHW5TWSKMRUPA3QDQGOVA5CNFSM4FGTDQ22YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD33OJNA#issuecomment-519496884, or mute the thread https://github.com/notifications/unsubscribe-auth/ACQBSNLYA23ARP2NQGC7OIDQDQGOVANCNFSM4FGTDQ2Q .

IanAdd commented 5 years ago

Doing this on my phone so code review not practical!! Will happily comment this eve.

Ian

On Thu, 8 Aug. 2019, 14:06 Ian Addinsell, ian@addinsell.co.uk wrote:

Since the only data needed in the 5 day fields is for the now/future of today, then daily highs for days after today they should be available . So getting 6pm temp for today at 4pm should be fine. Current time zone.

Now if i bend my brain over the distant time zone I think the same would be fine.

Ian

On Thu, 8 Aug. 2019, 13:28 tmechen, notifications@github.com wrote:

[image: Bildschirmfoto 2019-08-08 um 14 24 21] https://user-images.githubusercontent.com/9123876/62702969-689a3a80-b9e8-11e9-987b-9eb1686cf611.png

tried some incredible hacky solution to rewrite the forecast object and its weather entrys to fix the timestamps... but i am not really sure if this should be included because of hacky hackness.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/home-assistant/home-assistant/issues/15105?email_source=notifications&email_token=ACQBSNLQUDHW5TWSKMRUPA3QDQGOVA5CNFSM4FGTDQ22YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD33OJNA#issuecomment-519496884, or mute the thread https://github.com/notifications/unsubscribe-auth/ACQBSNLYA23ARP2NQGC7OIDQDQGOVANCNFSM4FGTDQ2Q .