Weather Underground no longer provides free API Keys

[joeymitchll]

Weather Underground no longer provides API Keys for free, and I have tried replacing the config URL with a separate free API service and put in a key to the ApiKey.py file, but no weather data appears. I also get a Json decoding error - raise ValueError ("No JSON object could be decoded") Value Error: No JSON object could be decoded The Clock and Map data work, but no radar or forecast data appears. Any solutions?

[Knackster]

My current install x2 are working fine, even through the storm we had early this morning.

[kd7eir]

Weather underground will give you access to YOUR data for the small sum of $800 per month... The solution is for everyone to stop submitting data to them and CWOP, since CWOP forwards their data to WU as well. I've personally cut off three feeds to WU.

On Mon, Jun 4, 2018 at 4:06 PM joeymitchll notifications@github.com wrote:

Weather Underground no longer provides API Keys for free, and I have tried replacing the config URL with a separate free API service and put in a key to the ApiKey.py file, but no weather data appears. I also get a Json decoding error - raise ValueError ("No JSON object could be decoded") Value Error: No JSON object could be decoded The Clock and Map data work, but no radar or forecast data appears. Any solutions?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/n0bel/PiClock/issues/103, or mute the thread https://github.com/notifications/unsubscribe-auth/ADdwuKwI6xUS5GFei9QrK6pi7mly_f3Xks5t5b1igaJpZM4UZ6cF .

[martymoose]

Weathercloud is ok Weather bug haven't tried

On Tue, 5 Jun. 2018, 9:15 am kd7eir, notifications@github.com wrote:

Weather underground will give you access to YOUR data for the small sum of $800 per month... The solution is for everyone to stop submitting data to them and CWOP, since CWOP forwards their data to WU as well. I've personally cut off three feeds to WU.

On Mon, Jun 4, 2018 at 4:06 PM joeymitchll notifications@github.com wrote:

Weather Underground no longer provides API Keys for free, and I have tried replacing the config URL with a separate free API service and put in a key to the ApiKey.py file, but no weather data appears. I also get a Json decoding error - raise ValueError ("No JSON object could be decoded") Value Error: No JSON object could be decoded The Clock and Map data work, but no radar or forecast data appears. Any solutions?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/n0bel/PiClock/issues/103, or mute the thread < https://github.com/notifications/unsubscribe-auth/ADdwuKwI6xUS5GFei9QrK6pi7mly_f3Xks5t5b1igaJpZM4UZ6cF

.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/n0bel/PiClock/issues/103#issuecomment-394529679, or mute the thread https://github.com/notifications/unsubscribe-auth/AA0PD5TNoxo89Kw3TUxyhsVyoL3XOlQMks5t5b-RgaJpZM4UZ6cF .

[n0bel]

Looks like current API keys still work.

It'll take a bit of time but I'll need to pick out another provider and make it work with their api.

The forecast is easy to find a replacement provider. Good radar providers is harder.

[JoshuaKimsey]

OpenWeather is possibly the best replacement in that case, @n0bel. Unless you could physically pull from the NWS itself. It is possible, they offer API’s for that data, but it’s definitely a build it from the ground up experience. I know, I looked into it for another project.

The other option would be to scrape the data off of wunderground, which can easily be done with Python. I can’t nor won’t speak to the ethicalness of this, but it is an option I have used to gather such data as weather warnings and whatnot when needed

EDIT: Wow at the angry comments to that forum thread announcing the cessation of the API! It’s hard to believe that WU is gonna ruin the work of so many people...

[joeymitchll]

Any ideas on why replacing the config API URL to a separate provider and have the correct API key in the ApiKeys.py file not working? The clock and map data show up, but no forecast or anything appears in the yellow borders. Is there any extra editing I should do in the config file?

[n0bel]

Some background info..... The weather channel bought weather underground some years ago. weather.com Then starting last year we saw some changes in the WU apps on android, (to make them less useful I think). In April WU apps on android started being deprecated, you are being prompted to install a new app from weather.com as its replacement. I believe we are seeing the merging of WU into Weather.com... It just took some years. And of course weather.com does nothing for free. There are many adjectives I'd apply to weather.com.

[n0bel]

@joeymitchll because no two weather APIs are the same. There is no standard.

[n0bel]

@hsoj95 OpenWeather? I can find OpenWeatherMap. I don't see anything about animated radar images in their api, just static ones. I suppose PiClock could do the animation by keeping x images. Less than optimal, but I guess workable. I'd love to hear more solutions.

[n0bel]

OpenWeathermap.org's free API says that it updates weather api information "< 2 hours" so.. If the radar image is 2 hours old this is simply not workable. Even the current temperature being 2 hours hold is pretty useless.

[JoshuaKimsey]

@n0bel This is truly unfortunate... I didn't realize the 2 hour issue with OpenWeatherMaps. Would web scraping be an option? Or what about using information straight from the NWS?

[JoshuaKimsey]

Here's the link to the NWS API page btw: https://graphical.weather.gov/xml/

[maserowik]

Would https://darksky.net/dev/docs/libraries be a good option to use to replace the wu api?

[n0bel]

@maserowik It looks good for everything except for radar. Hey have radar maps, but I can't find any api access. I hesitate to get radar from NWS, because its of course not world wide.

[merky1]

temps.zip

weather.gov - pros - least likely to change over time (gotta love USG) cons - CONUS only.

aerisweather.com - pros - seems to be a drop in replacement?
cons - could be back in the same situation in a year.

openweather.com - pros - the price is free!!! still referenced by google?(not sure if this + or -) cons - sketchy / delayed weather

I like the weather.gov approach, but then I'm a greedy fat 'Merican :). Not sure how much time I have to parse the code, but I remember going through it 6-8 months ago when wunderground radar API lost reliability. That and as a native PERL coder python gives me a rash :)

EDIT: weather.gov is highly unreliable for current observations. I just polled weather station KJYO for 1 hour, and got results that were "stale" | null | or just plain wrong based on every other API provider I use including the thermometer outside. Frustrating, since I would have thought weather.gov would be more likely to keep things accurate.

[merky1]

After running temp grabs over night, it feels like openweather may be the best replacement. Aeris limits to 750 calls per day, and seems laggy. It felt that Aeris was really just pulling from weather.gov, where as openweather had additional sources to pull from.

My vote has switched over to openweather, at least for current observations. temps.zip

[merky1]

Sorry, hate to spam. I pulled together a few things, and wrote an example using node.js as the backend and HTML / JS as the frontend. Mainly because that's what I work on all day. I merged openweather and weather.gov data, so it is definitely not ready for prime-time. I am also trying to work out the map / radar layers, but there is no weather today, so the test is invalid.

I'm not sure if this approach is maintainable, but it does validate that the datapoints exist. I'm also not sure of the etiquette (fork project/generate a new one/etc) to share this out.

Latest screenshot taken off of a RaspberryPI LCD (800x480) :

[maserowik]

merky1, care to share you code?

[merky1]

Ok - no warranty, may cause universe implosion. Use at your own risk. standard GPL.

Here is my extremely early work. You will need to adjust the lat / lon in both the .js and index.html. piclock.js will need your openweather api key. index.html will need your google maps api. Eventually I will clean up all the js in the index.html, but I only spent part of the weekend on it thanks to a sprained ankle.

install instructions (wholly untested): 1 - install chromium-broswer 2 - install node.js 3 - npm install request cron 4 - node piclock.js 5 - nohup chromium-browser --kiosk http://localhost:8081 &

"should work" radar images are CONUS only as far as I can tell. Direct pulls from weather.gov included way too much radar noise. Resolution is less than optimal. Animation is not implemented yet. I just refresh current every 5 minutes.

[JoshuaKimsey]

@merky1 Dude, this is amazing work for such a short amount of time! I have a few ideas though. Instead of using a Node.JS server and having the user launch a browser to localhost, why not use an Electron shell to make it an all in one app? That way all they have to do is open the app and have it work!

[merky1]

I forked the repo for now. I figured since it was a major deviation it made more sense to keep things seperated. https://github.com/merky1/PiClock

[ScottChapman]

I've been passively following this for a while hoping to see some reasonable resolution. My clock had been running great for a number of years. But when I got back from vacation I noticed that my radar images appeared to be missing the underlying maps (but did show the rain radar correctly).

Doesn't seem like that would be a side-effect of this issue... I appear to still have a 500calls/day 10calls/min free plan. Looks like my clock does < 200 GIF calls/day and < 40 JSON calls/day. Does that seem right?

[merky1]

@ScottChapman it looks like you may need to update / redo your google maps key. Google recently updated their T&C's and forced everyone to renew the key.

[ScottChapman]

@merky1 - I actually rebooted it this morning and it seems good again.

So why am I not impacted by either the google thing or the wunderground thing??

[merky1]

@ScottChapman for the google key, Google slides the t&c's into everything, so I'm willing to chalk that up to tachyons. Wunderground is in a weird legacy working state, but new keys are a no go, and there is no SLA for when it will go away forever.

If it works for you, hang on to it until it dies. My research into replacements led me to Openweather for the textual aspects, but radar is CONUS (US government sources) only.

[n0bel]

@ScottChapman existing for WU keys continue to work at this time, yes. I also I saw some google maps outages as well. All back online now.

[n0bel]

@merky1 Interesting that this is the approach I took on my original PiClock, before it moved to python. It was a cool saturday afternoon project, that you can read about in my blog.

[n0bel]

I'm going to do a simple plugin to replace the WU provider. This will require a common return data set for all providers, which is something I wish wasn't needed.. (flexibility).. The dataset will probably be formatted much like the WU return data. The radar is more flexible since it does paint it self. It is simply provided a box to paint into.

[n0bel]

I'm torn between 1) handing the painting task to the provider give it a forecast block (right side) a current block (top left) Sunrise/sunset (bottom), and it paints within the box. 2) coming up with a common api dataset format (with potential extra useful data going unrealized) 3) (last option) just get it done -- recode for a new provider and leave it hard coded.

Input wanted from all interested parties. @dankolbrs Are you still interested in this project?

[ScottChapman]

I'm an incrementalist. I suggest #2.5; just get it done, but take the oppty to reasonably normalize the dataset format.

If this were a NodeJS app I'd love to help, just not too smooth with Python...

[merky1]

I used nodejs in my fork. I figured better to fork than to rationalize python vs js. I guess I should have done Perl, but everyone calls me old for using that.

[dankolbrs]

@n0bel, I intended to standardize the weather returns to ease adding new apis, but never got around to completing it. I'm not sure I've even got enough work on it on some branch to even give a head start.

I like the weatherapi superclass approach and separate classes that standardize the data. While this could run into issues with standardizing data or losing features, it would be much easier to switch around. So, I vote for 2.

With the amount of comments from people having the issue, looks like getting it done whatever way possible and quickest is the way to go. Either way shouldn't take too terribly long to do, I'm still interested in the project, but may be a bit before I've got time to contribute this piece.

[harmfulmushroom]

Unfortunately no coding talent here, but came to drop a comment suggesting that my API key which I've had for a little over a year and a half appears to not be working any longer. Been a good run though, now it's just gray boxes :(

This was meant as a PSA as up until now it looked like pre-existing API's worked. I can confirm mine at least no longer works.

[merky1]

My fork works within the US. I use open weather and weather.gov as providers (both free). It is based on node.js though, and is a significant deviation that may require a “reflash” of the pi.
https://github.com/merky1/PiClock

I would pull into this repo, but the python vs. node difference is pretty significant.

[dankolbrs]

@merky1, nice!

I've been playing around with the openweather API and their radar API isn't quite as nice as wunderground's. I see the function in your code that calls the radar, and I wonder if I can get some info from you on how to determine the x and y for that tile. Openweathermap uses tiles for their radar API and I'm just not seeing how to translate lat/lon to a tile.

$ grep -nRiA7 gettileurl | tail -9
--
PiClock/public/client.js:115:       getTileUrl: function(tile, zoom) {
PiClock/public/client.js-116-           return "https://mesonet.agron.iastate.edu/cache/tile.py/1.0.0/q2-n1p-900913/" + zoom + "/" + tile.x + "/" + tile.y +".png?"+ (new Date()).getTime(); 
PiClock/public/client.js-117-       },
PiClock/public/client.js-118-       tileSize: new google.maps.Size(256, 256),
PiClock/public/client.js-119-       opacity:0.60,
PiClock/public/client.js-120-       name : '1hr Precipitation',
PiClock/public/client.js-121-       isPng: true
PiClock/public/client.js-122-   });

https://github.com/merky1/PiClock/blob/master/public/client.js#L115

What determines the tile numbers? And what's up with that mesonet.agron.iastate.edu URL? Is that a mirror of weather.gov or another api/web scrape altogether?

I looked quick at weather.gov but didn't see an API reference (I could easily have scanned it too quick, apologies if I overlooked it and it's an easy answer (and because I don't know JS)).

[merky1]

Easy first - mesonet is a us government grant that takes the NEXRAD data and cleans it up. Otherwise, the raw radars return too much “noise”. US only though..

X,y is part of the google maps get tile function based on the map display. Basically does the math for me so I don’t have to figure out bounding boxes. I’m sure similar functionality exists in python. The only “work” I put in was a way to cycle 5 minute increments over 20 mins.

[ghost]

Data point on old wunderground keys working. They sort of do, but I just accessed http://api.wunderground.com/api/$key/conditions/forecast10day/astronomy/q/KABQ.json (where $key is the key I've been using for years), and got back data, but:

"observation_time":"Last Updated on September 27, 8:52 AM MDT"

And I downloaded this data at October 2, 09:39AM MDT, so the data I'm getting back is really old.

[JoshuaKimsey]

@barrycarter It's very possible that an EOL for the WU API may not be "officially" announced, and instead something like this will happen where the data just stops being updated. hich makes me rather ornery, as I'd like a way to gather my weather data from my weather station. But it basically only works with WU, and the highly specific MyAccurite app... :/

[victoriagardner]

Weather Underground's API's end of service was announced here: https://apicommunity.wunderground.com/weatherapi/topics/end-of-service-for-the-weather-underground-api

Additional information for personal weather station (PWS) owners will be forthcoming.

Feel free to direct questions about the API to me via private email or on the WU Forum.

Victoria Gardner victoria.gardner@ibm.com

[JoshuaKimsey]

Well... At least we'll maybe have access to the data on our own PWS's...

[ScottChapman]

@merky1 - looks like I'm going to have to cut over to your implementation before that date!

Do you have a list of stuff that isn't working yet (areas where it deviates from original PiClock)?

[victoriagardner]

@SouthernWolf95 -- PWS owners will have uninterrupted access to their data.

[merky1]

I pull forecasts from usg, so currently us only. Analog clock is broken.

Everything else works.

[feh123]

Hi - From the UK I still get the weather and radar but goggle maps is blank. So usable until end of year. Is there a way to get to full function via another radar and weather provider for UK/EU? I would be willing to pay for the Google maps but little point if I can get no radar to work. From the thread I get the impression there is little hope! Thanks

[rcomito]

Hi feh123 - Google maps now requires that you sign up for an API key.  They charge .02 USD per pull, but the first $200USD (10,000) per month are free. Rick Comito

[feh123]

Thanks Rick. Victoria says the TWC will offer the API to non-commercial developers - can PiClock use this?

[rcomito]

Hi feh123 - They're talking about a new API.  It will probably require some recoding

-----Original Message----- From: feh123 notifications@github.com To: n0bel/PiClock PiClock@noreply.github.com Cc: Rick Comito rcomito@verizon.net; Comment comment@noreply.github.com Sent: Sun, Oct 28, 2018 4:46 am Subject: Re: [n0bel/PiClock] Weather Underground no longer provides free API Keys (#103)

Thanks Rick. Victoria says the TWC will offer the API to non-commercial developers - can PiClock use this?— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[ScottChapman]

considered something like this for images: https://www.rainviewer.com/api.html and darknet for actual weather?

[feh123]

Thanks. The radar looks great - with a good UK source. All I have to do is replace the WU references in PiClock with rainviewer - this is probably well beyond my coding abilities unfortunately. As anyone done something similar I could copy?

[dankolbrs]

I've looked into this a bit but haven't been able to make the changes and PR yet due to recently moving and all the fun stuff that goes along with that.

OpenStreetMap tiles can take the place of google maps (but no satellite view).

OpenWeatherMap would cover both radar and forecast data.

The changes for the weather would be a bit more involved as it'd be a matter of mapping to the different json response, but shouldn't be too bad.