mendhak / waveshare-epaper-display

At-a-glance dashboard for Raspberry Pi with a Waveshare ePaper 7.5 Inch HAT. Date/Time, Weather, Alerts, Google/Outlook Calendar
https://code.mendhak.com/raspberrypi-epaper-dashboard/
438 stars 65 forks source link

Cllimacell's API appears to return a High and Low temperature for the next 24 hours that may not be accurate #18

Closed lifeofbrian closed 2 years ago

lifeofbrian commented 3 years ago

It's returning 58F/45F for my area, when I'd expect to see 67 to 69F/46F to 49F. I'm not sure if this has more to do with latitude/longitude precision or something else.

@mendhak - does the high and low temp seem consistent for your area?

mendhak commented 3 years ago

The Climacell high/low today for me is -1, and -3 C. Strangely today, that matches up exactly with MetOffice (in the UK, the MetOffice is the national weather service). Usually it's slightly off and I've just accepted it as part of each of these providers' networks, or whatever sources they have available.

Forecast for next few days:

image

And on the Met Office site:

image

lifeofbrian commented 3 years ago

Thanks -- I'll try out a few other locations (e.g. Seattle, WA; Boise, ID; San Diego, CA) in the coming days to further validate. This might be a symptom of where the weather stations are closest to me (I'm at the top of a valley, and the elevation increases as you drive north -- so the weather station it's keying off of may vary significantly from my city's main forecasts

mendhak commented 3 years ago

This week I was twice given inaccurate weather codes (but temperatures were fine). On days where it's supposed to be hazy (fog) I was given the code for freezing rain, and then flurries.

So here's something to try, next time there are strange or 'wrong' readings. Try the timelines API with timestep = 1d and then again try it with timestep = 1h

https://data.climacell.co/v4/timelines?location=x,y&fields=temperatureMin&fields=temperatureMax&fields=weatherCode&timesteps=1d&apikey=xxxxxxxxxxxxx

https://data.climacell.co/v4/timelines?location=x,y&fields=temperatureMin&fields=temperatureMax&fields=weatherCode&timesteps=1h&apikey=xxxxxxxxxxxxx

In my case I noticed that the flurry weather code (5001) only appears when timestep=1d, but not when timestep=1h.

mendhak commented 3 years ago

I've emailed Climacell support but I'm not really expecting a proper response.

feh123 commented 3 years ago

Hi @mendhak I am part of a project (PiClock) that use Climacell too. We had a response from them over inaccuracies:

_Hey Vin,

PiClock does look super cool.

Roni here, from the product team. I read through the threads, some of the issues reported go back to December and are not really relevant to the current state of the API. I'll elaborate.

Initially, v4 used the same precipitation models as v3 - so there shouldn't have been discrepancies between the two at the time of the report. Since then, v4 was upgraded to our most advanced model - even better than the previous one. Moreover, we added a new timestep to make it easier to fetch current conditions (even though they were also available at the time of the report, by querying the minute closest to realtime).

I recommend asking the maintainers of PiClock to revisit our new API. It has major advantages to the previous ones in regards to performance, current and incoming features and developer experience altogether.

Let me know if I can be of further assistance.

Best, Roni_

On Thursday, 4 March 2021, 08:55:39 GMT, mendhak ***@***.***> wrote:  

I've emailed Climacell support but I'm not really expecting a proper response.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

mendhak commented 3 years ago

Hey @feh123 thanks for posting that. I didn't get a response from Climacell but my observations were the opposite of what they were describing, the 'better' version was back in December, and now it's a bit hit and miss. At least for my location

(also I agree with their response, PiClock looks cool!)

I've been thinking about this situation a bit more. I was recently adding in the Outlook integration as another calendar provider for this epaper display and realized it would be useful if I allowed for a few, just a few, other weather providers.

If users don't have success with Climacell maybe they could try another API. At the same time I don't want this repo to explode with complexity so it'll only be a select few.

I'm going to start experimenting with this concept soon. The top providers I can think of are WeatherBit, OpenWeatherMap. Any others? I've not used AccuWeather, is it good for US forecasts?

feh123 commented 3 years ago

Hi @mendhak this very debate continues on PiClock - Dark Sky is not the first weather provider it has lost. They are following your idea - give users choices. Currently I think (I have Dark Sky until end 2021 so not updated my PiClock yet) they have Climacell V3, OpenweatherMap and METAR. All offer pluses and minuses. They are waiting to see if Climacell V4 improves! PiClock members like very frequent weather updates and lots of forcasts so there discussion is very useful. You might like to link to them and take part too. Incidentally I am about to switch on my updated waveshare-epaper-display. Thanks for all the extra stuff and the events number - I am tryng 7 - I will let you know how it goes!

feh123 commented 3 years ago

Hi @mendhak I have updated and all seems good - I like all your additions. Successfully added seven events! Just one thing - is there a run log file at all - I just wanted to see if everything was running correctly - no errors on ./run.sh but I would like to check the calendar and weather update rates. Thanks a lot.

mendhak commented 3 years ago

That's good to hear.

Yes there's a log file, you can look for the run.log, that's assuming you've set up the cron job like this:

* * * * * cd /home/pi/waveshare-epaper-display && bash run.sh > run.log 2>&1

There will be a little info in there and errors should appear too.

It's possible to turn on detailed logging by changing the 'LogLevel.INFO' to 'LogLevel.DEBUG' at the top of the Python files, but it might be a bit overwhelming - I plan to make that log level configurable later.

feh123 commented 3 years ago

Thanks @mendhak I forgot the cronjob created the log file! Everything in the log looks fine.Good luck with the weather providers.

feh123 commented 3 years ago

Hi @mendhak thanks for getting all these providers! I did think of another one: my Home Assistant uses the Norwegian Meteological Institute - it seems quite good with several updated current forcasts a day.

I had a question for you - my system has been modified as you know - how best to update it now you have added all these choices? Currently my V4 believed today would be overcast with drizzle (its a clear sky and bright sunshine) so I think I would like to change! Thanks.

mendhak commented 3 years ago

@feh123
I'm currently working in a separate branch, it's not available in the main application yet.

If you want to start using the new weather providers, there's two ways for you to do that.

First way - you can wait a bit (few weeks?) until that branch gets merged, then on your Raspberry Pi just download the latest update (git pull).

The second way, if you do want to test out these new providers early on, then on your Raspberry Pi you'd have to switch branches to the one I'm working on.

git pull 
git checkout weatherproviders

I have some instructions in that branch's README on how to add the different weather providers.


Yesterday I added providers for the Met Office and OpenWeatherMap.

I will look at AccuWeather - I'm assuming US people find it more accurate?

I could also look at the Met.no API.

I had a look at METAR but it's for reporting current conditions, I do believe many weather APIs do use METAR as part of their own number crunching though.

feh123 commented 3 years ago

Hi @mendhak Happy to wait - but if I clone the merged git will it not overwrite the .svg and .py files I have modified too? It's not major problem - I will keep copies and add back my modifications - I just wanted to check that was the way to update.

I do use Accuweather (on Home Assistant) and it seems good too. So currently I am looking at V4, Dark Sky, Accuweather and Met.No. If I had to rank them I would go Dark Sky>Accuweather~Met.No>V4. Part of this will be due to the update rate - how often the software checks it's current conditions. It's more difficult to compare forcasts so I do not find it easy to rank them. This must also be location dependent so maybe not relevant to other countries. Although my run log file says every hour it checks the V4 data it is still saying drizzle so V4 does appear fairly poor.

mendhak commented 3 years ago

Yeah that sounds right - you'll have to git clone (you should be able to git pull from within the directory, and it ought to complain if it notices you have modified some files). And then restore your modifications.

For this piece of work there shouldn't be too many changes in the SVG. The only one I've had to make is for AccuWeather, which supplies a very useful "short phrase" - which requires two lines, so I've split the WEATHER_DESC into a _1 and a _2 to accommodate this.

AccuWeather does look good, only niggle is it has a very small rate limit, about 50/day which makes development a bit restrictive. That's why there's caching in place though.

mendhak commented 3 years ago

@feh123 @lifeofbrian

I have something workable now and it's merged into the main branch. That means if you pull down the latest version of this git repo you will get all the new weather providers; OpenWeatherMap, AccuWeather, Met Office, Met.no. Plus some extra icons in the SVG (including volcano and tornado!). Also the weather description now spans two lines if the description is long.

Anyway the important part: I've added instructions on configuring the different weather providers.

Pick a weather provider, register on their site, perform whatever gymnastics they require, get API keys, put it in env.sh. Comment out other weather providers.

Switching between weather providers is easy. Comment out the ones you're not using, uncomment the one you want to use, and delete weather-cache.json. The weather-cache.json will contain the HTTP response from the weather provider API, for troubleshooting. That's why when switching providers, you need to delete that json file.

Some notes: I didn't add weatherbit, maybe some other time. I mainly wanted to get the "provider" structure in place. And get some feedback about whether this is working out.

Met.no only provides a 6 hour forecast at a time, so you'll often see high/low quite close to each other. It's a bit more work to get the code to walk over the Met.no response and try to figure out the high and low for that day, but again left that for another time.

feh123 commented 3 years ago

Hi @mendhak sorry for the slow response. More problems with my other RPi devices this week! Thanks for this I will try out the new version in the next few days. I look forward to being able to switch - should be a good test of the best providers.

feh123 commented 3 years ago

Hi @mendhak - just another one - https://www.visualcrossing.com/  PiCLock is thinking about using it.

On Wednesday, 7 April 2021, 21:20:40 BST, mendhak ***@***.***> wrote:  

@feh123 @lifeofbrian

I have something workable now and it's merged into the main branch. That means if you pull down the latest version of this git repo you will get all the new weather providers; OpenWeatherMap, AccuWeather, Met Office, Met.no. Plus some extra icons in the SVG (including volcano and tornado!). Also the weather description now spans two lines if the description is long.

Anyway the important part: I've added instructions on configuring the different weather providers.

Pick a weather provider, register on their site, perform whatever gymnastics they require, get API keys, put it in env.sh. Comment out other weather providers.

Switching between weather providers is easy. Comment out the ones you're not using, uncomment the one you want to use, and delete weather-cache.json. The weather-cache.json will contain the HTTP response from the weather provider API, for troubleshooting. That's why when switching providers, you need to delete that json file.

Some notes: I didn't add weatherbit, maybe some other time. I mainly wanted to get the "provider" structure in place. And get some feedback about whether this is working out.

Met.no only provides a 6 hour forecast at a time, so you'll often see high/low quite close to each other. It's a bit more work to get the code to walk over the Met.no response and try to figure out the high and low for that day, but again left that for another time.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

jmason commented 3 years ago

for what it's worth, I've switched to openweathermap and it's working great...

mendhak commented 3 years ago

@feh123 I had a look at VisualCrossing and it didn't look difficult at all to implement so I just went ahead and did it. I've even merged it into master (yolo...) If you get the repo's latest version you can try it out. Just add export VISUALCROSSING_APIKEY=xxx into env.sh.

There is a limitation though, maybe just for now - the icon set is very limited.

There is a positive, they have a description phrase, like "Becoming cloudy in the afternoon", just like AccuWeather.

It's not clear on the VisualCrossing website, but you don't need a credit card. You can sign up free, and you're limited to 1000 req/day. You only need to pay if you go above that.

@jmason me too! OpenWeatherMap is working well for me, and AccuWeather too. I like AccuWeather's 'description' phrase, because it allows for variation.

I emailed Met Office to ask if their API could provide the description phrase but they said their API is purely data.

feh123 commented 3 years ago

Hi @mendhak I just updated and I will start with Accuweather too. Everything looks great. Thanks!

mendhak commented 2 years ago

Closing this as it's been 1 year since any activity. Since the original problem I've added several other weather providers.