search

earthobservations / wetterdienst

Open weather data for humans.
https://wetterdienst.readthedocs.io/
MIT License
360 stars 55 forks source link

Implementing weather interpolation #519

Closed gutzbenj closed 2 years ago

gutzbenj commented 3 years ago

Discussed in https://github.com/earthobservations/wetterdienst/discussions/325

Originally posted by **Arcadia197** January 21, 2021 Hello dear community of wetterdienst. I recently became aware of your project and am interested in its development and would possibly want to contribute (as far as I will have spare time avaiable for more projects). I didn't fully explore all capabilities of the package yet, but as far as I noticed, you mainly focus on weather data avaiable from the stations at specific locations. Have you already thought about implementing an interpolation of nearby stations to approximate weather results at a desired location (for example your own position)? This may be of high interest for users wanting to extract high quality weather data for their positions. Of course for some weather parameters it doesnt really make sense, but especially for steady values (temperature, pressure) this should be easily achieveable with simple interpolation models. Greetings, Julius

Dear @Arcadia197,

since the latest PR we now have a streamlined set of parameter names with which we can identify any parameter coming from any weather service. I had this as a requirement in mind to define which parameters can be interpolated.

Now we can e.g. check if a parameter is of temperature nature, precipitation nature, ... and apply different measures such as minimum required correlation for incorporated neaighbouring stations etc. I had also written down some notes at #513 where @StefanR44 is currently working on some stuff.

Would you like to join me on implementing a feature for interpolation based on neighbouring stations? I'd suggest for now going with a simple MLR model and later on advance the method.

Cheers, Benjamin

meteoDaniel commented 3 years ago

@gutzbenj this can be easily solved by using a cKDTree to return n nearest stations.

I guess wetterdienst should not be the place to hold complex interpolation algorithms, what do you think?

Arcadia197 commented 3 years ago

Interpolation is not complex though. I thought about just implementing Spline interpolations for now with different stencil size. Dependent on the outcome and accuracy, one could always implement other, higher order interpolations later.

However, while I'm fit with the math and scientific programming part, I have no clue where to start and what to look out for regarding modern python and packaging programming. Maybe you could lend me a hand with that, @gutzbenj .

I think getting an estimate at your exact location is an important feature which could open up a lot of possible connections to other services. I'm pretty sure that for example smart-home community would love to get their hands on interpolated weather data at a location from reliable sources here in germany.

gutzbenj commented 3 years ago

@meteoDaniel I think interpolation per individual weather service is not too much for this lib, as we already have certain methods for the selection of stations and interpolation will only be the cherry on top ;)

@Arcadia197 I've quickly did a check and we could go with scipy respectively interp2d [1]. While this probably already has everything we need I have three concerns:

[1] https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.interp2d.html#scipy.interpolate.interp2d

StefanR44 commented 3 years ago

Hey guys, I'm done with exams and progressing through an python course. Have you done anything in the last days? I am reading through the interpolate docs right now

gutzbenj commented 3 years ago

Dear @StefanR44 ,

I have just got some free time the next days and will try so set something up following the notes. If you want you can fork the repository and create a new branch and from that I could set up a PR! If you need further help let me know! I may also hand you some books (pdf) for Python if you are interested.

StefanR44 commented 3 years ago

Sounds great, maybe we can have a little chat over discord or something else so you can give me a hands-on overview of the API and what could be worked on?

amotl commented 3 years ago

Dear Stefan,

I also would like to recommend our discussion forum [1] for having such a conversation beyond the scope of this one, for the benefit of saving this information as a reference for others asking for the same thing. I would appreciate that, so feel welcome to join us at [1]. At [2], I just created a corresponding topic where this could be handled.

In this manner, we can keep this very thread focused on the interpolation topic.

With kind regards, Andreas.

[1] https://community.panodata.org/ [2] https://community.panodata.org/t/wip-a-hands-on-introduction-to-wetterdienst/224

StefanR44 commented 3 years ago

@amotl That sounds great, thanks! I am currently trying some things (especially related to #513 ) and will post my problems and progress here and on the forum you have mentioned.

gutzbenj commented 3 years ago

@StefanR44 If you'd like to, I could also guide you via WhatsApp, which I believe is much faster for smaller issues. If you are fine with that just send me an email.

StefanR44 commented 3 years ago

@gutzbenj cool! sent you a mail with my number

gutzbenj commented 3 years ago

I added a PR Draft that I'll be working on the next days.