Problem reading solar data

[pedroalencar1]

Hi! I am having issues requesting data from solar datasets.

When I run the code below I keep getting a message back: "The combination of solar, daily, historical is invalid." Note that I didn't set historical period.

station_3988 = DwdObservationRequest(
    parameter=DwdObservationDataset.SOLAR,
    resolution=DwdObservationResolution.DAILY,
    start_date = datetime(1950,1,1),
    end_date = datetime(2021,12,31)).filter_by_station_id(station_id=[3988])

I have just started using the module and not quite familiar with it, so it might be just that I am making a mistake that I simply can't see (in that case, I am really sorry 😅 ). However, I basically copied the code from here (line37; which also yields the same message when I simply copy-paste).

Many thanks for the great work and I hope you can help me out on this one 😃

[gutzbenj]

Dear @pedroalencar1 ,

thanks for writing in about this issue! Apparently solar data (both the one of hourly and daily resolution) has no real period in the sense of what the DWD uses as category [1], but the new data is just appended on historical data instead of creating those period chunks seen on the file server.

The period you've seen is smartly selected by our request depending on the start and end date you've entered. However the Period.HISTORICAL was missing in our internal period mapping per each DWD observation dataset.

https://github.com/earthobservations/wetterdienst/blob/e77d121489db441897c39ec958273627fd1708d4/wetterdienst/provider/dwd/observation/metadata/dataset.py#L199

I've just added period historical.

After fixing our mapping this bug should be fixed. I will asap draw a new release which you can then install to continue your research!

If you have further problems using the lib please don't hesitate and contact us!

Cheers Benjamin

[1] https://opendata.dwd.de/climate_environment/CDC/observations_germany/climate/daily/solar/

[gutzbenj]

Dear @pedroalencar1 , the latest version includes the fix! Please update and your code should work as expected! Btw the station you selected doesn't exist for daily data!

Cheers Benjamin

[pedroalencar1]

Dear @gutzbenj,

many thanks for the quick return. I was until recently on holidays and only now I got the time to check the package. It is working perfectly! The station number I sent was the one from the example I found online. Today I successfully got the data from Potsdam (3987).

One other small issue that caused me a bit of trouble to figure out is the column names in the solar data frames. In python they come as radiation_sky_long_wave | radiation_sky_short_wave_diffuse | radiation_sky_short_wave_direct | sd_strahl, but in R (where I am more familiar), they are ATMO_STRAHL | FD_STRAHL | FG_STRAHL | SD_STRAHL. The latter is the same as the one from the DWD documentation

At first, when working on python, I summed the three columns (radiation_*) to get the global radiation, but the values where different from those in R. Just then I realized that the values at radiation_sky_short_wave_direct are equal to those in FG_STRAHL (in python and R respectively), that would be the global radiation.

Maybe it is only a terminology that I am not familiar with, and please disregard this comment then.

Once again many thanks for the great work on this module. It's been really useful!

Cheers

Pedro

[gutzbenj]

Dear @pedroalencar1 , Sorry for the laaate answer!

After reading through the description once again I think I've made a mistake naming FG_Strahl radiation_global_direct. It should rather be radiation_global and then for the user to determine the radiation_global_direct one would simply have to subtract the diffuse radiation FD_Strahl from it.

For you to get the global radiation you would simply have to query FG_Strahl (currently named radiation_global_direct, will rename it to radiation_global). No summation needed for you in this case. ATMO_Strahl does not count into global radiation as it is long wave radiation, whereas global radiation refers to short wave radiation!

Cheers Benjamin