openclimatefix / pvoutput

Python code for downloading PV data from PVOutput.org
Apache License 2.0
34 stars 17 forks source link

Convert downloaded data from localtime to UTC #14

Open JackKelly opened 5 years ago

peterdudfield commented 2 years ago

so make sure all data is in UTC?

JackKelly commented 2 years ago

Yeah, converting to UTC sounds good... If I remember correctly, the data from PVOutput.org doesn't have an explicit timezone but is in London timezone. I think somewhere in the PVOutput code it'll be necessary to do something like df = df.tz_localize("Europe/London").tz_convert("UTC") (and then removing these two lines from nowcasting_dataset/data_sources/pv/pv_data_source.py)

peterdudfield commented 2 years ago

some of it done in - https://github.com/openclimatefix/pvoutput/pull/102

JackKelly commented 2 years ago

Just in case we come back to this later and need to remind ourselves (!): BB at PVOutput.org says that PVOutput.org returns data in the PV system's local time.

When we get round to downloading PVOutput.org data from other countries (which Sheffield Solar are already doing, I think) then we'll need to be able to give the pvoutput code multiple timezones. e.g. we need to tell pvoutput "these PV systems are in Europe/London timezone. These other PV systems are in Rome/Italy timezone. Please convert them to UTC".

JackKelly commented 2 years ago

A neat way to solve issue https://github.com/openclimatefix/nowcasting_dataset/issues/620 would be to modify pvoutput so the new code explicitly sets the timezone to UTC (i.e. the times would have an explicit UTC offset, like this: 2022-03-17 08:00+00:00). That way, nowcasting_dataset (and any other code that consumes data downloaded by pvoutput) can be certain which timezone the data is in :slightly_smiling_face: (and, in general, perhaps we should always default to saving timestamps with explicit UTC offsets, to avoid any ambiguity. Although I seem to remember that NetCDF doesn't like timezone-aware timestamps, although I could be wrong!)

peterdudfield commented 2 years ago

I agree, that would be neat.

Needs a bit of thinking as somewhere we have to set the timezone, and that could proably be loaded from metadata