metno / pyaerocom

Python tools for the AeroCom project
https://pyaerocom.readthedocs.io/
GNU General Public License v3.0
23 stars 13 forks source link

EBAS reader: Handle time step sizes that differ over the provided time series #1228

Open jgriesfeller opened 1 week ago

jgriesfeller commented 1 week ago

This concerns all types of EBAS readers...

From EMEP report 2022:

Precipitation:
Amberd looks VERY high

Remaining issue: Når jeg ser på Amberd, så er måleperiodene totalt irregulære. Det varierer mellom en halv dag og flere dager 
(feks 10 jan fra 10:20 til 19:10, og 4.3 kl 7 til 9.3 kl 7). Dette klarer ikke Pyaerocom å håndtere (ihvertfall ikke enda), 
så disse dataene må vi kaste ut.

How it is today (to my best knowledge): Because the colocation can't handle time colocation with start and stop times. it has to assume a certain time step size and do the colocation on the middle points. We do accept small deviations from the time step sizes we have (details are here: https://github.com/metno/pyaerocom/blob/b06cfde9eeb62efc8e4d5dee38afda00a1196632/pyaerocom/tstype.py#L22-L42

But the devil might like in the details of the reader implementations

lewisblake commented 1 week ago

I would say we should ask the modelers how they want the comparison done and then we implement that rather than us coming up with a solution which they might not be satisfied with. If we are to include this data, what should we do with it? If no solution is given, then it is fair to throw out this station.

jgriesfeller commented 1 week ago

@lewisblake agreed. The "probblem" right now is that the station is not ignored automatically. (Yes, I should have mentioned that above) Pyaerocom's focus is to get as much data as possible.