The main idea for drift detection is to compare one barometer with some other in its vicinity, and to look at how they differ. If the mean difference after some time is large, then this might indicate a drift. For non-drifting barometers we expect white noise around the mean (which is primarily determined by the baro difference in altitude):
Although the two barometers are 126km apart, their differences are nicely constant over the years, which suggests that either both are drifting the same way, or that none is drifting. Here and here is an other nice one.
On the other hand, if one is drifting and the other is not, then we would get a clear slope in the mean over time like in the following image:
Sometimes the difference patterns seem cyclic, which I do not understand. I have made a separate post for this here: #49. Anyone a clue? Anyway, there are 4428 unique informative plots based on pairwise comparisons like this. (In total there are many more -- some 200 x 200 barometer place comparisons -- but their timestamps do not always match, they do not overlap, have more than 10 % outliers based on gwloggeR::detect_outliers(), etc. So there is quite some data wrangling to be done to compare them correctly.)
Sometimes, they both seem to drift in opposite directions and thus the difference effect is reinforced:
And if they would drift in the same direction, we wouldn't even see a difference. I haven't found a telling example of this kind.
Anyway, pairwise comparisons are thus problematic because:
they do not tell us which one is drifting,
and if they both drift the same way, we do not even see it.
To remove these problems we compare one baro with many others. The idea is that even is one or two of the many are drifting, they would not obscure our inference about drift.
The dots are differences with other barometers. The red is the time-wise median. The yellow curve is a spline and mainly showing how the mean is behaving, but since the mean is heavily influenced by outliers, is not a good measure here. The median (red) is the one that should be stable.
BAOL815X is a good barometer. The red noise is quite stable. And the following one is clearly drifting:
This one seems to have started drifting:
And what about this one:
Don't understand what might be causing this periodic pattern.
Some problems with this approach:
Non-matching timestamps: one barometer has measurements on 22h, other on 23h. and some even have minutes and seconds. Some "universal" way is needed to match the observations. Due to timestamp mismatching the plots are sometimes unbalanced in certain ranges. See here for full reference.
In case of comparisons with many barometers, wrong conclusions might be drawn if e.g. at one point 100 barometers are available, and at some other point in time 50 others that stand at a higher (or lower) altitude are suddenly not available. The median (red) would shift at that point.
ToDo
Influence of temperature? See #52 and #54.
Seasonality effects? See #50.
Better way of matching timestamps.
Check auto-correlation: Although pressure is dependent of the previous one, the dependence disappears after enough time. So if the time-intervals are large enough, normal noise model fits well enough to even detect drifts without comparisons with other barometers.
The main idea for drift detection is to compare one barometer with some other in its vicinity, and to look at how they differ. If the mean difference after some time is large, then this might indicate a drift. For non-drifting barometers we expect white noise around the mean (which is primarily determined by the baro difference in altitude):
Although the two barometers are 126km apart, their differences are nicely constant over the years, which suggests that either both are drifting the same way, or that none is drifting. Here and here is an other nice one.
On the other hand, if one is drifting and the other is not, then we would get a clear slope in the mean over time like in the following image:
Sometimes the difference patterns seem cyclic, which I do not understand. I have made a separate post for this here: #49. Anyone a clue? Anyway, there are 4428 unique informative plots based on pairwise comparisons like this. (In total there are many more -- some 200 x 200 barometer place comparisons -- but their timestamps do not always match, they do not overlap, have more than 10 % outliers based on
gwloggeR::detect_outliers(), etc. So there is quite some data wrangling to be done to compare them correctly.)Sometimes, they both seem to drift in opposite directions and thus the difference effect is reinforced:
And if they would drift in the same direction, we wouldn't even see a difference. I haven't found a telling example of this kind.
Anyway, pairwise comparisons are thus problematic because:
To remove these problems we compare one baro with many others. The idea is that even is one or two of the many are drifting, they would not obscure our inference about drift.
The dots are differences with other barometers. The red is the time-wise median. The yellow curve is a spline and mainly showing how the mean is behaving, but since the mean is heavily influenced by outliers, is not a good measure here. The median (red) is the one that should be stable.
BAOL815X is a good barometer. The red noise is quite stable. And the following one is clearly drifting:
This one seems to have started drifting:
And what about this one:
Don't understand what might be causing this periodic pattern.
Some problems with this approach:
ToDo