DOV-Vlaanderen / groundwater-logger-validation

Analysis on validation methods for groundwater logger data
MIT License
2 stars 2 forks source link

Drifts 08/16: barometer altitude analysis #53

Open DavorJ opened 4 years ago

DavorJ commented 4 years ago

This analysis is mainly for detection of possible anomalous average (median) values of barometers using height information. I.e. if the calculated height based on barometric formula doesn't comply with the real height of the barometer, then there is a problem somewhere.

image

Here is a comparison of all the barometers (filter PRESSURE_VALUE < 1100 AND PRESSURE_VALUE > 975) with altitudes included with the name on the y-axis.

image

Zoom-in the plot for better view.

Here is the same plot without the vertical lines.

Here also the csv-file which might be used for comparison.

fredericpiesschaert commented 4 years ago

that doesn't seem to fit the real altitude very well ...

DavorJ commented 4 years ago

Drifts 16: difference between altitudes: barometric vs. TAW

OK, I see. Indeed, it looks quite off, but I think it can be explained partly.

First, the difference in altitude is mainly caused by the offset of the barometric formula. It assumes the 0 m seal level altitude at 1013.25 hPa at 15 °C, while TAW reference is based on the old measurements of the average low tide in Oostende. So expect to find an offset.

Secondly, once this offset is taken into account, I expect that if TAW is high, barometric altitude will also be high, and vice versa. In other words, there should be linear relationship between the two. And this is more or less what I find:

image

The slope is 0.86, while ideally it should be 1. The offset is -32.6 m. Actually, I would expect a perfect straight line with a slope of 1, so these outlying cases all must have their reasons. Part of the reason for the errors is that we need a very long timeseries (+4 years) to be able to reduce the error to +/-10 m (cf. Drifts 15).

Take BAOL015X_72527. According to barometric formula computed on the median, it should be around -84.6m below sealevel. In reality it is at 28.3m TAW. We adjust the barometric formula with offset -32.6 and slope 0.86, which results in: -8.3m. The absolute difference between -8.3m and -84.6m is 76.3m. Since for this barometer we have 4420 (12h interval) observations, we expect to make an error of approximately 17m based on the barometric formula. But the error of 76.3m is 4.5x higher (= relative error), which makes this barometer a big suspect. The reason for such a large relative error is the fact that it is drifting for quite some time, as we can determine from #51.

Here is a list of suspects with relative errors > 1.5:

                            logger    N altitude.m.bf RASTERVALU altitude.m.rel_err
 1:    barodata/BAOL015X_72527.csv 4420    -84.582470      28.29           4.495663
 2:    barodata/BAOL109X_P4024.csv 4477     78.818297      60.00           3.525451
 3:    barodata/BAOL004X_77561.csv 7667    -60.749429      35.72           3.467790
 4:    barodata/BAOL849X_A9771.csv  958     97.650572       4.45           3.068416 ***
 5:    barodata/BAOL029X_R6549.csv 3054    -72.348491      30.65           3.038848
 6:    barodata/BAOL019X_77560.csv 7425    -22.046760      70.25           2.936357
 7:    barodata/BAOL014X_R6519.csv 3478    -51.858903      25.82           2.395642
 8: barodata/BAOL845X_P2_15488.csv 2167    -80.559779      22.67           2.279869 ***
 9:    barodata/BAOL027X_77554.csv 6065    -10.962409      69.11           2.225562
10:    barodata/BAOL070X_B9393.csv  775     75.014820       6.59           2.220084 ***
11:    barodata/BAOL088X_B5548.csv  841     64.536759       7.39           2.068103 ***
12:    barodata/BAOL012X_D1095.csv 3355      7.058513       1.96           1.973851
13:    barodata/BAOL093X_78679.csv 4396    -11.125500      62.70           1.910527 ***
14:    barodata/BAOL064X_59976.csv 4967    -12.756265      59.96           1.867836
15:    barodata/BAOL054X_F6598.csv 2165    104.657061      94.64           1.847558
16:    barodata/BAOL071X_H4445.csv 1539     49.146993      22.42           1.794395 ***
17:    barodata/BAOL009X_78681.csv 4023    -37.837886      27.89           1.721346
18:    barodata/BAOL001X_D0939.csv 5391      7.197382      15.63           1.552928

I have placed an asterisk to the ones that do not seem to drift according to #51. Here they are:

image

image

image

image

image

image

So although these barometers do not drift, their median pressures seem to be way off from what we expect, so I would advise to double-check them.

Here is also the full dataset and a visualization of the relative errors:

image