DOV-Vlaanderen / groundwater-logger-validation

Analysis on validation methods for groundwater logger data
MIT License
2 stars 2 forks source link

v0.05: AO and LS detector based on apriori differences in time #34

Closed DavorJ closed 5 years ago

DavorJ commented 5 years ago

v0.05 extends v0.04 with level shift (LS) detection, and also fully makes use of apriori hydrostatic pressure information. The latter allows for dynamic threshold determination based on any specified time-interval.

Results: inbo and geotech

A comparison with tsoutliers: geotech and inbo.

Some examples.

image

A timeseries always starts with a green color. Levelshifts are represented with colors: red if the LS is up, and blue if it is down. The start and endpoints are represented with dotted vertical lines. Outliers are represented with red dots.

image

Here, beside the LS, there are also 2 outliers.

For inbo data, varying time-differences play a major role:

image

In the last LS, the timeinterval is a couple of months, and based on apriori information, the difference is deemed a LS, whereas here it is not:

image

An other clear example:

image

There are other cases which are falsely detected as LS. Some are TC, and since v0,05 has no degrees of freedom for TC, they are falsely considered LS. Others seem borderline cases or bugs and still need to be analysed.

Some advantages vs. tsoutliers:

Disadvantages:

For modeling and optimization, v0,05 is largely inspired by Chen, C., e.a., Joint Estimation of Model Parameters and Outlier Effects in Time Series, 1993.