GEUS-Glaciology-and-Climate / PROMICE-AWS-data-issues

PROMICE AWS QA/QC
http://promice.org
5 stars 0 forks source link

Flagging bad pressure data at GC-Net stations #140

Closed ladsmund closed 6 months ago

ladsmund commented 8 months ago

Note

The stations : NAE, CEN2, HUM, NAU, NEM, NSE, SDM, TUN are having issue with their pressure data beginning in December 2023.

What is the issues?

DMI has reported issue in the pressure data from our BUFR files. I noticed periodes with erroneous data and missing data in both our L0 datasets and L3 data products. The following figures shows the pressure data fro the stations in the period from 2023-11-01 to 2024-02-16.

SDM NSE NEM NAU HUM CEN2 NAE TUN

ladsmund commented 8 months ago

The error seems to be related to the low temperature issues with our luft sensor. It is also the case in April 2023 for TUN

image
ladsmund commented 8 months ago

There is something suspicious happening with the CEN2 L3 data in addition to the pressure data. It looks like there is a lower temperature threshold on -35C from midt December. The measurements are available as L0.

image
ladsmund commented 8 months ago

I looks like there is a lower pressure threshold at 709hPa.

image
ladsmund commented 8 months ago

I looks like there is a lower pressure threshold at 709hPa.

image

It is because of the percentile qc

PennyHow commented 8 months ago

If this is related to the lufft sensor, similar to the issue with temperature, then we should make an adjustment in pypromice L0toL1 (like in this PR) and/or alter the variable thresholds. Good catch!

BaptisteVandecrux commented 8 months ago

Hi Mads, I usually leave the issue open until the problem is also fixed on the dataverse, where most users get the data. @PennyHow is the monthly upload to dataverse reactivated?

PennyHow commented 8 months ago

It's not been reactivated yet, but I can just manually trigger it whenever it is needed. Next release is scheduled for beginning of March, but we can make exceptions if there are any urgent data updates.

BaptisteVandecrux commented 8 months ago

I am flagging additional pressure outliers at HUM for 2022-23: billede

BaptisteVandecrux commented 7 months ago

new flags on p_u p_l at NEM image image

No idea why p_i was not available

and NSE image

BaptisteVandecrux commented 6 months ago

Filtered out and released in https://dataverse.geus.dk/dataset.xhtml?persistentId=doi:10.22008/FK2/IW73UU