search

glamod / glamod_landQC

QC for land data
0 stars 0 forks source link

Check Germany stations for Distribution, World Records #120

Closed rjhd2 closed 2 months ago

rjhd2 commented 3 months ago

Relatively high flagging rates over Germany for these tests (and also others). Investigate to see about underlying cause, and if necessary pass up to mingle/IFF-conversion

rjhd2 commented 3 months ago

There are ~1400 stations for Germany alone, so 5% of total release (30k), which may be part of it.

rjhd2 commented 2 months ago

Testing on following stations:

GMA00108450 44 (T) GMA00109210 10 (T) GMI0000EDAJ 85 (T) GMI0000EDAP 268971 (SLP) GMI0000EDBH ~25 (T, D, SLP) GMI0000EDCA 119 (D, T) GMI0000EDCB 15 (T, D, SLP)

First station (GMA00108450) looks to be legitimate flags due to remaining Fahrenheit values (51F = 10.6C which is more in keeping with prior values):

GMA00108450|LEIPHEIM(GAFB)|1993|09|29|07|20|48.4330|10.2330|489.0|10.0||1|FM15-METAR|
GMA00108450|LEIPHEIM(GAFB)|1993|09|29|08|20|48.4330|10.2330|489.0|10.0||1|FM15-METAR|
GMA00108450|LEIPHEIM(GAFB)|1993|09|29|09|20|48.4330|10.2330|489.0|10.0||1|FM15-METAR|
GMA00108450|LEIPHEIM(GAFB)|1993|09|29|09|45|48.4330|10.2330|489.0|10.0||1|FM15-METAR|
GMA00108450|LEIPHEIM(GAFB)|1993|09|29|10|00|48.4330|10.2330|489.0|51.0||2|FM16-SPECI|
GMA00108450|LEIPHEIM(GAFB)|1993|09|29|10|20|48.4330|10.2330|489.0|51.0||1|FM15-METAR|
GMA00108450|LEIPHEIM(GAFB)|1993|09|29|11|20|48.4330|10.2330|489.0|20.0||2|FM15-METAR|
GMA00108450|LEIPHEIM(GAFB)|1993|09|29|12|20|48.4330|10.2330|489.0|10.0||1|FM15-METAR|
GMA00108450|LEIPHEIM(GAFB)|1993|09|29|13|20|48.4330|10.2330|489.0|10.0||1|FM15-METAR|
GMA00108450|LEIPHEIM(GAFB)|1993|09|29|13|27|48.4330|10.2330|489.0|10.0||1|FM16-SPECI|
GMA00108450|LEIPHEIM(GAFB)|1993|09|29|14|20|48.4330|10.2330|489.0|11.0||1|FM15-METAR|
GMA00108450|LEIPHEIM(GAFB)|1993|09|29|15|20|48.4330|10.2330|489.0|11.0||1|FM15-METAR|
GMA00108450|LEIPHEIM(GAFB)|1993|09|29|15|50|48.4330|10.2330|489.0|11.0||1|FM15-METAR|
GMA00108450|LEIPHEIM(GAFB)|1993|09|30|04|20|48.4330|10.2330|489.0|47.0||1|FM15-METAR|
GMA00108450|LEIPHEIM(GAFB)|1993|09|30|05|20|48.4330|10.2330|489.0|45.0||1|FM15-METAR|
GMA00108450|LEIPHEIM(GAFB)|1993|09|30|06|20|48.4330|10.2330|489.0|43.0||1|FM15-METAR|
GMA00108450|LEIPHEIM(GAFB)|1993|09|30|06|30|48.4330|10.2330|489.0|43.0||1|FM15-METAR|
GMA00108450|LEIPHEIM(GAFB)|1993|09|30|06|45|48.4330|10.2330|489.0|43.0||1|FM15-METAR|
GMA00108450|LEIPHEIM(GAFB)|1993|09|30|07|20|48.4330|10.2330|489.0||||||||||||||||||||
GMA00108450|LEIPHEIM(GAFB)|1993|09|30|08|20|48.4330|10.2330|489.0|47.0||1|FM15-METAR|
GMA00108450|LEIPHEIM(GAFB)|1993|09|30|09|20|48.4330|10.2330|489.0|28.0||2|FM15-METAR|
GMA00108450|LEIPHEIM(GAFB)|1993|09|30|10|00|48.4330|10.2330|489.0|10.0||2|FM16-SPECI|
GMA00108450|LEIPHEIM(GAFB)|1993|09|30|10|20|48.4330|10.2330|489.0|10.0||1|FM15-METAR|
GMA00108450|LEIPHEIM(GAFB)|1993|09|30|10|35|48.4330|10.2330|489.0|10.0||1|FM16-SPECI|
rjhd2 commented 2 months ago

GMI0000EDAJ : all values that fail test are -999.0 so it's poor missing value handling in MFF process for temperature GMI0000EDAP: same (-999.0) but in SLP field GMI0000ECBH: same but in T, D and SLP fields GMI0000EDCA: same but for T and D fields

From these tested files so far, it looks as if there's been some error in the missing data handling for these ICAO stations. Also the case in other German stations tested. Something for the mingle/merge code to handle as these should just be empty.

rjhd2 commented 2 months ago

Although only tested against world_record.py the -999.0 values will also be picked up by the distribution check and so this change should solve these issues.