Missing value check removes valid values

AMF-FLX / AMF-BASE-QAQC

AmeriFlux BASE Flux/Met Data QA/QC and Processing

Other

5 stars 7 forks source link

Open dschristianson opened 1 year ago

dschristianson commented 1 year ago

The missing value format check removed valid values.

Examples:

QAQC-1920: Values of -669.96 for MO-LENGTH were removed.
QAQC-4093: 1 instance of -996.6; 1 instance of -999.6; 7 instances of -696; 4 instances of -666; 6 instances of -699; 4 instances of -669; 3 instances of -966; 5 instances of -969; 1 instance of -999; and 2 instances of -999.9. These are in WTD.

The check needs to be tightened.

See DataUtils in the utils module for the regex code.

Related to original issue 271.

dschristianson commented 1 year ago

Consider adding fixes for other commonly found missing values, like -10000, -999900, and/or -999.9

We may want to make the -10000 variable-specific, since ZL and TAU(?) can have very negative numbers.

Perhaps just a warning for -999.99

Related to original issue 323