aodn / python-aodntools

Repository for templates and code relating to generating standard NetCDF files for the Australia Ocean Data Network
GNU Lesser General Public License v3.0
10 stars 3 forks source link

Product codes assume IMOS QC flag scheme but don't check for it #126

Closed mhidas closed 4 years ago

mhidas commented 4 years ago

The the aggregated product includes QC flags in the output file, while the hourly product excludes data values not flagged as "good" or "prodbably good". Both of these make the implicit assumption that the input file uses the "IMOS standard flags".

More specifically,

The assumption is correct for the vast majority of IMOS moorings data, except for the Acidification Moorings, which use the WOCE QC flags:

flag_values = 2b, 3b, 4b ;
flag_meanings = "good questionable bad" ;

Luckily this is sort of compatible, in that selecting only flags "1 or 2" gets us only the good data. However, for the aggregated product, re-labelling these flags as if they were from the IMOS scheme is misleading. In effect all the "good" data (flag 2) will now look like it's only "probably_good_data".