quince-science / KPI

Data quality reports for data processed in QuinCe
MIT License
0 stars 2 forks source link

Stacked bar plot shows misleading number of nan's #27

Open cla023 opened 4 years ago

cla023 commented 4 years ago

See figure below showing distribution of QC flags for the measured SST, salinity, and xCO2, and for the calculate fCO2. It looks strange that the total number of nan's for the measured values are much lower than the number of nan's for the calculated fCO2 (calcuated based on the other 3 parameters).

The reason for this miss-match is related to how QuinCe assigns QC flags. When the data column conatins value 'NaN', the related QC flag column is either 'NaN', or flagged as 4 (with QC comment: Missing). Steve's comment about this difference: "I think it's from when the machine is running in different modes. When it's doing gas standards, then we aren't recording the CO2 or H2O because they're not valid measurements. So the columns are NaN and the fact they aren't there is not an error so the flag is also NaN (no data to QC). Meanwhile we can always keep measuring SST, so the fact that it's not there is an issue, so it gets a missing flag. We could consider using 9 (not QCed), but that implies that there was a value to QC which could be misleading. So I think for now assume that you might see a NaN QC flag, which means that there was nothing to be QCed."

Suggested action:

overview_bar_plot