StateTL - QC of hourly telemetry data using daily 'improved' station data

[kelleythompson]

In StateTL, observed / uncorrected hourly telemetry is used throughout much of the system as it is the only hourly data available through REST services. Error and noise in the hourly telemetry data causes oddities in total river flows - but (although the reasons for this are not fully understood) this is causing some error and noise in evaluations of individual releases using j349. See the following figure (2018 WC 151415) which shows the release in red and modeled downstream hydrographs (blue-multiple urf / orange - single urf)

DWR hydrographers (and water commissioners?) often correct the uncorrected hourly telemetry. Traditionally this data was corrected and 'published' for the water year several months after the end of the water year, but data may be updated more regularly. This corrected dataset is available through the 'surfacewater' REST service, but only daily data (not hourly) is available. This improved daily station data is therefore valuable to QC or adjust the hourly data - this may be quite valuable for historical time periods and evaluations (such as calibration) but may not add much to current time period runs until observed data can be 'improved'.

[kelleythompson]

QC of hourly telemetry data with daily 'improved' station data was implemented. Two threshold values are used (currently set at 0.05 and 0.15), if the percent difference between mean hourly and daily is less than first threshold, do nothing. If equal or above first threshold, adjust hourly values so that the daily mean equals the daily value. If equal or above the second threshold, replace hourly with daily data (will go in 24hr steps). If hourly data is missing, it might be replaced by a daily value. If the daily data is missing but there is hourly data, it will insert a missing code as long as the daily data is not at the end of its dataset (for near current time periods there can be hourly data but last daily data is from previous day). Data declared as missing will be subsequently filled by the filling routine.

The following figure shows resolution of the error introduced into a water class release hydrograph. The figures after that show the QC in the primary gages for WD17 as well as the Catlin Canal.