Short time series

[ET-NCMP]

There are assumptions built into the station calculations (P2_Indices) that all stations have all 30 years of the user-defined reference period. The implications for the NCMP indices are:

• NCMP 1 and NCMP 2: As written, the code checks that there are fewer than 7 missing years within the user-defined reference period. So a single valid year in the reference period allows estimation of the “climatology”. However, flipping the code to not calculate the climatology if less than 80% of years are valid would exclude stations with as much as 23 years of valid data. The implication of not estimating a climatological mean is that the station will be ignored when estimating the variogram (P3_Variogram) or spatial average (P4_Regional_Average). To account for late-starting stations and missing data, ACORN-SAT climatologies are estimated from a minimum of 15 years of data. CRUTEM calculates short-period station climatologies, and inflates the estimated uncertainty when aggregating the station data. The WMO 1989 Technical Document considered defining a decadal mean as “provisional” climate normal; this might be a suitable compromise here.
• NCMP 3: Not directly affected by short time series, but as written the code estimates the Gamma distribution parameters by sampling for all years in the time series. Is this the correct procedure for the SPI?
• NCMP 4 and NCMP 5: The daily warm and cold percentiles are estimated in the “climdex.pcic” package, an implementation of the ETCCDI indices by the PCIC that is openly available (as opposed to the need to register to obtain the R code written for the ETCCDI). Although it notes a bias with missing years from the reference period, it attempts to allow for this when estimating the in-period percentiles (as described by Zhang et al. 2005). However, when testing the code, warnings were generated which were traced to the underlying function in the “climdex.pcic” package, for any station with less than 30 years of data in the reference period. Although percentiles were calculated, the nature of the warning and inspection of the results suggested that these were erroneous. The function has been modified (external to the “climdex.pcic” package) to eliminate the warnings, but PCIC should be consulted as to whether this patch generates appropriate in-period percentiles.
• NCMP 6: The setting of records is only done after the first 30 years of station data. A simple work-around here is to add one record for the first 30 years.

[ET-NCMP]

The code, fails if stations have nothing but missing data for a variable (e.g. all precipitation values for all dates are missing) with the following error message:

Error in check.var(tmax, tmax.dates, "tmax") : tmax must be of type numeric.

Error in check.var(prec, prec.dates, "prec") : prec must be of type numeric.

The code also fails if a station record ends (or starts, I think) during the climatology period with the following error message:

Error in validObject(.Object) : invalid class "climdexInput" object: 1: Quantiles for tmax are missing. invalid class "climdexInput" object: 2: Quantiles for tmin are missing. invalid class "climdexInput" object: 3: Quantiles for prec are missing.

It looks like we need to add some checks to the QC script to ensure that stations have a complete record through the climatology period and flag those that don't.

AND

Add a check to the index calculation (and subsequent calculations) to ensure that stations which are missing all their data don't cause an error. We still want some of these stations because we can still use stations that have just precip but no temperature data.