harphub / harpIO

IO functions for HARP
https://harphub.github.io/harpIO/
Other
6 stars 17 forks source link

Stricter date selection in read_point_forecast #39

Closed andrew-MET closed 1 year ago

andrew-MET commented 4 years ago

read_fctable, that is called by read_point_forecast uses:

fcst <- dplyr::tbl(fcst_db, "FC") %>%
  dplyr::filter(dplyr::between(.data$fcdate, start_date, end_date))

to select the forecast initialization times from an sqlite file. In general this is fine, but if, for example, multiple initialization hours are in a file (not the default behaviour, but is possible) this will result in reading more rows than are requested and could invisibly give incorrect verification statistics.

It would be better if all required dates are used, e.g.

fcst <- dplyr::tbl(fcst_db, "FC") %>%
  dplyr::filter(.data$fcdate %in% required_dates)

Care will need to be taken to select the correct set of dates for lagged ensembles.

In addition, it should be made possible to give read_point_forecast a vector of dates instead of start_date, end_date, by thus enabling extra flexibility.