reichlab / covidHubUtils

Utility functions for the COVID-19 forecast hub
http://reichlab.io/covidHubUtils/
17 stars 15 forks source link

new function to check whether a given forecast is "complete" #180

Open nickreich opened 3 years ago

nickreich commented 3 years ago

Use case would be for evaluation purposes, to identify whether a given forecast contains all desired quantiles, locations, targets. Skeleton might look like this:

input: a single forecast in standard covidHubUtils format (could be adapted for multiple forecasts?) output: logical, TRUE if forecast contains all Hub-required quantiles for listed targets and locations.

is_forecast_complete(fcast_data, targets, location_codes)
elray1 commented 3 years ago

Couple of questions:

nikosbosse commented 3 years ago

There is some functionality that does this in scoringutils that might be helpful for this. The function show_avail_forecasts() does essentially this. If we don't want it as a plot we could just pull out the relevant bits of the code.

Currently this treats all quantiles together as one forecast, but you can also count the number of quantiles specifying collapse_to_one = FALSE image