Potential File Structure

[craigjmcgowan]

I've been thinking over the weekend about potential functions/files and how they can fit with the existing submission template and info needed to score the forecasts. Here's what I've come up with - @jarad please let me know what you think.

Functions:

• create_point: Take probabilistic forecasts in submission format and add point forecasts. Outputs data frame (to run directly into verify_entry) and CSV if requested.
• verify_entry: Verifies that submission is in the proper format. Creates errors for implausible submission values. Returns warnings for missing values but allows their submission.
• create_targets: Using ILINet data, creates seasonal and weekly targets. Can either fetch ILINet data directly or use user-supplied data (which will be needed after Week 28 to allow final scores w/o additional backfill after that date).
• format_score: Take submission CSV or data frame and add submission date for scoring. Extracts date from CSV metadata, or date is supplied if data in form of data frame.
• bin_score: Calculates log score for a submission using output files from format_score and create_targets.

Teams could run create_point and verify_entry before submitting their forecasts to CDC. The other three functions would be used in scoring the forecasts.

[jarad]

I agree that these functions all seem useful, although I think it will make sense to tweak the names. I'm starting off the package by creating an S4 class called Entry that will be an R entry object. S4 classes have a feature that allows you to specify a function that checks validity of the object, so that is where verify_entry will go.

[jarad]

The S4 class is gone.

[jarad]

Much work has been done to create a verify_entry function that is helpful. It isn't completed, but a lot of it is there.

Two functions exist to create probabilistic forecasts and are in the this file.