jarad / FluSight

An R package containing functions used in the CDC Flu Forecasting competition
GNU General Public License v3.0
12 stars 7 forks source link

Fix score_entry function #13

Open jarad opened 7 years ago

jarad commented 7 years ago

The current version of the score_entry is simple and works if the scores did not include any additional bins. Due to the guidelines, these additional requirements need to be met

craigjmcgowan commented 7 years ago

Rather than try to expand the truth, perhaps it would be easier to have a function that expands our definition of the "correct" cells on the entry data frame after joining entry with truth. This is what we did in the past and the weeks are already in the correct order on the submission.

We also need a way to take into account the weekly submissions of the 1-4 wk ahead forecasts that will have different truths for each submission.

craigjmcgowan commented 7 years ago

Perhaps it would be easier to index submissions by MMWR week they are based on, rather than submission date. That way if submission date changes due to holidays any functions will be unaffected, as the forecasts will still be based on the same MMWR week.

craigjmcgowan commented 7 years ago

We also need to build in capacity for multiple peak weeks - this happened this past year in HHS Region 8 where both weeks 8 and 11 had the same peak ILINet values.

craigjmcgowan commented 7 years ago

I believe I made score_entry fully functional and also completed the expand_truth function. expand_truth is a little slow due to the loops but when I ran it on a full season's worth of observed data it still only took 35 seconds. I still need to write tests for these - I will probably add some "correct" data frames to the data folder to use in those tests.