Closed craigjmcgowan closed 7 years ago
I thought point forecasts were being delivered/specified by the teams? I definitely think having something other than the expected value would be good. Theoretically, the median value is the choice that would minimize the mean absolute error, but that would be different from picking the mode, as suggested. Finding the estimated median would require finding the bin that, when the bins are ordered by week, tips the total cumulative probability over 0.5.
Yes, they are provided by the team - this function is just provided as a way for a team to generate the point forecasts if that team's model is only producing the probabilistic outcomes. Perhaps all three options (mean, median, and mode) could be provided and the user could simply pick which they prefer to use.
Seems fine, although I vote for default being median since we are using mean absolute error as a metric.
Nicholas G. Reich, Ph.D. Assistant Professor Department of Biostatistics and Epidemiology School of Public Health and Health Sciences University of Massachusetts, Amherst
413.545.4534 nick@schoolph.umass.edu http://reichlab.github.io/ on twitter: @reichlab On 2 Nov 2016, at 11:14, Craig McGowan wrote:
Yes, they are provided by the team - this function is just provided as a way for a team to generate the point forecasts if that team's model is only producing the probabilistic outcomes. Perhaps all three options (mean, median, and mode) could be provided and the user could simply pick which they prefer to use.
You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/jarad/FluSight/issues/16#issuecomment-257895358
Good call Nick. I wasn't thinking about the scoring metric when I coded this up.
Currently the point forecasts are generated from the expected value based on the probabilities assigned. Do we want to give teams another option to simply pick the individual bin with the highest probability? If they're getting very skewed probability curves from their models this method could be preferred and it shouldn't be too hard to add to the function.