Tune our smoothing kernel and deduce confidence bounds

[GaelVaroquaux]

The smoothing kernel used in modeling.py was chosen in purely adhoc way. We should tune it. Proposal:

• Create a family of candidate smoothing kernel: varying windows in reasonable ways:
  • The ramp + flat approach currently used where the length of the ramp and the total
  • An exponential kernel (two parameters here, I believe)
  • More?
• Use a historical replay on the data to find the expected error associated to a weighted least square with each kernel. In terms of measure of error, I would use a relative one, such as the Mean absolute percentage error.
• Keep a reasonable number of kernels, that give good performance on the metric above and use an ensembling strategy
• Report their median (our prediction line), and their 25% / 75% prediction, to give error bars (ie multiple senarios)

[GaelVaroquaux]

I'm considering doing this one, as I think that error bars are important.

[GaelVaroquaux]

While doing #30, I found that the errors of multiple kernels where often autocorrelated. Hence the original idea of ensembling to get confidence intervals will not work.

I will fall down to using the parameter confidence bounds given by the weighted least square.

[GaelVaroquaux]

Closed in 95fce13