How to improve accuracy of daily forecast

[JoeFernando]

Thanks for giving us Prophet - love it.

When using daily revenues to forecast: 1) At a monthly level the forecast gets very close to actual - variance is within tolerance levels. 2) However on a daily basis the numbers are a bit off - mainly due to week-ends, as shown below:

DoW Actual Forecast VAR Mon 390,425 365,526 24,899 Tue 392,374 385,497 6,877 Wed 386,858 385,283 1,575 Thu 383,510 377,844 5,666 Fri 340,352 338,025 2,327 Sat 27,020 60,249 -33,228 Sun 10,433 53,274 -42,841

(paste the table in a worksheet - it will look a lot better)

I want to be able to tweak the model so that the weekends don't show as much revenue. How can I improve my forecast accuracy?

Please help.

Thanks in advance.

Joe

[ppstacy]

Hi Joe,

I met with the similar situation with you that the weekend number is way lower than the weekday. So I used the changed the weekly seasonality from AUTO to true and adjusted the seasonality.prior.scale to fit.

seasonality.prior.scale = 10,weekly.seasonality = TRUE

Hope this helps and let me know if you have better solutions.

Thanks, Stacy

[JoeFernando]

Thanks Stacy.

Per your note I played around with the setting, the best that worked for me were the following:

yearly.seasonality = TRUE, weekly.seasonality = TRUE, changepoint.prior.scale = 0.55 # default value is 0.05

The weekend still have issues but the full month number is unbelievably close to actual...so far variance is about +/-2.3%.

I really would like to keep this thread open, lets update this thread as we learn more of the model.

Kind regards, Joe

[TWBTWB]

Hey all,

I'm having almost the exact same problem! I have a series that I get extremely close predictions on weekdays, but weekends are +/- nearly 30% and I can't find any way to get them under control.

For an example, one set I used has an average Saturday value of 2,517 with the lowest number in three years being 1,556. Still, almost every Saturday is predicted in the ~800 range. My StdDev on Saturdays is ~400, so its almost like its predicting the lowest possible outcome.

The data naturally has a few NA values ( 8 over 3 years). Four of those fall on a Saturday. If I replace those values with an average of the leading and following Saturday and run the forecast again, then its comes out with ~1100 (which is still low).

The final 'spaghetti against the wall' try was running a series of only Saturday and Sunday values. This much, much more accurate with a prediction of 1,600 on an actual of 1,701.

Anyways, just wanted to report that I'm having the same issue. If I find a cause and fix I'll report back!

[ppstacy]

Hi Joe,

Glad to hear that full month number is close to actual. I think you also can play with the seasonality.prior.scale = 10 which can help adjust there.

Thanks, Jing

On Thu, May 25, 2017 at 5:36 PM, Joseph Fernando notifications@github.com wrote:

Thanks Stacy.

Per your note I played around with the setting, the best that worked for me were the following:

yearly.seasonality = TRUE, weekly.seasonality = TRUE, changepoint.prior.scale = 0.55 # default value is 0.05

The weekend still have issues but the full month number is unbelievably close to actual...so far variance is about +/-2.3%.

I really would like to keep this thread open, lets update this thread as we learn more of the model.

Kind regards, Joe

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/facebookincubator/prophet/issues/204#issuecomment-304160382, or mute the thread https://github.com/notifications/unsubscribe-auth/AS564E4bQVSVdJGZmKhbpcaxniL1ct_Aks5r9h6XgaJpZM4Nl4sz .

[bletham]

@ppstacy's recommendation seems like the right way to go to me also. This will allow the weekly seasonality to have more flexibility and provide a better fit. The only downside is that this will also increase the flexiblity of the yearly seasonality also; being able to set these independently is on the roadmap.

If anyone having this issue has data that they can share, I can take a closer look.

@JoeFernando increasing the changepoint.prior.scale gives the model more flexibility in including trend changepoints. This will lower fit error but if increased too much could lead to overfitting. Definitely look at the 'trend' part of the components plot and see how it looks.

[fanlu]

@bletham @ppstacy I have also encountered this problem, my data is on a monthly basis. There is a trend in the Chinese New Year's month is a fixed amount of small, and the overall trend is growing, and May and the month before New Year are relatively high peak, the business is more concerned about the peak is accurate or not, Is there any advice? 10 is not the default value of parameter seasonality.prior.scale? the data is below 2014-12-01 204877.0 2015-01-01 334144.0 2015-02-01 192404.0 2015-03-01 285759.0 2015-04-01 462487.0 2015-05-01 627196.0 2015-06-01 405835.0 2015-07-01 297060.0 2015-08-01 427951.0 2015-09-01 536284.0 2015-10-01 777104.0 2015-11-01 507941.0 2015-12-01 743897.0 2016-01-01 1022927.0 2016-02-01 285230.0 2016-03-01 645085.0 2016-04-01 624758.0 2016-05-01 1020629.0 2016-06-01 598457.0 2016-07-01 474958.0 2016-08-01 660721.0 2016-09-01 834091.0 2016-10-01 1156174.0 2016-11-01 803698.0 2016-12-01 1198490.0 2017-01-01 227078.0 2017-02-01 654797.0 2017-03-01 1094247.0 2017-04-01 994043.0 2017-05-01 1509607.0

[bletham]

@fanlu I don't have any insight into if that particular peak is accurate, Prophet will predict something like an average of the effect from the previous years. With this little data, I would recommend using MCMC sampling to get better uncertainty estimates. It will be cheap with this few data points. And yes 10 is the default for the seasonality prior scale. Given that it is monthly data and not especially long, you might want to crank that down, e.g. to 0.1. (In python:

m = Prophet(seasonality_prior_scale=0.1, mcmc_samples=1000)

).

[fanlu]

@bletham thanks for your reply How to set up changepoint_prior_scale and holidays_prior_scale with this few monthly data? 0.1 to 1 or 1 to 10 or 10 to 100 and why?

[maskani-moh]

@fanlu I think you should try different parameters and visually inspect the forecast.

Keep the parameters that give the best forecast according to you. That's what @bletham and @seanjtaylor call the analyst-int-the-loop modeling (cf. Figure 1 in their paper Forecasting at Scale).

As for the parameters themselves and what they mean, take a look at the documentation here

[fanlu]

thanks @maskani-moh

[kamnaroy2319]

Hi,I am using prophet for seasonailty of 120 days and weekly seasonailty,what parameters i should use?