facebookexperimental / Robyn

Robyn is an experimental, AI/ML-powered and open sourced Marketing Mix Modeling (MMM) package from Meta Marketing Science. Our mission is to democratise modeling knowledge, inspire the industry through innovation, reduce human bias in the modeling process & build a strong open source marketing science community.
https://facebookexperimental.github.io/Robyn/
MIT License
1.1k stars 326 forks source link

Robyn Budget Allocator max_budget scenario should scale x_hist_carryover for accurate response simulation #1014

Open NumesSanguis opened 3 weeks ago

NumesSanguis commented 3 weeks ago

Project Robyn

Describe issue

Simulated Adstock effect is wrongly calculated in max_budget scenario. The Adstock effect is not changed to reflect this budget change when simulated with a different budget. If I understand the code correctly.

The function objective.channel in eval_f, which calls fx_objective.chanel calculates Adstock with: xAdstocked <- x + mean(x_hist_carryover). is used to calculate the response:

optmResponseUnit <- -eval_f(optmSpendUnit)[["objective.channel"]]

https://github.com/facebookexperimental/Robyn/blob/main/R/R/allocator.R#L516

Due to using the mean, the number of weeks simulated does not matter. However, if the budget gets very small, the mean(x_hist_carryover)part is much bigger thanx`, meaning the whole simulated response is dominated by historical carryover at a different budget level.

Possible solution

Get the ratio: "mean historical spend" / "weekly budget" and multiply with the Adstock effect to scale the effect. However, I'm not sure if this is accurate. It might be that the Adstock effect is different at different spend levels though.

Environment & Robyn version

NumesSanguis commented 3 weeks ago

Sugested solution

Instead of using average historical Adstock effect:

xAdstocked <- x + mean(x_hist_carryover)

Use the carryover response ratio as shown in the "Immediate vs. Carryover Response Percentage". If e.g. carry over is 30% for a channel, then for that channel:

xAdstocked <- x * (carry_over_rate + 1)  # x * 1.3

The immediate response triggers in that week (assuming weekly data), e.g. 70%. The remaining 30% over the following x weeks. If we assume for the simulator that previous weeks (before week 1 of the simulation) has been on the same spend level, then the sum of all previous week is equal to the 30%, simplifying it to just x * 1.3.

This way changing the total budget in the simulator will be using the correct Adstock strength rather than historical (which can be at a different spend level)

For this, you only need to capture an array with a % value of carryover for each media channel and pass this along when calling the funcs like fx_objective.