search

facebookexperimental / Robyn

Robyn is an experimental, AI/ML-powered and open sourced Marketing Mix Modeling (MMM) package from Meta Marketing Science. Our mission is to democratise modeling knowledge, inspire the industry through innovation, reduce human bias in the modeling process & build a strong open source marketing science community.
https://facebookexperimental.github.io/Robyn/
MIT License
1.07k stars 323 forks source link

Model Refresh #990

Open jainnitk opened 3 weeks ago

jainnitk commented 3 weeks ago

Hi, I am struggling to understand how the Refresh module works. As per my understanding and documentation, the model refresh takes a base model, updates the range for hyperparameters from the base model and then updates the model with following criteria: 1) Stable baseline compared to the the base model 2) Media spend changes in the new period are reflected in the refreshed model which is the objective

However, I am unable to wrap my head around on how the above criteria is being handled in the code (pardon my R illiteracy). Further, with only a few data points added to refresh the model, how will the coefficients change to reflect the changes happening in new period added as the weight would be too less for a few data points.

Do we make some changes in the objective function to achieve the desired results. If yes, which part of the code handles this?

Please help here so that I can understand how the refresh module works to reflect the changes in the new period.

jainnitk commented 3 weeks ago

Found this in the Analyst's guide: "Robyn then sets the lower and upper bounds of hyperparameters for the new build, which will be in line with the selected hyperparameters of the initial build. This will therefore stabilise the effect of contextual and organic variables across initial and new builds as well as regulating the new share of effect of media variables towards the new added period spend level."

This seems to indicate that setting narrower bounds for refresh model hyperparameters automatically takes care of new share of effect of media variables to skew towards new data. Not sure how does that happen. Further, the following snippet says: "Robyn uses modified objective functions to achieve the above. To be precise, the NRMSE only accounts for the new period while refreshing, because the main goal of refreshing is to better describe the new period. The DECOMP.RSSD is modified to drive only effect share closer to the refresh spend share, while it also accounts for the similarity between new and old baseline."

Could you please help me what all measures are taken to achieve the above mentioned objectives and also which part of the code modifies the NRMSE and DECOMP.RSSD for refresh.

Thanks in advance !!!