Zero effect on channels

[RomanRudnitskiy]

Project Robyn

Hello! When using Robin, there is always an effect that one or more channels show a zero effect.

Here is some information on this case :

• data with weekly granulation for 2 years
• rolling window = 1 year (2022-2023)
• paid_media_vars are equal to spends, because I cannot collect impressions for all the spend channels (it should work in this setting as well, right?)
• Report is being picked from the experiment with 5 trials, 2k iterations. Majority of reports (models) share same effect or huge intercept value
• Channel 5 (zero effect)
• No gaps in spends, no zero values
• Doing 40 trials and 500 iteration like recommended in another issue Share of effect as zero #160 doesn’t help
• Also tried experiment with 5 trials, 8k iterations - doesn’t help too

Interesting cases:

• If I remove (multiply by zero) spends in this channel in last 3 weeks (points) (1% of spends in this channel) the zero-share effect disappears
• zero-effect appears in experiments with different target metrics (revenue/conversion)
• I tried different approaches on ROI calculations (D1/W1/M1...)
• I noticed that zeroing occurs at the decompMedia stage
• Removing channel spend data from data set doesn’t remove zero-effect channel: once you remove a particular zero-effect channel this effect appears in another
• There were thoughts on L1 regularization, but following final formula there is L2 regularization. There still can be same problem, if spends for this channel are constant, but they are not. Spend std is even not the minimal among the dataset.

-

[laresbernardo]

Hi @RomanRudnitskiy thanks for sharing your analysis. We are about to deploy a new penalizing system when calculating DECOMP.RSSD in which models with more coef-0 media variables are penalized and models with no coef-0 media are not, that way we can minimize these scenarios.

[laresbernardo]

Closing this task now. Feel free to reach back anytime.