search

google / lightweight_mmm

LightweightMMM 🦇 is a lightweight Bayesian Marketing Mix Modeling (MMM) library that allows users to easily train MMMs and obtain channel attribution information.
https://lightweight-mmm.readthedocs.io/en/latest/index.html
Apache License 2.0
902 stars 194 forks source link

how different it is from Robyn #200

Open Sandy4321 opened 1 year ago

Sandy4321 commented 1 year ago

how different it is from Robyn https://github.com/facebookexperimental/Robyn

uomodellamansarda commented 1 year ago

If someone thinks I am wrong please feel free to correct me.

0) Lightweight MMM Is written in Python and not in R.
One is an unofficial Google product the other one is an official Meta product.

1) In Robyn there is a Ridge Regression, the Google Lightweight is a Generalized Linear Regression. In Robyn you can't get statistical significance of coefficients, with the Lighweight you can. Here on this Stack Question you can find why

This is what you have in Robyn: image

2) In Robyn Adstock-Carryover Effects are modelled trough a Weibull Distribution or a Convergent Geometric Series, while in Lightweight MMM you have or a Convolution or a Geometric Series.

On this point I am still concerned on having three different ways in Lightweight MMM to model adstock-carryover effects, because it seems in that way that are three different things. Instead is the same phenomenon just treated differently from a mathematical perspective. This can be misleading for a junior analyst.

From github Lightweight repo: image

4) For both libraries I feel a strong lack of the impact of price elasticity in the sales attribution and consequently price promotion. Don't get me wrong, you can model them, but both documentations doesn't stress this point properly. I noticed that mistake with a lot of modeller in the Robyn user group. From Lightweight MMM image

5) Both uses the hill function for diminishing returns, a quite complex formula that was used in Jun, Shobowale, Koehler and Case (2012) to describe the relationship between unique reach and Gross Rating Points or TV Impressions. I am pretty sure Robyn guys studied the Google paper "Challenges And Opportunities In Media Mix Modelling" or the other Google Paper both co-authored by David Chan.

In general the goal is to understand when the Ad spend is no more efficient. If you show something like that to a client he will understand better. The hill function assumes also the possibility of zero effects from the media activity, which is something I am still skeptical.

image

This are some of the key aspects from a modeller perspective, didn't have the time to investigate how on the same dataset how different results can be based on the optimization engine in the back-end.

Hope it helps and there are no big mistakes from my side.

uomodellamansarda commented 1 year ago

Just an additional note.

At the moment you can run Lightweight MMM on Colab, but the same is not possible for Robyn, but you can still run Robyn on Vertex AI