facebookincubator / GeoLift

GeoLift is an end-to-end geo-experimental methodology based on Synthetic Control Methods used to measure the true incremental effect (Lift) of ad campaign.
https://facebookincubator.github.io/GeoLift/
MIT License
181 stars 55 forks source link

Is Seasonality being accounted for? #149

Closed Snowcatcat closed 1 year ago

Snowcatcat commented 1 year ago

Hi @ArturoEsquerra,

I have two questions hope that you can help with:

  1. Is Seasonality being accounted for in the package? If so, is it done by using the prophet engine?

  2. How are the counterfactual values calculated?
    a. Does it exclusively consider the pre-treatment values of the control and test groups? b. Or does it consider both the pre-treat and post-treatment of values of the control and test groups? c. or does it only consider the pre and post-treatment values of the control group?

Thanks

raphaeltamaki commented 1 year ago

Hi,

  1. Seasonality isn't directly modelled in the package. However, that is not to say that seasonality isn't accounted for, since the weights used to predict the values on Test are based on the values on Control locations on a daily basis. So if the ideal weights to predict Test using Control data are the same throughout time, then there is no advantage in modelling seasonality. However if the weights vary significantly depending on seasonality (ex: weekend vs weekday), then the predictions won't be ideal. However, if that do happen, then the control location are not ideal, because there are some hidden cofounder that is affecting the behaviour on the ideal weights depending on the seasonality that may very well affect the estimated effect itself (ex: on the weekday vs work day, it may be that the test location has more poor families that work long hours, which would be more sensitive to price changes on a given experiement)

  2. The counterfactual value uses pre- and post-treatment data from control and pre-tretament data from test