Add example analysis of multiple geo lift test analysis

Closes #320
Closes #165
Adds a notebook to demonstrate 2 methods (pooled and unpooled) of analysing multi-cell geo lift tests.
Also adds code to generate a synthetic dataset.

Create new classes?

The PR currently does not add any additional code or classes. All the new functionality is embedded within the example notebook. It is relatively simple, we are just creating an aggregate treated region or iterating through each treated geo. So at the moment this uses an approach of transparency so that users can see what is being done etc.

However, part of the point of CausalPy is to provide a simple to use API, not requiring the user to do that much manual python coding. So the question is whether we should create new classes / an API. It would be something along the lines of:

For the pooled approach:

result = multi_cell_geo_test_aggregate(
    df, 
    agg_func=median,
    experiment=cp.pymc_experiments.SyntheticControl,
    expt_kwargs={"treatment_time": treatment_time,
                 "formula": formula},
    model=cp.pymc_models.WeightedSumFitter,
    model_kwargs={sample_kwargs: {"target_accept": 0.95, "random_seed": seed}},
)

For the unpooled approach:

results = multi_cell_geo_test_unpooled(
    df,
    treated_geos=treated,
    untreated_geos=untreated,
    experiment=cp.pymc_experiments.SyntheticControl,
    expt_kwargs={"treatment_time": treatment_time,
                 "formula": formula},
    model=cp.pymc_models.WeightedSumFitter,
    model_kwargs={sample_kwargs: {"target_accept": 0.95, "random_seed": seed}},
)

So the trade-off would be having a relatively clean API, but at the expense of making the operation a little more opaque. The manual python code in the notebook (as it stands right now) is not that complex. So I think it's not overwhelmingly obvious which we should go with.

TODO's based on feedback so far

[ ] ~~Check the pre-commit checks are applying ruff formatting to notebooks.~~ We'll do this in that in #340 and apply before the next release.
[x] We're currently getting negative sales in the synthetic dataset. So need to check the synthetic data generation code and potentially consider a weighted sum model operating on log outcome data.
[ ] ~~Fix the legends overlapping with plot content~~. We'll deal with that in #341 and run it on this notebook (and others) before the next release.
[x] Add a section which compares the results from the pooled and unpooled approaches. The similarity or difference will be dependent on the nature of the synthetic data of course - are we simulating with identical causal impacts in all test geos, or heterogeneous causal impacts in the test geos.
[ ] Think about using fixed effects approaches https://matheusfacure.github.io/python-causality-handbook/14-Panel-Data-and-Fixed-Effects.html. And/or thinking about "What if you de-mean the data (in time and in unit) as in the book and apply the synthetic control model instead of the classic linear regression" This is a great suggestion, but I think we are holding off and waiting until @juanitorduz investigates. We can certainly update this notebook in the future.

View / edit / reply to this conversation on ReviewNB

juanitorduz commented on 2024-05-08T12:57:25Z ----------------------------------------------------------------

nip: use " or ' for strings (not both) ... btw: do we have ruff for notebooks? :)

drbenvincent commented on 2024-05-08T19:57:06Z ----------------------------------------------------------------

Fixed. Created https://github.com/pymc-labs/CausalPy/issues/340 to double check ruff for notebooks.

review-notebook-app[bot] commented 1 month ago

View / edit / reply to this conversation on ReviewNB

juanitorduz commented on 2024-05-08T12:57:26Z ----------------------------------------------------------------

shall we use tab20 palette so that we do not reepear colors https://matplotlib.org/stable/users/explain/colors/colormaps.html?

I see (from the plot) negative sales? Is this expected?

drbenvincent commented on 2024-05-08T20:03:07Z ----------------------------------------------------------------

Applied the tab20 colormap.

Good spot with the negative sales. This is not expected. I'll add this to the todo list - I'll go back to the synthetic data generation and ponder whether we need to operate on log sales etc.

review-notebook-app[bot] commented 1 month ago

View / edit / reply to this conversation on ReviewNB

juanitorduz commented on 2024-05-08T12:57:27Z ----------------------------------------------------------------

Canw e take the legend outside the plots? they are hard to read

drbenvincent commented on 2024-05-08T20:06:20Z ----------------------------------------------------------------

Good point. I'll actually create an issue about this because this is an important plot type in CausalPy and we want a decent solution which will be useful in many contexts.

review-notebook-app[bot] commented 1 month ago

View / edit / reply to this conversation on ReviewNB

juanitorduz commented on 2024-05-08T12:57:28Z ----------------------------------------------------------------

i think is worth adding a section where we compare the results of the two methods.

drbenvincent commented on 2024-05-08T20:12:51Z ----------------------------------------------------------------

Agreed. Will add this to the todo list and update very soon

juanitorduz commented 1 month ago

@drbenvincent this is very interesting!

In practice, I have used a fixed effect model as described in https://matheusfacure.github.io/python-causality-handbook/14-Panel-Data-and-Fixed-Effects.html. In this case, we can use all the info from both groups without aggregating. It would be super interesting to compare the results of these approaches (I think the fixed effects model is very popular)

juanitorduz commented 1 month ago

Actually! What if you de-mean the data (in time and in unit) as in the book and apply the synthetic control model instead of the classic linear regression💡

This could be the better way 🤔