raphaeltamaki
commented
4 months ago Hi @niramay447,
In this case, am I right to assume that ,in the Control group, you will be turning off all existing campaigns? If that is the case, then if you turn these campaigns off immediately before the start of the test, there will still be some carry over effect of the campaigns to the number of sales in the control group. This carry over effect will decay over-time, but will nudge toward under-estimating effect of the treatment (i.e. the marketing campaign). Be aware that this effect of marketing campaings having a long-lasting effect on Sales also affects the Test group, which might cause the first days of the test to have lower number of sales than the final days of the test.
My suggestion here is to ideally discard the very first days of the test (1~4 days). The best number of days, and whether it is actually needed, will depend on the type of product that you are advertising. Products with long purchase cycles (e.g. cars) will require more days to be discarded versus products with short-purchase cycles (e.g. mobile games).
My other suggestion is to have a relatively longer treatment period, so that the carry-over effect on the total estimated lift will diluted.
when we are creating the test control, how do we capture and/or exclude the impact of past campaigns(not geo tests) that might have run in some of the markets.