search

jamalsenouci / causalimpact

Python port of CausalImpact R library
Apache License 2.0
262 stars 62 forks source link

Calculation error of p-value #49

Open CYW0018 opened 8 months ago

CYW0018 commented 8 months ago

I'm not sure whether there is an error in the calculation of p_value of the code. As the following figure shows, the p_value is calculated by the mean value of the synthetic control group's predictor rather than the point effect value. I guess you may be want to use the latter? image Another evidence is that when I use the example from https://colab.research.google.com/drive/1HkJ9zm0LY36Wz-wB_bSHq68w8Cef6qJO?usp=sharing#scrollTo=AqyItZ3Hggoh, even if I don't change the value of y after 3000, the p-value is still significant and inconsistent with the confidence interval given(including zero)

mare011rs commented 3 months ago

I also think there is something off with the p-value, as I'm seeing constant 100% confidence level.

patrick-le-shopify commented 2 months ago

I am seeing the same problem as @mare011rs : p_value = 0, Prob of causal effect 100% even when I explicitly synthesize data to have zero effect. This seems to be a rather big bug.

mare011rs commented 2 months ago

This package is outdated, I suggest you use tfcausalimpact.

patrick-le-shopify commented 2 months ago

Got it. Thanks!