search

py-why / dowhy

DoWhy is a Python library for causal inference that supports explicit modeling and testing of causal assumptions. DoWhy is based on a unified language for causal inference, combining causal graphical models and potential outcomes frameworks.
https://www.pywhy.org/dowhy
MIT License
6.99k stars 923 forks source link

test_stat_significance()['p_value'] #1006

Closed ankur-tutlani closed 1 year ago

ankur-tutlani commented 1 year ago

Is there any change in the way test_stat_significance()['p_value'] default behavior works? I can see complete opposite values in dowhy 0.10 version compared with 0.9.1 version? Using the same dataset with exact causal model and refutation model giving different p value results. Earlier (0, 0.001) with 0.9.1 version and now (0.999, 1) in 0.10 version.

causal_estimate = model.estimate_effect(identified_estimand, method_name=model_name)

causal_estimate.test_stat_significance()['p_value']

Expected behavior pvalues showing complete opposite with same dataset and parameters.

Version information:

Additional context Add any other context about the problem here.

amit-sharma commented 1 year ago

That might be a bug. Can you share a minimum working example so I can reproduce the error?

ankur-tutlani commented 1 year ago

` from dowhy import CausalModel testcausal.csv model=CausalModel( data = df, treatment='v0', outcome='y', common_causes = ['W0','W1','W2','W3','W4'] ) identified_estimand = model.identify_effect(proceed_when_unidentifiable=True) causal_estimate = model.estimate_effect(identified_estimand, method_name='backdoor.propensity_score_stratification')

pvalue1 = causal_estimate.test_stat_significance()['p_value'] pvalue1`

df is attached csv file. pvalue using 0.9.1 version = (0,0.001) pvalue using 0.10 version = (0.999,1)

amit-sharma commented 1 year ago

Thanks for the sharing the full example. Yeah, this was a bug introduced due to the refactor in this function. I have added a PR. will release a patch version soon.