interpretation of refutation test results and decision making

[ianisadreamer]

Ask your question A clear and concise question, potentially including what was tried and what was observed. This can also include a verbatim copy of outputs, or screenshots.

Expected behavior If applicable, a clear and concise description of what you expected to happen.

Version information:

• DoWhy version [e.g. 0.7]

Additional context Add any other context about the problem here.

[ianisadreamer]

Hi, I'm wondering does an estimate has to pass all the possible refutation tests to give us confidence say "this estimation is not problematic?" my situation is: the estimation passed random cause, subset and bootstrap, but only failed placable. Should I trust this estimate? From another threads, I saw the author suggested placebo test is more about the estimator. Should I change the estimator to see if it passes? Thanks!

[amit-sharma]

These refutations are necessary tests. So, a good analysis should pass all the tests. So if your estimator fails the placebo test, there is something wrong. Placebo test is not just about the estimator, it points to an error anywhere in your analysis: either in the modeling stage (Graph) or the estimation stage.

In practice, changing the estimator is a good first step to debug. If multiple estimators fail the test, you may need to look at your graph too and make sure that it is correct.

For more information, you can refer to the user guide: https://www.pywhy.org/dowhy/v0.10/user_guide/refuting_causal_estimates/index.html

[github-actions[bot]]

This issue is stale because it has been open for 14 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 7 days since being marked as stale.