search

py-why / dowhy

DoWhy is a Python library for causal inference that supports explicit modeling and testing of causal assumptions. DoWhy is based on a unified language for causal inference, combining causal graphical models and potential outcomes frameworks.
https://www.pywhy.org/dowhy
MIT License
7.01k stars 922 forks source link

KL Divergence: what is a good value for the estimate using sample drawn from the causal graph and observed data? #944

Closed pvizan-artefact closed 1 year ago

pvizan-artefact commented 1 year ago

Hi!

I came across this link: https://www.pywhy.org/dowhy/v0.9.1/user_guide/gcm_based_inference/draw_samples.html where it is explained how to estimate the KL divergence between data sampled from a learned graph and observed data. I understand ideally this metric should be as close to 0 as it is possible, and that that is equivalent to the graph resembling the observations of the real world. However, what is, in this case, a good methodology to define a threshold? In my understanding, the estimate of KL divergence has an upper bound of infinity, so it is hard in this context to decide when the estimate is good, and when it is not.

Any experience from any user evaluating causal models using this metric, and how did you proceed with it? Thanks a lot

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 14 days with no activity.

github-actions[bot] commented 1 year ago

This issue was closed because it has been inactive for 7 days since being marked as stale.