Interpretation

[dbaranger]

I have a question about how to interpret the results of CAUSE.

The tutorial states:

In this case we see that the causal model is significantly better than the sharing model from the thrid line of the table. The z-score is -4.34 corresponding to a p-value of 7.2^{-6}.

But the manuscript states:

If ΔELPD = ELPDC − ELPDS is positive, then the posteriors from the causal model predict the data better, so the causal model is a better fit. If ΔELPD ≤ 0, then the sharing model fits at least as well, indicating that the data are not consistent with a causal effect.

These statements seem to contradict each other. I assume the manuscript is correct and the tutorial is a typo?

[jean997]

They are both correct, but I definitely see that it is confusing.

In the paper, we defined \Delta ELPD = ELPDC - ELPDS because people are generally more used to thinking about large z-scores as being significant than small z-scores. However, at that point the software was already written and in use and the software reports ELPDS - ELPDC. So the software tutorial correctly describes how to interpret the software results and the paper correctly describes the mathematical idea. There is just a sign difference in the definitions used by the two. Were I doing it again, I would make them match but alas we are trapped in this imperfect reality.

The way to remember it is that in the results table, delta_elpd will always be elpd(Model 1) - elpd(Model 2) and bigger elpd means a better model. So if delta_elpd is positive Model 1 is better and if delta_elpd is negative model 2 is better.

[jean997]

I should add a note about this to the software tutorial.

[dbaranger]

Thanks for the clarification and the fast response!