How to interpret ELPD p-value?

I have been running CAUSE and I have hard time interpreting p-values in the model comparison step. The paper states that a positive dELPD indicates that the causal model is better fit than the sharing model. It also says that the p-value estimates the probability of obtaining a z-score larger than the one observed if the true value of ΔELPD were less than or equal to zero. My results are the following.


Model1: Sharing
Model2: Causal
dELPD: 0.91
se_dELPD: 0.36
Z: 2.5
P: 0.99

Give the positive dELPD and large Z-score, I don't understand the large p-value. Given that the p-value signifies the probability of observing a Z-score larger than the observed, I would compute the p-value as such P[X > x]:

pnorm(2.5, lower.tail=F)
0.0062

Rather the p-value from CAUSE matches the one computed as such P[X ≤ x]:

pnorm(2.5, lower.tail=T) 
0.99

So this is the probability of observing a Z-score of 2.5 or below. This matches the p-value from CAUSE, but this is not the probability of observing this larger Z-score as it says in the paper, rather the probability of observing a smaller. How does this test makes sense? And how do I interpret these results?

Best regards

Hi Kasper, In the code, delta ELPD = ELPD(model 1) - ELPD (model 2). So a negative delta ELPD is in favor of model 2 and a positive is in favor of model 1. In the ELPD table, the more complex model is always model 2 and the p-value is computed for P(delta ELPD < d). In your results you have a positive delta ELPD which indicate that the posteriors estimated under the sharing model fit the data better than the posteriors estimated under the causal model. There are some details on this in the help file for the cause function (?cause) and also here: https://jean997.github.io/cause/ldl_cad.html#Step_5:_Look_at_Results.

In the paper we oriented the z-score to be positive for ease of explanation but left the orientation in the code for back compatibility.

Jean

jean997 / cause

How to interpret ELPD p-value? #21