search

stan-dev / posterior

The posterior R package
https://mc-stan.org/posterior/
Other
167 stars 24 forks source link

`ps_khat_threshold` can return values greater than 0.7 #360

Closed n-kall closed 7 months ago

n-kall commented 7 months ago

Currently ps_khat_threshold is just returning 1 - 1 / log10(S). As the recommendation in PSIS paper is to use 0.7 as an upper bound on the threshold, should this be changed to min(1 - 1 / log10(S), 0.7) as it is in loo? @avehtari

avehtari commented 7 months ago

I would not change that, as it's still telling whether the error can be small. Capping with 0.7 is related to reliability of MCSE estimate which is not good if bias dominates, but then if sample size is big enough, bias can be small too. We can talk more when I'm backl to campus

n-kall commented 7 months ago

Ok, that sounds good. Closing this