Closed jgabry closed 11 months ago
maybe something like,
"which may indicate poor approximation of the target distribution; re-parameterization may help."
And if re-parameterization may help — hope springs eternal — are there other pathfinder diagnostics or hints on where to start?
@avehtari What do you think?
maybe something like,
"which may indicate poor approximation of the target distribution; re-parameterization may help."
I think it may still be a good idea to mention misspecification because that's still a possibility. Maybe:
Pareto k value is greater than 0.7 which may indicate a poor approximation
to the target distribution or model misspecification.
Or we could say more:
Pareto k value is greater than 0.7 which indicates infinite variance
of the importance ratios used for resampling.
This may be due to a poor approximation to the target distribution or model misspecification.
That's a bit verbose but much more informative. Maybe @avehtari will have a better idea.
The model misspecification is not relevant here, and probably was mistakenly carried over from LOO-CV diagnostic.
I would write "Pareto k diagnostic value is greater than 0.7 which may indicate a poor approximation and indicates that importance resampling is not able to improve the approximation."
Description:
This warning message
https://github.com/stan-dev/stan/blob/6d35a762bf56c72fef574302e5af4b9c70703625/src/stan/services/pathfinder/psis.hpp#L263-L268
is not quite right. Misspecification is a possibility, but the model may often be specified just fine and the Pareto k diagnostic could be high because Pathfinder + PSIS might still be a poor approximation to the posterior.
Reproducible Steps:
Look at the code I linked to above to see the message, you don't need to run a model.
Current Output:
The warning in the code linked above.
Expected Output:
A more accurate warning message indicating that it might be misspecification but could also likely be that the approximation is poor.
Additional Information:
Provide any additional information here.
Current Version:
v2.32.2