Parameter estimation for complex physical problems often suffers from finding ‘solutions’ that are not physically realistic. The PEUQSE software provides tools for finding physically realistic parameter estimates, graphs of the parameter estimate positions within parameter space, and plots of the final simulation results.
Currently, PEUQSE checks if the mu_AP and MAP are relatively close, and does not do more than that.
Regarding Convergence: My view agrees with the first 28 slides of this pdf: It basically just says it is normally impossible to know when an MCMC solution is right and complete enough. There is some art and experience involved.
There are some methods mentioned in 2nd presentation:
Geweke --- evidently just shows when chain isn’t changing. That doesn’t exclude getting trapped in a bad solution.
Gelman-Rubin --- compares different chains. This is good to do, but still doesn’t exclude that most of the chains may have gotten trapped in a bad solution. Also, assumes a single correct solution.
Heidelberg-Welch --- looks for stationarity, so same problem as Geweke.
Raftery-Lewis --- This is interesting. It tries to estimate how long an MCMC run is required to get a certain accuracy in a posterior, based on the sampling so far. Still some of the same problems since the initial data can already be trapped, but it is related to one of the ideas I have (see further below).
Ashi’s Idea [but not for this year]: In response to the repeated questions of how to know when it is converged [which I note again is impossible to know for our kinds of problems] … this morning I came up with what I think is a general strategy for making some confidence intervals of sufficient exploration.
Currently, PEUQSE checks if the mu_AP and MAP are relatively close, and does not do more than that.
Regarding Convergence: My view agrees with the first 28 slides of this pdf: It basically just says it is normally impossible to know when an MCMC solution is right and complete enough. There is some art and experience involved.
https://astrostatistics.psu.edu/RLectures/diagnosticsMCMC.pdf
This other presentation agrees at the end that there is no way of knowing. The best one can do is exclude convergence, not prove convergence. https://www2.stat.duke.edu/courses/Fall09/sta290/Lectures/Diagnostics/param-diag.pdf
There are some methods mentioned in 2nd presentation: Geweke --- evidently just shows when chain isn’t changing. That doesn’t exclude getting trapped in a bad solution. Gelman-Rubin --- compares different chains. This is good to do, but still doesn’t exclude that most of the chains may have gotten trapped in a bad solution. Also, assumes a single correct solution. Heidelberg-Welch --- looks for stationarity, so same problem as Geweke. Raftery-Lewis --- This is interesting. It tries to estimate how long an MCMC run is required to get a certain accuracy in a posterior, based on the sampling so far. Still some of the same problems since the initial data can already be trapped, but it is related to one of the ideas I have (see further below).
Ashi’s Idea [but not for this year]: In response to the repeated questions of how to know when it is converged [which I note again is impossible to know for our kinds of problems] … this morning I came up with what I think is a general strategy for making some confidence intervals of sufficient exploration.
This paper looks at the trend in the 'evidence' value, which is also a good idea, but I still think that won't work well enough for complex problems. https://academic.oup.com/mnras/article/437/4/3918/1011939?login=true