More ad-hoc diagnostics in kons-9! (See also #37.)
This time it's a Sequential Monte Carlo simulation doing Bayesian parameter inference. The X-Y-Z axes are mapped to abstract values: three continuous parameters of a statistical model.
(If you must know it's a linear regression with Gaussian noise. The axes represent the intercept and gradient of the line and the standard deviation of the noise.)
I am manually stepping the simulation forward one step at a time. On each step the latest point-cloud of candidate parameters are plotted and the older ones are faded out. Over time we see how the simulation is moving towards the most representative set of credible parameters for the model.
There is something cool here!
See how the simulation is moving outside of the bounding box of the original particles? That's not what you want to happen: the simulation is being drawn towards "impossible" parameter values that weren't assigned any prior weight.
How can "impossible" parameters even be considered? It's thanks to the particle rejuvenation ("jittering") step of the simulation. It wiggles each particle around on a Metropolis random walk. That allows the particles to escape the model's preconceptions albeit at a painfully slow pace.
Conclusion: Good diagnostic, bad simulation. Has to be repeated with more suitable initial parameters to yield meaningful results.
Thanks again, kons-9! These visualizations are fantastic for "unknown unknowns." The problems that I might be slow to specifically check for with a narrow statistic.
More ad-hoc diagnostics in kons-9! (See also #37.)
This time it's a Sequential Monte Carlo simulation doing Bayesian parameter inference. The X-Y-Z axes are mapped to abstract values: three continuous parameters of a statistical model.
(If you must know it's a linear regression with Gaussian noise. The axes represent the intercept and gradient of the line and the standard deviation of the noise.)
https://user-images.githubusercontent.com/13791/235346188-6947f535-07f6-40eb-87ff-cfb3d279ab53.mp4
I am manually stepping the simulation forward one step at a time. On each step the latest point-cloud of candidate parameters are plotted and the older ones are faded out. Over time we see how the simulation is moving towards the most representative set of credible parameters for the model.
There is something cool here!
See how the simulation is moving outside of the bounding box of the original particles? That's not what you want to happen: the simulation is being drawn towards "impossible" parameter values that weren't assigned any prior weight.
How can "impossible" parameters even be considered? It's thanks to the particle rejuvenation ("jittering") step of the simulation. It wiggles each particle around on a Metropolis random walk. That allows the particles to escape the model's preconceptions albeit at a painfully slow pace.
Conclusion: Good diagnostic, bad simulation. Has to be repeated with more suitable initial parameters to yield meaningful results.
Thanks again, kons-9! These visualizations are fantastic for "unknown unknowns." The problems that I might be slow to specifically check for with a narrow statistic.