kons-9 for ad-hoc data visualization

[lukego]

kons-9, the 3D IDE in Lisp, really came in handy this week for ad-hoc data visualization while debugging a Sequential Monte Carlo (aka particle filter) simulation.

It feels like a superpower to load a 3D modeling environment directly into the process where my simulator is running and to fluidly insert new data into the visualization as it is being generated. There turned out to be a whole lot of diagnostics I could run just by spinning the model around and eyeballing it.

This turned out to be much more productive than my initial approach of dumping simulation data into DuckDB tables for separate inspection with ggplot2-based notebooks. Thanks @kaveh808 for a great tool in the toolbox!

Here's some eye candy with rough notes below:

https://user-images.githubusercontent.com/13791/235121210-e47544b2-fc7d-4924-bf29-7b4d2421e52b.mp4

Background:

• Simulation is considering Gaussian distributions with various mean (X-axis) and standard deviation (Y-axis.)
• The first step (backmost layer) is just wild guesses drawn uniformly from -10 to 10.
• Subsequent layers are conditioned on the contrived data points:
  • 1, 2, 3, 4, 5, 6, 7, 8, 9, 10
• Conditioning happens by killing low-likelihood particles, duplicating high-likelihood particles, and jiggling them all around.
• Likelihood tempering is used to gradually become more ruthless about killing the low-likelihood particles.
• The final layer is the posterior distribution: you can do Bayesian inference by looking at those particles!

Observations:

• The posterior samples are quite spread around. There's a lot of remaining uncertainty.
• But we can still make some broad inferences:
  • The mean value must be positive: all the particles left of the X-axis were culled.
  • The standard deviation couldn't be a small value like 1: particles that low on the Y-axis all died.
  • The X/Y axes are correlated in a "V" shape: mean values close to five make lower standard deviation values plausible.
• ... though we mustn't forget that this is nonsense since our values were not drawn from a Gaussian distribution.
• The shape of the particle cloud changed most rapidly in the beginning: might this suggest that the likelihood tempering schedule is a bit aggressive? I was expecting a smoother cone-like shape on the Z-axis.

Cute aspects:

• You can marginalize out a variable just by spinning the simulation to align with its axis.
• You can see particle rejuvenation at work:
  • the outermost particles follow a wobbly trajectory as the simulation jitters them with metropolis moves on each step.
• You can interact with the simulation in interesting ways:
  • I used breakpoints to drop into the Lisp debugger between each simulation step. This way I have access to the complete simulation state if I happen to notice something funny.

Three cheers for kons-9!

and especially because coding the hook from simulator to 3D model was no trouble at all:

;;; trail.lisp -- diagnostic to follow the tracks of a simulation

(defpackage #:smc-trace
  (:use #:permo #:permo-lisp)
  (:export #:reset #:smc-step #:*step*))
(in-package #:smc-trace)

(defparameter *z* 0)
(defvar *step* nil)

(defun reset ()
  "Reset the SMC visualization."
  (setf *z* 0)
  (kons-9::clear-scene kons-9::*scene*))

(defun smc-step (particles)
  "Visualize the next simulation step with PARTICLES.
   Particles is a list of parameter-vectors."
  (draw-particles particles))

(in-package #:kons-9)

(defun smc-trace::draw-particles (ps)
  (setf (shading-color *drawing-settings*) (c-rand))
  (let ((pc (particles-to-point-cloud ps)))
    (allocate-point-colors pc)
    (add-shape *scene* pc)
    (when smc-trace::*step* (break))))

(defun particles-to-point-cloud (particles)
  "Return a point-cloud mapping the two dimensions of PARTICLES onto X/Y.
   Step the Z-axis forward."
  (loop with n = (length (first particles))
        with point-array = (make-array n)
        for i below n
        do (setf (aref point-array i)
                 (p (aref (first particles) i) (aref (second particles) i) smc-trace::*z*))
        finally (progn
                  (incf smc-trace::*z* 0.25)
                  (return (make-point-cloud point-array)))))

(defun p (x y z)
  "Return the point (X Y Z)."
  (p:vec (coerce x 'single-float)
         (coerce y 'single-float)
         (coerce z 'single-float)))

[aykaramba]

This was a great read, thank you for putting in the time to blog about this. It's a good entry point for those of us that want to explored this sort of thing a bit more, particularly if we have no familiarity with the topic.