Create summary table of all DP benchmarks

[luiarthur]

The DP models benchmark for a given sample size can be summarized in the table below, with the cells being compile times (if applicable) and execution times. Timings for forward passes and gradient computations of the models will also be computed if high-level API is available and exposed. The appropriate accompanying data visualizations (line plots, surface plots) will be included to make the results more interpretable. Links will be included to MWEs.

• [ ] Make a dynamic table which changes N. (e.g. N=50, 100, 200, 400, 800, 1600).
• [ ] Make interactive line plots where
  • x-axis is N, and y-axis is execution time or compile time
  • be able to select which implementations / PPL to show on the plot

Model (PPL), sample size = N	Collapsed Gibbs	SMC	Particle Gibbs	HMC/NUTS (SB)	ADVI (SB)
DP mixture of Gaussians (Turing)	[ ] TODO	[ ] TODO	[ ] TODO	[X] TODO	[X] TODO
DP mixture of Gaussians (STAN)	NA	NA	NA	[X] TODO	[X] TODO
DP mixture of Gaussians (Nimble)	[X] TODO	[ ] TODO	[ ] TODO	NA	NA
DP mixture of Gaussians (Pyro)	NA	[ ] TODO	NA	[X] TODO	[X] TODO
DP mixture of Gaussians (TFP)	NA	NA	NA	[X] TODO	[X] TODO
DP mixture of non-Gaussians (Turing)	[ ] TODO	[ ] TODO	[ ] TODO	[ ] TODO	[ ] TODO
DP mixture of non-Gaussians (STAN)	NA	NA	NA	[ ] TODO	[ ] TODO
DP mixture of non-Gaussians (Nimble)	[ ] TODO	[ ] TODO	[ ] TODO	[ ] TODO	[ ] TODO
DP mixture of non-Gaussians (Pyro)	NA	[ ] TODO	[ ] TODO	[ ] TODO	[ ] TODO
DP mixture of non-Gaussians (TFP)	NA	NA	NA	[ ] TODO	[ ] TODO

Notes:

• Prefer doing the experiments by column as opposed to by row in this table. This way, if we are run out of time, we have benchmarks across PPLs, for a few implementations; and not benchmarks across a few implementations for one or two PPLs.
• For now, for fix truncation level for number of stick-breaking components. Time permitting, see if a benchmark can be done to include varying truncation levels.

[luiarthur]

This Google sheet contains the timings.

[luiarthur]

• Some changes in decorators provided significant speedups in TFP implementation. Need to update
  • [ ] timings
  • [ ] notebook
  • [ ] snippets