quantified-uncertainty / squiggle

An estimation language
https://squiggle-language.com
MIT License
156 stars 23 forks source link

Performance is bad in Observable Notebooks. #1341

Open Hazelfire opened 2 years ago

Hazelfire commented 2 years ago

Description:

I'm not sure why, but performance is really poor in observable notebooks. My best guess currently is that KDE takes a lot of time, as the performance issues seem somewhat correlated with the number of graphs I render at the same time.

This could give more reason to use histograms rather than KDE to display sample set distributions. Which would also fix the issue of rendering samples below 0.

Steps to reproduce:

Some of my notebooks here: https://observablehq.com/@hazelfire/quantifying-uncertainty-in-givewell-ceas (particularly Helen Keller and Malaria Consortium, but also just the main notebook)

mlochbaum commented 2 years ago

I think it's more likely that the cost is somewhere in rendering point sets. In that case we'd want to cut down the number of points before rendering by averaging groups of them together or something. It should be possible to test this by running a notebook that plots many symbolic distributions converted directly to point sets, as this doesn't go through KDE.

1339 greatly improves the performance of KDE itself, and I don't think it should be significantly different from histogramming now: the per-sample operations are about the same, with the difference for KDE being post-processing on the points, and there are usually a lot less points than samples. The overall conversion has a lot of other costs that I haven't fully accounted for yet. Sorting samples is a big one.