search

tvondra / tdigest

PostgreSQL extension for estimating percentiles using t-digest
PostgreSQL License
88 stars 13 forks source link

WIP: experimental building of histograms from tdigest #5

Open tvondra opened 4 years ago

tvondra commented 4 years ago

This implements two simple SQL functions, building different types of histograms from a t-digest. Equi-width histograms are the "usual" type with each bin covering about 1/N of the domain, while equi-height histograms have bins representing the same fraction of the data.

The functions are fairly trivial SQL functions, calculating boundaries on either the domain or frequency, and then computing the other values using the t-digest.

I'm not sure how accurate the histograms are - considering how t-digests work, the bins at the ends should be the fine while the bins in the middle could be somewhat less accurate.

One thing this made me realize is the extension only provides aggregate functions, so when we need to work with t-digests in non-aggregate contexts it's necessary to do unnecessary groupings etc. Might be handy to provide some non-aggregate functions to make that unnecessary.

sporty81 commented 4 years ago

These look pretty good to me at first glance, and from my testing of them. I will spend some more time thinking about how I would want to build out histograms for my own usage and how this all relates. Thank you!