Open a10y opened 2 months ago
I think we want this instead of HLL++: https://www.cidrdb.org/cidr2019/papers/p23-freitag-cidr19.pdf
(In particular, it gives good estimates of cardinality of arbitrary combinations of attributes rather than just attributes, which is cool / handy for compound join keys)
if we're taking off the shelf, this crate looks potentially better: https://github.com/cloudflare/cardinality-estimator/tree/main
previously mentioned in #85
Useful for compressor to decide if Dict compression is worthwhile.
There's a Rust crate already implementing it: https://docs.rs/hyperloglogplus/latest/hyperloglogplus/struct.HyperLogLogPlus.html
Can be used: