search

ayazhafiz / xorf

Xor filters - efficient probabilistic hashsets. Faster and smaller than bloom and cuckoo filters.
MIT License
129 stars 27 forks source link

Add from_iterator/try_from_iterator for the filters. #60

Closed vlovich closed 1 year ago

vlovich commented 2 years ago

Add a helper to construct using an iterator. I'm not convinced these benchmark results aren't indicative of some kind of machine setup issue on my end. Can't imagine why there'd be an across-the-board perf improvement like that and on the previous run I did (before I added the new benchmarks) I saw a mix of improvements and regressions.

    BinaryFuse16/from/500000
                            time:   [19.930 ms 19.993 ms 20.060 ms]
                            change: [-17.320% -8.1279% +1.8364%] (p = 0.17 > 0.05)
                            No change in performance detected.

    BinaryFuse32/from/500000
                            time:   [19.161 ms 19.305 ms 19.526 ms]
                            change: [-33.901% -21.972% -9.3098%] (p = 0.01 < 0.05)
                            Performance has improved.

    BinaryFuse8/from/500000 time:   [18.473 ms 18.537 ms 18.670 ms]
                            change: [-36.472% -26.878% -17.103%] (p = 0.00 < 0.05)
                            Performance has improved.

    Fuse16/from/500000      time:   [29.237 ms 29.529 ms 30.126 ms]
                            change: [-53.901% -34.098% -9.1085%] (p = 0.07 > 0.05)
                            No change in performance detected.

    Fuse32/from/500000      time:   [30.281 ms 30.392 ms 30.601 ms]
                            change: [-35.093% -20.946% -6.5932%] (p = 0.03 < 0.05)
                            Performance has improved.

    Fuse8/from/500000       time:   [29.309 ms 37.546 ms 45.843 ms]
                            change: [-56.022% -43.412% -23.294%] (p = 0.00 < 0.05)
                            Performance has improved.

    Xor16/from/500000       time:   [42.116 ms 44.413 ms 46.674 ms]
                            change: [-44.116% -32.964% -18.902%] (p = 0.00 < 0.05)
                            Performance has improved.

    Xor32/from/500000       time:   [42.222 ms 42.648 ms 43.433 ms]
                            change: [-6.5170% +0.2829% +7.5735%] (p = 0.95 > 0.05)
                            No change in performance detected.

    Xor8/from/500000        time:   [40.687 ms 41.027 ms 41.439 ms]
                            change: [-43.893% -32.745% -17.619%] (p = 0.00 < 0.05)
                            Performance has improved.

Resolves #59

ayazhafiz commented 1 year ago

This LGTM other than the comments by @jakelee8 above (thanks @jakelee8 !)

vlovich commented 1 year ago

Sorry for the late reply. Didn't notice this until recently & freed up some time. Review comments addressed.

ayazhafiz commented 1 year ago

Thanks @vlovich! Looks like there are a few lint warnings to fix.

vlovich commented 1 year ago

Fixed I think.

vlovich commented 1 year ago

ping @ayazhafiz

ayazhafiz commented 1 year ago

Sorry for the delay @vlovich . This looks good, I am merging and will release a patch with some of the latest changes shortly.

ayazhafiz commented 1 year ago

@vlovich , this is available in https://github.com/ayazhafiz/xorf/releases/tag/0.9.0. Thank you so much!