search

blugelabs / bluge

indexing library for Go
Apache License 2.0
1.88k stars 122 forks source link

bitsets are faster than roaring #104

Open joeblew99 opened 2 years ago

joeblew99 commented 2 years ago

https://github.com/kelindar/bitmap#boolean-algebra shows a clean and simple lib as well as a data store at https://github.com/kelindar/column which uses the bitmap

I think this is faster than roaring, and has the ability to do facets and terms

mschoch commented 2 years ago

We are open to learning more about it, but as is often the case "faster" typically comes with qualifications and trade-offs. Just looking at the readme, I see the following, "designed for dense small or medium collections", which may not make it suitable for all the places we use bitsets inside of Bluge.

What do you have in mind as a next step?

joeblew99 commented 2 years ago

next steps...

bench it and check its properties match the needs of Bluge.

Did check out https://github.com/kelindar/column ?It has some nice properties for reactive style architectures, which i really like. Great docs too: https://pkg.go.dev/github.com/kelindar/column#section-readme

Let me know how you progress. I am just trying things out for now. It looks like column is designed as a very fast store for a game system...