tidwall / buntdb

BuntDB is an embeddable, in-memory key/value database for Go with custom indexing and geospatial support
MIT License
4.57k stars 289 forks source link

Question: cost/benefit of supporting generic types; and persisting secondary indexes #110

Open EmmanuelOga opened 1 year ago

EmmanuelOga commented 1 year ago

Hey, just to be clear this is not a feature request, just some design questions.

I was wondering if you have any thoughts about cost/benefit of making the DB accept a generic K/V type instead of just strings.

Since tidwall/btree supports generics, maybe BuntDB could support them too. Whatever type is used for the key/values would still have to support serialization for the AOF, but in a lot of cases the data could be accessed as is, without the need to de/serialize.

I understand this would be less safe because of increased chance for inconsistency (because of the possibility of mutations on the items), but should also relieve a lot of the back-and-forth work of serialization, probably also be more efficient in terms of memory consumption and GC.

The other question is whether or not it would make sense to persist secondary indexes. Currently each secondary index needs to be rebuilt in memory each time the app restarts... i'm guessing for an app with many indexes this could become more and more costly each time the app is updated for a new deploy. Perhaps this is fast enough that is not really a problem for production, depending of data size / etc.

Thx!