search

timshannon / bolthold

BoltHold is an embeddable NoSQL store for Go types built on BoltDB
MIT License
643 stars 45 forks source link

readme.md: boltholdIndex, boltHoldIndex #113

Closed nicewook closed 4 years ago

nicewook commented 4 years ago

In the readme.md

boltholdIndex and boltHoldIndex is mix used

timshannon commented 4 years ago

Canonically it should be boltholdIndex. I'll update the README.

Thanks for the heads up.

nicewook commented 4 years ago

@timshannon Thank you for your fast response. I actually came from storm. coz storm does not have index support for the query. I have some questions. Is there any proper place to ask? or issuing here is okay?

I think store.Insert() is getting too slow when there are a lot of records (for me it's over 200,000 and it takes 500ms) moving from bolt to badger would help in this case?

timshannon commented 4 years ago

It should be pretty easy to switch to badgerhold and try it out. The slow inserts may be due to Bolt as I believe inserts are slower than with Badger. However if you are using one or more non-unique indexes, you may be running into issue #106.

I'm still trying to figure out the best way to handle that. I looked through storm's code to see if they were handling it better, but they do the same thing. The problem is the index is just a slice, and when that slice gets really large it can take a long time to read, write and serialize.

Real databases break their indexes into btree pages the same as any other data, so I'd like to do something similar, but I haven't gotten around to working through all the implications of doing that yet.

cbrake commented 4 years ago

yes, this is an interesting problem, and probably the next step to scaling bolthold into larger databases. Are implementations in databases like sqlite relevant for reference? I assume they solve this problem.