Asynchronus batched commits

krisskross commented 9 years ago

This is a question to users of LMDB JNI so please speak your opinion.

Is an API for doing batched asynchronous commits into LMDB something useful or would it only add unnecessary complexity? This is not related to MDB_MAPASYNC.

Add method Database.batch(DirectBuffer key, DirectBuffer value)
Uses a ring buffer to avoid thread contention.
The ring buffer accept DirectBuffer key/value to avoid copies.
A background thread poll the ring buffer periodically, read a configurable number of entries and commit them in a single transaction.

phraktle commented 9 years ago

The usefulness of this depends on the circumstances. I don't think a general mechanism for this should be built in to the core lmdbjni API. The low-level API should match the basic LMDB functionality as much as possible.

If provided, this should be in a cleanly separated higher utility layer (similar to issue https://github.com/deephacks/lmdbjni/issues/12).

krisskross commented 9 years ago

Yes, I think you're right. A separate util layer is more appropriate. Maybe even a separate jar so that users can choose to add it or not?

deephacks / lmdbjni

Asynchronus batched commits #22