search

ahupowerdns / lmdb-safe

A safe modern C++ wrapper of lmdb
MIT License
61 stars 21 forks source link

Comparison with lmdbxx? #1

Open hoytech opened 5 years ago

hoytech commented 5 years ago

Hi, neat looking project! I was wondering if you could weigh in a bit on how this compares to the lmdbxx library by Arto Bendiken (@artob): https://github.com/drycpp/lmdbxx

The automatic conversion stuff looks pretty interesting, especially with respect to string_view.

Thanks!

ahupowerdns commented 5 years ago

Hi! I checked out lmdbxx, and in fact, within PowerDNS we were already using it. We found several issues with it (which could be fixed), but I was not looking for a straight translation of the API. lmdbxx adds some RAII which is great but I really wanted a 'fire and forget' solution that took care of LMDBs object lifetime rules. Also, check out: https://github.com/ahupowerdns/lmdb-safe#lmdb-typed

hoytech commented 5 years ago

Thanks for the info. An interface to automatically maintain indices is a cool concept. It reminds me of BDB's associate: https://web.stanford.edu/class/cs276a/projects/docs/berkeleydb/api_c/db_associate.html

One nice thing about associate was that you could use arbitrary code to compute the index value. Am I understanding it right that with lmdb-typed all values to be indexed have to be stored in member variables of a deserialised representation?

Also, does lmdb-typed assume a serialisation step? I like to use flatbuffers or cap'n proto when possible so that there is no deserialisation step required to read parts of the values (or remove values, add indices, etc).

One last thing: Can you please report the issues you found in lmdbxx? I'm relying on it pretty heavily and would appreciate knowing about any bugs or problems that may bite me in the future.

ahupowerdns commented 5 years ago

Thanks for the input, will take a look! I'm in fact working on a computed index right now because PowerDNS needs something like that for DNSSEC.

i want to abstract out the serialisation so you can pick. Flatbuffers looks very exciting but the zero-copy nature may be hard to exploit (except for selection, where it may be very very useful).

Regarding lmdbxx, the main thing I personally ran into is that it serialized a std::string as the actual binary contents of the string class, which took some debugging. This made me pretty worried about if they got the rest of it right.

hoytech commented 5 years ago

Ahhh got it. Yep I've never tried to serialize a std::string directly. I didn't even know lmdbxx supported that to be honest. I've always just made values with lmdb::val(my_str.c_str(), my_str.size())

Anyway I appreciate you publishing this code, I'm going to take a more detailed look soon when I get a chance. Cheers!

tiendq commented 5 years ago

It seems that lmdbxx is a dead repo now but it's small enough for a hard fork :D

hoytech commented 5 years ago

I just forked lmdbxx for anyone interested:

https://github.com/hoytech/lmdbxx

The main difference is I got rid of lmdb::val and replaced it with C++17's std::string_view.

@ahupowerdns - I also removed the templated convenience methods so it shouldn't be possible to serialise an std::string as its internal representation as you experienced.