search

helium / plumtree

Epidemic Broadcast Trees
Apache License 2.0
193 stars 51 forks source link

Basho's eleveldb backend limits the environments plumtree can be used in #11

Open ixmatus opened 9 years ago

ixmatus commented 9 years ago

Plum (our company really is named that, hilariously) is using Plumtree in a highly constrained ARM environment. Basho's eleveldb and leveldb forks, unfortunately, are tuned for 64bit systems and therefore cause segmentation faults in the Erlang VM on ARMv5 chips.

I needed Plumtree and because we have a very very low-write environment (we use it to replicate a few configuration keys, track membership, etc...) I ported hashtree.erl's dependency on eleveldb to the lets library: https://github.com/plumlife/plumtree/tree/lets-port

NOTE: lets does not expose leveldb iterators in its public interface so re-using the eleveldb iterator pattern for snapshots was currently not straight-forward. We snapshot by doing the equivalent of tab2list which is dangerous if you have a large dataset!

ixmatus commented 9 years ago

This does, mean, that lets could be dropped for ets or dets too. The other backends for lets can also be used now.

jrwest commented 9 years ago

Some background here re: the initial implementation. The original goal did not include using leveldb for hashtree.erl. Instead we (Basho at the time) wanted to build a snapshot-able ets table but ran out of time before a release. So +1 on getting rid of leveldb but I think ideally the hashtree backend should be pluggable and support an in-memory implementation as well (since the trees are thrown away on startup)

One other note, it is important for the correct operation of the hashtree that read-isolated snapshots are provided by the backend.

ixmatus commented 9 years ago

Because lets is pluggable, there’s a configuration option when creating the table to specify using ets instead of leveldb, so that’s very easy.

The snapshots are read-isolated, I perform a lets:tab2list/1 and save that list to the state record and use it to fold / compare the hashtree in a read-only state. ᐧ ​ ᐧ

On Wed, Jun 10, 2015 at 10:52 AM, Jordan West notifications@github.com wrote:

Some background here re: the initial implementation. The original goal did not include using leveldb for hashtree.erl. Instead we (Basho at the time) wanted to build a snapshot-able ets table but ran out of time before a release. So +1 on getting rid of leveldb but I think ideally the hashtree backend should be pluggable and support an in-memory implementation as well (since the trees are thrown away on startup)

One other note, it is important for the correct operation of the hashtree that read-isolated snapshots are provided by the backend.

— Reply to this email directly or view it on GitHub https://github.com/helium/plumtree/issues/11#issuecomment-110808808.

Parnell Springmeyer parnell@digitalmentat.com | digitalmentat.com | 0xDCCF89258EAD874A http://pgp.mit.edu/pks/lookup?op=get&search=0xDCCF89258EAD874A

tsantero commented 9 years ago

I agree the backend for hashtree ought to be pluggable, and for our purposes eleveldb is not an ideal solution either. @ixmatus I am happy to collaborate on work to move toward this goal.

ixmatus commented 9 years ago

If you look at my fork, I've already done this work and it uses lets which has four pluggable backends: hyperleveldb, rocksdb, leveldb, and ets. https://github.com/plumlife/plumtree/tree/lets-port ᐧ

On Wed, Jun 10, 2015 at 3:03 PM, Tom Santero notifications@github.com wrote:

I agree the backend for hashtree ought to be pluggable, and for our purposes eleveldb is not an ideal solution either. @ixmatus https://github.com/ixmatus I am happy to collaborate on work to move toward this goal.

— Reply to this email directly or view it on GitHub https://github.com/helium/plumtree/issues/11#issuecomment-110894518.

Parnell Springmeyer parnell@digitalmentat.com | digitalmentat.com | 0xDCCF89258EAD874A http://pgp.mit.edu/pks/lookup?op=get&search=0xDCCF89258EAD874A

ixmatus commented 9 years ago

All of the tests on my lets-port fork pass, btw. ᐧ

On Wed, Jun 10, 2015 at 3:12 PM, Parnell Springmeyer < parnell@digitalmentat.com> wrote:

If you look at my fork, I've already done this work and it uses lets which has four pluggable backends: hyperleveldb, rocksdb, leveldb, and ets. https://github.com/plumlife/plumtree/tree/lets-port ᐧ

On Wed, Jun 10, 2015 at 3:03 PM, Tom Santero notifications@github.com wrote:

I agree the backend for hashtree ought to be pluggable, and for our purposes eleveldb is not an ideal solution either. @ixmatus https://github.com/ixmatus I am happy to collaborate on work to move toward this goal.

— Reply to this email directly or view it on GitHub https://github.com/helium/plumtree/issues/11#issuecomment-110894518.

Parnell Springmeyer parnell@digitalmentat.com | digitalmentat.com | 0xDCCF89258EAD874A http://pgp.mit.edu/pks/lookup?op=get&search=0xDCCF89258EAD874A

Parnell Springmeyer parnell@digitalmentat.com | digitalmentat.com | 0xDCCF89258EAD874A http://pgp.mit.edu/pks/lookup?op=get&search=0xDCCF89258EAD874A

benoitc commented 9 years ago

Another backend that could be used is rkvs which handle many backends as well. The read snapshot should be added though, I will try to make a branch asap.