search

appy-one / acebase-server

A fast, low memory, transactional, index & query enabled NoSQL database server for node.js that easily syncs with browser and node.js clients and servers
MIT License
32 stars 14 forks source link

Idea: hashring replication #14

Closed rcoder closed 2 years ago

rcoder commented 2 years ago

So this is kind of a wild experiment, but I decided to spend a minute building a proof of concept to replicate data across acebase servers using a hash ring (ala memcached) instead of a single global IPC server:

https://github.com/rcoder/acebase-ring

The bulk of the actual replication logic is here: https://github.com/rcoder/acebase-ring/blob/main/src/server.ts#L80-L195

I'm almost certainly doing something wrong, and the code is just a big mess, but it's an interesting thought experiment at least. The advantage over a single IPC coordinator is of course the lack of a single coordinator that is persistently connected to all of the storage servers. (I.e., it's a true p2p cluster model.)

If I really find myself with time to burn I might try connecting replicas via WebRTC, but that's a science project for another day. :)

rcoder commented 2 years ago

Speaking of "doing something wrong", I think the above example may be accidentally exponential (b/c write amplification, perhaps?) based on the amount of log spam I'm seeing. :)

rcoder commented 2 years ago

Also, just in case it wasn't clear: this really is just a, "hey, this works and it's neat!" issue, not "please support this in addition to your tested, stable clustering model." Feel free to close w/o further discussion, and I can let you know if I go further with my experiments. :)

appy-one commented 2 years ago

Sounds cool! So, how would you describe the workings of the ring? Is it multiple servers with their own local db that distribute changes to all other known peers?