maxpert / marmot

A distributed SQLite replicator built on top of NATS
https://maxpert.github.io/marmot/
MIT License
1.7k stars 42 forks source link

Support for 2 nodes only? #68

Closed oz123 closed 11 months ago

oz123 commented 11 months ago

Earlier versions of marmot didn't require JetStream. As far as I can tell, I can still connect to a regular nats server - I can see the connections - but it seems that sync isn't working.

Another question is: If I understand this properly, writes from one database trigger inserts to a dedicated table which will then be read by marmot and sent to nats. The question is - how often does marmot read changes, and is this configurable?

maxpert commented 11 months ago

Very early versions of Marmot (before 0.4.x) were built on DragonBoat. I abandoned that library mostly due to extremely slow responses from author, and missing support for IPv6 (which I saw as a blocker for myself). From that point onwards Marmot always depended on JetStream, because that is the only RAFT reliable way of persisting change logs in NATS. You can still connect to regular NATS server, but it has to launch with -js flag. This should be a non-issue now because if you don't specify any cluster binding then NATS is launched in non-cluster mode with JetStream turned on for Marmot.

Marmot keeps a running read transactions, with WAL support turned on. Which means ASAP frames are written to disk the WAL log file will change, and then depending on OS notification infra (since we are watching the WAL files), Marmot almost instantly picks up those changes. There is also an polling_interval configuration here that you can use to set interval time, but that should not be required and I plan to deprecate that in future.

oz123 commented 11 months ago

Thank you for the quick response. I am actually not so interested in RAFT. Which coerces me to use 3 nodes. I am happy with just running nats-server --jetstream as a binary outside of marmot I'll continue this in a discussion.

https://github.com/maxpert/marmot/discussions/70

oz123 commented 11 months ago

If anyone ever looks for this: I posted an example of a two node marmot setup in the linked discussion.

bharath82d commented 3 months ago

Does Marmot requires minimum three nodes ?