Add ability to use Gossip strategy

[KamilLelonek]

Proposal

Allow to use mnesiac with Cluster.Strategy.Gossip, i.e. when hosts are not known upfront.

Background

Currently, it looks like hosts are required when configuring mnesiac which requires to know them in advance. I'd like to dynamically read them instead of explicitly list in a config.

Workarounds

Not sure, but I guess - provide Node.list/0 as hosts maybe.

[beardedeagle]

Great suggestion!

[justinmichaelvieira]

@beardedeagle are you able to confirm Kamil's workaround above should work?

Are there cases we should be aware of, where that workaround would not work?

[xadhoom]

Is not so easy, normally when you use Gossip strategy is because you add/remove nodes without the need to know them in advance. Passing Node.list() will work only when the multicast discovery of the Gossip strategy has finished connecting nodes. And still you cannot add nodes later, at least not easily.

Maybe the correct way is to ignore any predefined nodes list, listen for :net_kernel.monitor_nodes node events and act accordingly. But is not so easy because of mnesia design. So still you can use the Gossip strategy, but you need to know in advance which nodes will hold mesia stuff.

Or just create your own Genserver that listens to nodes events and when you have a quorum, start mnesiac using Node.list().

jm2c

[beardedeagle]

Hrm, I have some thoughts on how this should be handled, but need to mull them over for the night or so before I'm ready to commit to a specific method and put hands to keyboard. Unfortunately, the solution won't be as simple as using Node.list/0 if we wan't to do this safely, given how mnesia prefers to operate. It is something I've been meaning to tackle for some time though (it is, in fact, a personal stretch goal of mine for 0.4.0), so I'll definitely give this priority now that it's an actual ask.

[KamilLelonek]

Thanks! Looking forward to see any solution for that. Feel free to ask for any help if needed.

[beardedeagle]

Alright, apologies, poor work/life balance has lead to some priorities slipping. Getting back to this, and I can see two strategies (much like @xadhoom mentioned):

a) have mnesiac wait until gossip quorum is achieved, then start mnesia with nodes in quorum. this can be achieved relatively easily, however it would only work for a static gossip cluster, without the ability for dynamic membership, and even then conflict resolution comes into play.

b) have mnesiac allow for nodes to be added dynamically, but this entails ensuring mnesia is properly handled during these events which could lead to loss of data during handling due to how mnesia is designed. This could be solved via queuing, etc. But again, conflict resolution comes into play.

So I think before we can solve for true dynamic membership, we first need to solve for conflict resolution of records. I've got idea's on how to handle conflict resolution, both by offering some default strategies and by allowing users to define their own. Even with that though, documentation will be key as this is an area where a ton of foot gunning can come into play.

[KamilLelonek]

We currently use b). When a node is connected, we setup a mnesia copy there.

[beardedeagle]

@KamilLelonek that's how I've used it for a couple of years now too. The issue is I don't have anything built into this library to handle this scenario SAFELY. I instead farmed this functionality out to another library I wrote at my current employer, but am working on having it released to me. That aside, I have several improvements that could be made to that library that I haven't had time to make, that could be folded into this library, or made into another library to accomplish the same thing effectively.

[beardedeagle]

I'm hoping to wrap up #53 by EOY, then I can focus on this