search

bitwalker / swarm

Easy clustering, registration, and distribution of worker processes for Erlang/Elixir
MIT License
1.21k stars 103 forks source link

Incorrect statement about syn in README #135

Closed ostinelli closed 3 years ago

ostinelli commented 5 years ago

Hello, In your README you are stating the following:

I also attempted to use syn, but because it uses mnesia, dynamic node membership as a requirement means it's dead on arrival for my use case.

Since you're writing this in your repo's README, and I've seen other people react to this and write in forums, I need to clarify that your statement paints a wrong picture.

syn supports dynamic node addition very well. A node can be added to an existing cluster and start syncing all of its data. The only caveat is, as stated in syn's README, that you need to have the node connected to the cluster before you initialize syn. That is all.

Hence, I'm not sure what you mean with "dynamic node membership", but if it's the addition / removal of nodes, this has always been supported. If you are curious on how this works, you can see it here.

Please rectify your statement so that it doesn't generate misunderstanding in syn's features, such as the one that I had to clarify here.

Best, r.

ostinelli commented 4 years ago

Update: syn v2 is not based on mnesia anymore, and all limitations re addition/removal of nodes are gone.

bitwalker commented 4 years ago

@ostinelli I'm happy to add a note about v2 addressing those issues.

Cluster topologies with nodes that come and go under normal conditions (e.g. in response to auto-scaling, or that appear dynamic due to nodes always coming online with fresh/unique names) do not play well with Mnesia in general, and quickly become problematic when actual network issues occur, e.g. partitions. Mnesia was really was designed for clusters composed of a fixed set of nodes, where membership changes are carefully coordinated. Even when Mnesia is run that way, it is prone to issues when netsplits occur. Naturally, this doesn't reflect on syn, but it does necessarily inherit those issues by building on it.

So if we're defining syn's compatibility with dynamic cluster membership based on the support provided by Mnesia, then no, I don't agree that it supports that environment well at all. That said, since syn no longer relies on Mnesia, that definition is irrelevant, so its appropriate to update the comparison here.

If you are up to it, I'm happy to merge a PR that updates the comparison as you see fit.

ostinelli commented 4 years ago

Hi @bitwalker, I understand where you're coming from, as I'm familiar with the genesis of mnesia. However, I've used syn v1 for years in autoscaling setups which did experience netsplits (syn has resolution mechanisms), and I can confirm that it is possible to use dynamic node membership with it. Successfully.

Now, I do agree with you that there are some quirks, which I wrote about and that ultimately were one of the reasons to remove mnesia completely from syn v2, but it doesn't mean that it could not be done, something that your statement implies.

Of course you're entitled to your opinion, but I simply do not think it's appropriate to have such an opinionated statement on a library inside of the README of another library, where it cannot be discussed. That's all I've been saying.

ostinelli commented 3 years ago

@bitwalker could you please just remove the incorrect statement about syn in your README.

I also attempted to use syn, but because it uses mnesia, dynamic node membership as a requirement means it's dead on arrival for my use case.

Syn hasn't been using mnesia for years. There isn't a need for a PR, just a removal of a line. Thank you.

bitwalker commented 3 years ago

Closed via #148, thanks @arjan!