Network Support

[prologic]

In another issue (don't recall which one) you mentioned that you run a small IRC network with ergonomadic -- How do you do this? Is there support for server linking and some kind of server <-> server protocol builtin that I'm not seeing?

[edmund-huber]

Nope, there is no multi-server support at all. That's what https://github.com/edmund-huber/ergonomadic/issues/2 is about.

[prologic]

I see -- I could be wrong but I see "federation" #2 more of a identity/authorization thing. Supporting some kind of multi-server thing should probably be a separate issue? have you thought about what server<->server protocol you'd like to see implemented?

[prologic]

Alternatively have we considered just implementing (effectively) clustering support via raft and call it a day? This gives us the added advantage of not having to implement a traditional IRC server<->server protocol whilst at the same time maintaining a cluster (network of servers) to share network I/O load, tolerate failover, and may even handle network partitions too?

[edmund-huber]

So let's say what we're trying to solve is that if a client connects to one server in the network, it is equivalent to connecting to any other server in the network. All servers have the same state -- same messages, same users (ops, voice, etc). The protocol must also handle servers entering and exiting the network.

Raft might not be a bad way to do it. It sounds like what is involved (based on 5 minutes of reading) is figuring out what to put in the blobs that form the shared log. Maybe that blob just needs to contain info like:

{
  users: [
    {nick: ed, mode: abcd},
    {nick: prologic, mode: efg},
    ..
  ],
  messages: [
    {nick_or_chan: #bla, text: "bla bla bla"},
   ..
  ]
}

Since in the strawman I just mentioned, all servers are aware of all channel activity (and privmsg), there must also be a way for servers to authenticate themselves to the network. We wouldn't want someone to be joining the raft group and seeing everything that's going on.

Can you think more about how it would work for raft to be the consensus/network protocol?

[prologic]

Yeah this makes sense except for the "blob" part -- from my own research into Raft you'd basically build a set of "primitives" (I believe) that form the log; these would be (for example):

• {"event": "user_connected", "data": {...}}
• {"event": "channel_created", "data": {....}}

etc... I think; the consensus here being that all members of the network agree on the same state. each node would back this by (for example) a boltdb for persistance that can be replayed on startup.

Pretty sure there is some form of "authentication" in Raft (need to look into that) so that shouldn't be hard to solve per se.

I mean let's face it -- traditional IRC servers of the day were basically solving consensus / distributed computing stuff really -- Raft just seems ideal here.

[prologic]

In fact... It looks like RobustIRC internally uses Raft. So this actually isn't as crazy as it first sounds. This may in fact be the best way today to support a "clustered IRC network".