Proxy to hand off bridge connections to, allowing restarts of the "main" process.

[Half-Shot]

We've investigated this work a bunch of times, with things like https://github.com/half-shot/irc-conntrack over the years but it's never quite taken off due to the amount of invasive work required.

With that in mind, we're thinking of perhaps doing a simpler approach involving implementing SOCKSv5 for node-irc, and a simple proxy server that can sit there handling connections. It's not IRC aware and wouldn't solve all of our problems, but would decrease restart noise and startup speed.

• node-irc PR https://github.com/matrix-org/node-irc/pull/99
• matrix-appservice-irc PR https://github.com/matrix-org/matrix-appservice-irc/pull/1669

[progval]

Note that either the bridge or the proxy need to either keep track of state somewhere, or send WHO requests in order to repopulate the state; as you wouldn't get RPL_NAMREPLY automatically on joins. Though it's still an improvement for both the bridge's startup speed and the IRC network's bandwidth, as only one WHO request per channel will be needed no matter how many users are in it.

[Half-Shot]

It would be expected that we'd offload the state still, as the Client in node-irc is veeeerry stateful. That's a easy problem to solve though, we can have the IRC bridge periodically cache the current state of things to postgres.

Or alternatively, we can just run the right IRC commands to discover state about the channels and users the client is talking to (as you say, WHO) as we see data arrive for that connection. It's probably even a good thing to do to cut down the bridge's overall memory footprint.

[Half-Shot]

Having spent a few days on this, there is some progress. I've:

• Dropped the idea of doing SOCKS. Nice idea, but ultimately didn't provide the flexibility, and felt like the wrong way to think about the problem.
• Separated the node-irc Client state from logic, so you can now pass in your own state at creation time to "resume" a client's session.
• Built a client pooling system attached to Redis that creates and maintains connections in it's own process.
• Built a "fake" client that talks to our pool to (re)establish connections and pass data around.
• Built a state store backed by Redis for each connection.

There is a ton of leftover cleanup work, but today we got enough of this working that I feel like the idea has legs. There are concerns over performance of using Redis for something like this, although I think it's worth trying and seeing first. If need be we can do something even more low level for the data channel between client <-> connection, but at this stage let's not overcomplicate.

[progval]

Why would Redis performance be an issue? You don't even need it to flush to disk as a system reboot would close the sockets anyway (unless you're planning to have TLS resumption as well?)

[Half-Shot]

Nah, if the underlying host goes then all bets are off. We're explicitly solving the case of "I want to restart my IRC bridge to apply minor updates without spamming", which I think we're happy with. And yeah, so far perf looks good.

[Half-Shot]

Merged and deployed to libera.chat for now. We'll continue to make improvements as we go.