search

42wim / matterbridge

bridge between mattermost, IRC, gitter, xmpp, slack, discord, telegram, rocketchat, twitch, ssh-chat, zulip, whatsapp, keybase, matrix, microsoft teams, nextcloud, mumble, vk and more with REST API (mattermost not required!)
Apache License 2.0
6.43k stars 604 forks source link

IRC puppeting #667

Open qaisjp opened 5 years ago

qaisjp commented 5 years ago

This project looks amazing. I don't use this project yet. I currently have a custom setup for three-way bridging.

  1. custom software for irc <-> discord
  2. open-source bridge for slack <-> discord.

My custom bridge for Discord/IRC which has some advanced features, namely, dummy accounts! See qaisjp/go-discord-irc.

I'd like to merge this into matterbridge.

Features:

I currently have two instances of go-discord-irc deployed to two IRC servers / discord guilds:

  1. one with no custom config (so it has no spoofing, only one central listener user), and
  2. one with custom config.

The one with custom config has the following features:

This would also solve the several issues for IRC/Discord, at least when dummy user support is enabled (see list above with tagged issues).

Thoughts? I am happy to send pull requests to the project to make this a reality.

I would need: support from the community (i.e, coping with any questions I have whilst learning the codebase and implementing features).

The aim is to deprecate my project completely and for me to move completely to matterbridge.

42wim commented 5 years ago

I'd welcome support for this, but we have to do some changes to make this a reality. At the moment, bridges aren't aware of users on other bridges, so this needs to be done first before we can integrate your irc spoofing feature

patcon commented 5 years ago

This all sounds super fucking rad @qaisjp! 🎉

It might take a bit to work through all these pieces as atomic feature requests, but my experience is that if you are patient in working things in, then others (incl myself in whatever way I can) would be really eager to support :)

qaisjp commented 5 years ago

Slowly but steadily we can make this happen! 🙂

mithro commented 4 years ago

This sounds super awesome and I would love to see it happen!

qaisjp commented 4 years ago

I've been looking into this, and I'm not sure what's the best way to go about this config-wise.

Example base config to work off

Take this simple bi-directional configuration that bridges between Discord and IRC.

[discord]
[discord.compsoc]
Token="<some value>"
Server="<some value>"
WebhookURL="<some value>"

[irc]
[irc.compsoc]
Server="localhost:6697"
Nick="DiscordBot"

[[gateway]]
name="compsoc.bottest"
enable=true

[[gateway.inout]]
account="irc.compsoc"
channel="#bottest"

[[gateway.inout]]
account="discord.compsoc"
channel="bottest"

Naming

Discord already support puppets, but they are... "ephemeral". We don't want to people to use a Puppets setting and realise they actually need to use WebhookURL. That can be considered poor user experience.

We know that Discord puppets are not persistent, unlike IRC. So we could name our setting something like PersistentPuppets. Assuming that we set it to a boolean value... which we shouldn't, because....

Flow of knowledge

User information flows from discord to irc. How do we represent this information in the config?

And we probably want a configuration design that can gracefully support more than just IRC puppets.

Suggestion

What if we did something like this? I think including the term bridge might confuse people that are writing their config:

[[gateway]]
name="compsoc.bottest"
enable=true

[[gateway.bridge]]
account="irc.compsoc"
channel="#bottest"
chat="inout"
users="out"   # output

[[gateway.bridge]]
account="discord.compsoc"
channel="bottest"
chat="inout"
users="in"    # input

What do you think? How should this be designed? Please give me feedback! I want to know if there's a better way to design the config.

poVoq commented 4 years ago

Hmm, as for naming, I always considered the webhook method as user "spoofing" while what you are proposing for IRC seems to be proper user "puppeteering".

qaisjp commented 4 years ago

I only called it spoofing because when I wrote the issue I had not found a better word for it (which is 'puppeting')

jlu5 commented 4 years ago

Hi all,

Lately I've been working on an alternative implementation that allows doing this in a stateless fashion. It involves an IRCd extension and a separate command to translate relayed messages into PRIVMSGs, and allows user spoofing in a way fairly similar to e.g. Discord webhooks. Although this limits the UI slightly (virtual users can't be DMed for example because they aren't actually connected), it simplifies how much state relay bots need to track.

I've tentatively placed the details in a repo: relaymsg.md. At least one IRCd vendor has shown interest in this, so I'm hoping to finalize a draft spec soon and send it to IRCv3 for consideration. See https://github.com/ircv3/ircv3-specifications/pull/417

Justinzobel commented 3 years ago

I'm running a private bridge just for myself to bridge three IRC rooms to private Telegram rooms. Is there a way I could have IRC PMs go to the bot's PM inside Telegram and responses like discussed above? ie user2@irc: Here's my reply to your PM.

qaisjp commented 3 years ago

I'm running a private bridge just for myself to bridge three IRC rooms to private Telegram rooms. Is there a way I could have IRC PMs go to the bot's PM inside Telegram and responses like discussed above? ie user2@irc: Here's my reply to your PM.

Not at the moment

Justinzobel commented 3 years ago

I'm not at my PC at the moment. Do private messages show up in the debug log?

On Thu, 6 Aug 2020, 7:46 pm Qais Patankar, notifications@github.com wrote:

I'm running a private bridge just for myself to bridge three IRC rooms to private Telegram rooms. Is there a way I could have IRC PMs go to the bot's PM inside Telegram and responses like discussed above? ie user2@irc: Here's my reply to your PM.

Not at the moment

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/42wim/matterbridge/issues/667#issuecomment-669842253, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKOQBMEWE27AB7UGRW4YA3R7J7G5ANCNFSM4GMXSKNA .

westor7 commented 3 years ago

Will @qaisjp go-discord-irc be merged on @42wim matterbridge ? does it needs anything else to be finished?

This is a most-wanted merge :P i hope to be in the next version..

qaisjp commented 3 years ago

I've slowly been adding some of the smaller features — the long term aim is to replace my bridge entirely with matterbridge, yes

Mikaela commented 3 years ago

I am not sure if this is the right place to ask, but are there any plans to support the RELAYMSG cap mentioned above (https://github.com/42wim/matterbridge/issues/667#issuecomment-634214165) or merging the support back to upstream Matterbridge?

I have been lately using a couple of Oragono based networks which would support it and make Matterbridge more pleasant to use on them.

qaisjp commented 3 years ago

The RELAYMSG cap is nice but I still plan to work on a stateful approach that makes it possible for users to actually show up in user lists.

One problem with this stateful approach is that restarting the bridge causes a massive influx of joinquit spam. I'd like to solve this issue (in my bridge) before bringing this feature to matterbridge.

I'm thinking of solving this problem by making it possible (but not required) for a separate process of matterbridge to "manage" the irc connections.

We'd need to decide a way for matterbridge to communicate with this standalone "irc process".

The general idea is for the API/ABI to be fairly low level — examples of RPC functions would be:

The last RPC function being the magic that allows matterbridge to restart, connect to an existing matterbridge --irc-srv, and recover.

@42wim + anyone else — any comments or thoughts on the tech stack for this component?

Additionally, Wim, you've expressed reservations about keeping the "make many connections" system, and mentioned you were interested in replacing it with an ircd such as Oragono. I imagine both systems could be abstracted away from matterbridge using this same RPC-based API.

westor7 commented 3 years ago

@qaisjp One problem with this stateful approach is that restarting the bridge causes a massive influx of joinquit spam. I'd like to solve this issue (in my bridge) before bringing this feature to matterbridge.

The same is happening when netsplit servers, i think this is not a problem but its by design, you cannot avoid that stuff and i personally think its ok, modern IRCd's having configuration options to hide these joins/quits, the only thing that you can do in that case is to have a specific quit message on matterbridge restart/terminate so ircd or bots could save the nicknames and not take action after on join.

Mikaela commented 3 years ago

One problem with this stateful approach is that restarting the bridge causes a massive influx of joinquit spam.

This is currently Oragono-specific, but would it be possible for a stateful bridge to generate SSL certificates for all bridged users and then register (with) those and set the accounts into always-on mode? Then they wouldn't quit unless the Matterbridge instance they belong to was turned off for a long time (30 days by default if I recall correctly).

Another idea would be for Matterbridge to pretend to be a IRC server, but that would practically require selfhosting IRC as at least freenode won't allow any third party to link that way to my understanding. It wouldn't resolve netsplits, for which there would be https://ircv3.net/specs/extensions/batch-3.2 which again requires a compatible client. I think IRC servers also generally have different linking protocols, so it wouldn't be universal solution (and Oragono doesn't even support that currently https://github.com/oragono/oragono/issues/26).

qaisjp commented 3 years ago

The same is happening when netsplit servers

Yes, but I've observed that a genuine netsplit happens less frequently than wanting to update the bridge or the bridge crashing.

Another idea would be for Matterbridge to pretend to be a IRC server, but that would practically require selfhosting IRC as at least freenode won't allow any third party to link that way to my understanding.

Am I correct in believing that this idea is in line with the "IRCd mode" I mentioned at the very end of https://github.com/42wim/matterbridge/issues/667#issuecomment-775870347?

I think IRC servers also generally have different linking protocols, so it wouldn't be universal solution (and Oragono doesn't even support that currently oragono/oragono#26).

It's unfortunate that Oragono does not yet support S2S. "IRCd mode" would have to wait on that feature from Oragono (unless there's an existing stable ircd that supports S2S and is written in Go).

Mikaela commented 3 years ago

Am I correct in believing that this idea is in line with the "IRCd mode" I mentioned at the very end of https://github.com/42wim/matterbridge/issues/667#issuecomment-775870347?

I am not sure, I have difficulties following the comment.

westor7 commented 3 years ago

@qaisjp

The same exactly is happening with ircd/services/znc/etc.. servers, when they need update, they going down updating then going up, i think this is the most common way to make it, and it would not be a problem, also 1 or 2 maximum times per month its not huge problem.

qaisjp commented 3 years ago

I am not sure, I have difficulties following the comment.

Yeah, the last bit in my comment is referencing a prior conversation on Discord where we discussed "[adding an] irc server to matterbridge, thinking about doing that anyway in other issue [...] like matterircd, but a cleaner one [...] integrating ircd opens up "easy" irc puppeting for the other protocols too".

qaisjp commented 3 years ago

The same exactly is happening with ircd/services/znc/etc.. servers, when they need update, they going down updating then going up, i think this is the most common way to make it, and it would not be a problem, also 1 or 2 maximum times per month its not huge problem.

3:30:26 PM <oper> the point is not whether they are different but the disruption they cause
3:30:54 PM <oper> whether you use C2S or S2S, the point is to keep the IRC part in a separate process from the bridge part
3:31:28 PM <oper> so you can restart the bridge part leaving the IRC users in limbo for a period of time and then resynchronise which IRC users should exist
3:32:08 PM <oper> the ircd cannot hide the activity, only the IRC clients can do that
3:33:47 PM <oper> it's easier to do it with C2S, especially as S2S varies per ircd
42wim commented 3 years ago

We'd need to decide a way for matterbridge to communicate with this standalone "irc process".

My vote goes to net/rpc + gob. We can do this over domain sockets, don't really need TCP or TLS I think. (protobufs are a PITA to put it mildly)

singpolyma commented 3 years ago

Wouldn't restarting the irc specific service still result in the big joinquit? Is the idea just that one might restart matterbridge more often?

qaisjp commented 3 years ago

Is the idea just that one might restart matterbridge more often?

yes

poVoq commented 2 years ago

@qaisjp I guess the plan to integrate your go-discord-irc bridge with Matterbridge is postponed indefinitely?

qaisjp commented 2 years ago

Yes, these days I have not had much motivation to work on open source.

ajar commented 1 year ago

I'm game to take a crack at this one. It will be very useful for our community which still spans multiple services, including IRC which has been active since around 2013. I'll keep this thread updated.

poVoq commented 1 year ago

You could also try: https://github.com/OpenTTD/dibridge

ajar commented 1 year ago

You could also try: https://github.com/OpenTTD/dibridge

Oh neat, that is very interesting indeed. Thanks for sharing this :-) We'll survey the options for this, and glad to see we're not the only ones in this predicament (misery loves company).