bridge between mattermost, IRC, gitter, xmpp, slack, discord, telegram, rocketchat, twitch, ssh-chat, zulip, whatsapp, keybase, matrix, microsoft teams, nextcloud, mumble, vk and more with REST API (mattermost not required!)
Apache License 2.0
6.66k
stars
618
forks
source link
Matterbridge gets into endless loop of trying to connect to Discord after failure, causing bot token to be revoked #2158
I don't quite understand what's happening but it appears that occasionally matterbridge will fail to connect to Discord due to a connection issue. Once that happens, the logs get flooded with the same message over and over about failure to send heartbeat to Discord. Then presumably starting from the time the bot token was reset by Discord for all these repeated failed attempts, then it enters an endless loop where it successfully connects to everything except Discord and since that failed, it restarts and tries again. It will never succeed because the bot token was revoked.
So the main issue is that simply failing to connect even once is enough for it to enter a bad state that will inevitably cause me to have to go through the annoying process of getting a new bot token, adding the bot to the channels, etc. And the second issue that it keeps trying even though it will never succeed. I would rather matterbridge just shut down when this happens. But more importantly, it needs to not enter the endless loop that causes the bot token to be revoked.
This is what the log looks like each time this happens:
Jun 5 03:03:25 synapse matterbridge[92386]: 2024/06/05 03:03:25 [DG0] wsapi.go:302:heartbeat() error sending heartbeat to gateway wss://gateway.discord.gg/?v=8&encoding=json, write tcp 10.0.0.2:28996->162.159.136.234:443: write: broken pipe
Jun 5 03:03:32 synapse matterbridge[92386]: 2024/06/05 03:03:32 [DG0] wsapi.go:302:heartbeat() error sending heartbeat to gateway wss://gateway.discord.gg/?v=8&encoding=json, websocket: close sent
Jun 5 03:03:37 synapse matterbridge[92386]: 2024/06/05 03:03:37 [DG0] wsapi.go:302:heartbeat() error sending heartbeat to gateway wss://gateway.discord.gg/?v=8&encoding=json, websocket: close sent
Jun 5 03:03:50 synapse matterbridge[92386]: 2024/06/05 03:03:49 [DG0] wsapi.go:302:heartbeat() error sending heartbeat to gateway wss://gateway.discord.gg/?v=8&encoding=json, websocket: close sent
Jun 5 03:04:13 synapse matterbridge[92386]: 2024/06/05 03:04:13 [DG0] wsapi.go:302:heartbeat() error sending heartbeat to gateway wss://gateway.discord.gg/?v=8&encoding=json, websocket: close sent
Jun 5 03:04:41 synapse matterbridge[92386]: 2024/06/05 03:04:41 [DG0] wsapi.go:302:heartbeat() error sending heartbeat to gateway wss://gateway.discord.gg/?v=8&encoding=json, websocket: close sent
Jun 5 03:04:44 synapse matterbridge[92386]: 2024/06/05 03:04:44 [DG0] wsapi.go:302:heartbeat() error sending heartbeat to gateway wss://gateway.discord.gg/?v=8&encoding=json, websocket: close sent
Jun 5 03:04:53 synapse matterbridge[92386]: 2024/06/05 03:04:53 [DG0] wsapi.go:302:heartbeat() error sending heartbeat to gateway wss://gateway.discord.gg/?v=8&encoding=json, websocket: close sent
Jun 5 03:04:59 synapse matterbridge[92386]: 2024/06/05 03:04:59 [DG0] wsapi.go:302:heartbeat() error sending heartbeat to gateway wss://gateway.discord.gg/?v=8&encoding=json, websocket: close sent
Jun 5 03:05:05 synapse matterbridge[92386]: 2024/06/05 03:05:05 [DG0] wsapi.go:302:heartbeat() error sending heartbeat to gateway wss://gateway.discord.gg/?v=8&encoding=json, websocket: close sent
Jun 5 03:05:09 synapse matterbridge[92386]: 2024/06/05 03:05:09 [DG0] wsapi.go:302:heartbeat() error sending heartbeat to gateway wss://gateway.discord.gg/?v=8&encoding=json, websocket: close sent
Jun 5 03:05:15 synapse matterbridge[92386]: 2024/06/05 03:05:15 [DG0] wsapi.go:302:heartbeat() error sending heartbeat to gateway wss://gateway.discord.gg/?v=8&encoding=json, websocket: close sent
Jun 5 03:05:21 synapse matterbridge[92386]: 2024/06/05 03:05:21 [DG0] wsapi.go:302:heartbeat() error sending heartbeat to gateway wss://gateway.discord.gg/?v=8&encoding=json, websocket: close sent
Jun 5 03:05:24 synapse matterbridge[92386]: 2024/06/05 03:05:24 [DG0] wsapi.go:302:heartbeat() error sending heartbeat to gateway wss://gateway.discord.gg/?v=8&encoding=json, websocket: close sent
Jun 5 03:05:30 synapse matterbridge[92386]: 2024/06/05 03:05:30 [DG0] wsapi.go:302:heartbeat() error sending heartbeat to gateway wss://gateway.discord.gg/?v=8&encoding=json, websocket: close sent
Jun 5 03:05:39 synapse matterbridge[92386]: 2024/06/05 03:05:39 [DG0] wsapi.go:302:heartbeat() error sending heartbeat to gateway wss://gateway.discord.gg/?v=8&encoding=json, websocket: close sent
Jun 5 03:05:44 synapse matterbridge[92386]: 2024/06/05 03:05:44 [DG0] wsapi.go:302:heartbeat() error sending heartbeat to gateway wss://gateway.discord.gg/?v=8&encoding=json, websocket: close sent
Jun 5 03:05:49 synapse matterbridge[92386]: 2024/06/05 03:05:49 [DG0] wsapi.go:302:heartbeat() error sending heartbeat to gateway wss://gateway.discord.gg/?v=8&encoding=json, websocket: close sent
Jun 5 03:06:34 synapse matterbridge[92386]: 2024/06/05 03:06:34 [DG0] wsapi.go:302:heartbeat() error sending heartbeat to gateway wss://gateway.discord.gg/?v=8&encoding=json, write tcp 10.0.0.2:42984->162.159.136.234:443: write: broken pipe
Jun 5 03:06:34 synapse matterbridge[92386]: 2024/06/05 03:06:34 [DG0] wsapi.go:302:heartbeat() error sending heartbeat to gateway wss://gateway.discord.gg/?v=8&encoding=json, websocket: close sent
Jun 5 03:06:36 synapse matterbridge[92386]: 2024/06/05 03:06:36 [DG0] wsapi.go:302:heartbeat() error sending heartbeat to gateway wss://gateway.discord.gg/?v=8&encoding=json, websocket: close sent
Jun 5 03:06:40 synapse matterbridge[92386]: 2024/06/05 03:06:40 [DG0] wsapi.go:302:heartbeat() error sending heartbeat to gateway wss://gateway.discord.gg/?v=8&encoding=json, websocket: close sent
Jun 5 03:07:38 synapse matterbridge[92386]: 2024/06/05 03:07:38 [DG0] wsapi.go:302:heartbeat() error sending heartbeat to gateway wss://gateway.discord.gg/?v=8&encoding=json, websocket: close sent
Jun 5 03:07:39 synapse matterbridge[92386]: 2024/06/05 03:07:39 [DG0] wsapi.go:302:heartbeat() error sending heartbeat to gateway wss://gateway.discord.gg/?v=8&encoding=json, websocket: close sent
Jun 5 03:07:43 synapse matterbridge[92386]: 2024/06/05 03:07:43 [DG0] wsapi.go:302:heartbeat() error sending heartbeat to gateway wss://gateway.discord.gg/?v=8&encoding=json, websocket: close sent
Then it keeps going with just that last line, "close sent" over and over again 1000s of times. It's very non-deterministic. I've gone months without this happening. Seems like it happens more lately. I wonder if Discord is less reliable. I'm certain there's nothing wrong with my matterbridge server's connection. It just doesn't handle not being able to connect well.
I don't quite understand what's happening but it appears that occasionally matterbridge will fail to connect to Discord due to a connection issue. Once that happens, the logs get flooded with the same message over and over about failure to send heartbeat to Discord. Then presumably starting from the time the bot token was reset by Discord for all these repeated failed attempts, then it enters an endless loop where it successfully connects to everything except Discord and since that failed, it restarts and tries again. It will never succeed because the bot token was revoked.
So the main issue is that simply failing to connect even once is enough for it to enter a bad state that will inevitably cause me to have to go through the annoying process of getting a new bot token, adding the bot to the channels, etc. And the second issue that it keeps trying even though it will never succeed. I would rather matterbridge just shut down when this happens. But more importantly, it needs to not enter the endless loop that causes the bot token to be revoked.
This is what the log looks like each time this happens:
Then it keeps going with just that last line, "close sent" over and over again 1000s of times. It's very non-deterministic. I've gone months without this happening. Seems like it happens more lately. I wonder if Discord is less reliable. I'm certain there's nothing wrong with my matterbridge server's connection. It just doesn't handle not being able to connect well.
Environment info:
Config file section for discord: