search

museofficial / muse

🎧 a self-hosted midwestern Discord music bot that doesn't suck
MIT License
894 stars 236 forks source link

Bot not receving heartbeat after a few hours of being online #905

Open JakeTheInsane opened 1 year ago

JakeTheInsane commented 1 year ago

What's happening? I run this bot in 2 discord servers for friends, no matter if we're using the bot or not, it seems to lose the discord heartbeat after a few hours, but continues to receive the API reply from Spotify every 30 minutes. Continuing to get the API reply from Spotify makes me assume that the bot has not crashed.

I'm running this within Docker, on a Debian 11 VM, which is running on Proxmox Virtual Environment 7.3-3 (more version info below)

Logs

  muse [WS => Shard 0] [HeartbeatTimer] Sending a heartbeat. +41s
  muse [WS => Shard 0] Heartbeat acknowledged, latency of 42ms. +42ms
  superagent POST https://accounts.spotify.com/api/token +30m
  superagent POST https://accounts.spotify.com/api/token +1ms
  superagent POST https://accounts.spotify.com/api/token +24ms
  superagent POST https://accounts.spotify.com/api/token -> 200 +145ms
  superagent POST https://accounts.spotify.com/api/token -> 200 +7ms
  superagent POST https://accounts.spotify.com/api/token -> 200 +12ms
  muse [WS => Shard 0] [HeartbeatTimer] Sending a heartbeat. +41s
  muse [WS => Shard 0] Heartbeat acknowledged, latency of 45ms. +45ms
  muse [WS => Shard 0] [HeartbeatTimer] Sending a heartbeat. +41s
  muse [WS => Shard 0] Heartbeat acknowledged, latency of 43ms. +44ms
  muse [WS => Shard 0] [HeartbeatTimer] Sending a heartbeat. +41s
  muse [WS => Shard 0] Heartbeat acknowledged, latency of 42ms. +42ms
  muse [WS => Shard 0] [HeartbeatTimer] Sending a heartbeat. +41s
  muse [WS => Shard 0] Heartbeat acknowledged, latency of 42ms. +42ms
  muse [WS => Shard 0] [HeartbeatTimer] Sending a heartbeat. +41s
  muse [WS => Shard 0] Heartbeat acknowledged, latency of 50ms. +50ms
  muse [WS => Shard 0] [HeartbeatTimer] Sending a heartbeat. +41s
  muse [WS => Shard 0] Heartbeat acknowledged, latency of 42ms. +42ms
  muse [WS => Shard 0] [HeartbeatTimer] Sending a heartbeat. +41s
  muse [WS => Shard 0] [HeartbeatTimer] Didn't receive a heartbeat ack last time, assuming zombie connection. Destroying and reconnecting.
  muse     Status          : 0
  muse     Sequence        : 19
  muse     Connection State: OPEN +41s
  muse [WS => Shard 0] [DESTROY]
  muse     Close Code    : 4009
  muse     Reset         : true
  muse     Emit DESTROYED: true +0ms
  muse [WS => Shard 0] Clearing the heartbeat interval. +0ms
  muse [WS => Shard 0] [WebSocket] Destroy: Attempting to close the WebSocket. | WS State: OPEN +1ms
  muse [WS => Shard 0] [WebSocket] Close: Tried closing. | WS State: CLOSING +0ms
  muse [WS => Shard 0] [WebSocket] Adding a WebSocket close timeout to ensure a correct WS reconnect.
  muse         Timeout: 5000ms +0ms
  muse [WS => Shard 0] [WebSocket] Clearing the close timeout. +5s
  muse [WS => Shard 0] [WebSocket] Close Emitted: false +0ms
  muse [WS => Shard 0] [WebSocket] did not close properly, assuming a zombie connection.
  muse Emitting close and reconnecting again. +0ms
  muse [WS => Shard 0] [CLOSE]
  muse     Event Code: 1011
  muse     Clean     : false
  muse     Reason    : INTERNAL_ERROR +0ms
  muse [WS => Shard 0] [CONNECT]
  muse     Gateway    : wss://gateway.discord.gg/
  muse     Version    : 10
  muse     Encoding   : json
  muse     Compression: none +1ms
  muse [WS => Shard 0] Setting a HELLO timeout for 20s. +0ms
  muse [WS => Shard 0] [CONNECTED] Took 165ms +165ms
  muse [WS => Shard 0] Clearing the HELLO timeout. +0ms
  muse [WS => Shard 0] Setting a heartbeat interval for 41250ms. +0ms
  muse [WS => Shard 0] [IDENTIFY] Shard 0/1 with intents: 1153 +0ms
  muse [WS => Shard 0] [READY] Session 8f4fad4def88538e1fd3db583da1c50c. +172ms
  muse [WS => Shard 0] [ReadyHeartbeat] Sending a heartbeat. +0ms
  muse [WS => Shard 0] Shard received all its guilds. Marking as fully ready. +37ms
  muse [WS => Shard 0] Heartbeat acknowledged, latency of 50ms. +13ms
  muse [WS => Shard 0] Clearing the heartbeat interval. +25s
  muse [WS => Shard 0] [CLOSE]
  muse     Event Code: 1006
  muse     Clean     : false
  muse     Reason    :  +1ms
  muse [WS => Shard 0] Session id is present, attempting an immediate reconnect... +0ms
  muse [WS => Shard 0] An open connection was found, attempting an immediate identify. +0ms
  muse [WS => Shard 0] [RESUME] Session 8f4fad4def88538e1fd3db583da1c50c, sequence 3 +0ms
  superagent POST https://accounts.spotify.com/api/token +30m
  superagent POST https://accounts.spotify.com/api/token +2ms
  superagent POST https://accounts.spotify.com/api/token +1ms
  superagent POST https://accounts.spotify.com/api/token -> 200 +170ms
  superagent POST https://accounts.spotify.com/api/token -> 200 +1ms
  superagent POST https://accounts.spotify.com/api/token -> 200 +21ms
  superagent POST https://accounts.spotify.com/api/token +30m
  superagent POST https://accounts.spotify.com/api/token +2ms
  superagent POST https://accounts.spotify.com/api/token +19ms
  superagent POST https://accounts.spotify.com/api/token -> 200 +147ms
  superagent POST https://accounts.spotify.com/api/token -> 200 +2ms
  superagent POST https://accounts.spotify.com/api/token -> 200 +72ms

Screenshots N/A

Additional context Bot runs perfectly fine in any other circumstance, as soon as I restart the docker container it works fine for another few hours

Runtime I'm running Muse:

Versions

kunumigab commented 1 year ago

I have the same problem, but hosting on arm/oracle using portainer.

For a fix i set the bot config on

Wait before leaving after queue empty: never leave Leave if there are no listeners: no And leave the bot connect on a vc

s-rog commented 1 year ago

experiencing this as well, bot randomly goes offline after a while (sometimes days)

ohmstance commented 1 year ago

I'm not sure if it's related but I'm running the bot in Docker and if internet connection is interrupted the bot goes offline on Discord and doesn't reconnect. However, the bot does not crash.

s-rog commented 1 year ago

I think the bot goes offline when its IP changes, I moved muse to a static IP and so far it has survived routine minor disconnections.