Open wolfen351 opened 2 years ago
Here are some logs to show what I mean:
Game Container:
[23:49:02] [Server thread/WARN]: Can't keep up! Is the server overloaded? Running 11949ms or 238 ticks behind
[Autopause loop] No client connected since startup / knocked - stopping
[23:53:34] [RCON Listener #1/INFO]: Thread RCON Client /0:0:0:0:0:0:0:1 started
[23:53:34] [Server thread/INFO]: [Rcon: Saved the game]
[23:53:34] [RCON Client /0:0:0:0:0:0:0:1 #5/INFO]: Thread RCON Client /0:0:0:0:0:0:0:1 shutting down
[2022-05-04T23:53:34+00:00] [Autopause] Pausing Java process
Backup Container:
2022-05-04T23:47:20+0000 INFO waiting initial delay of 1m...
2022-05-04T23:48:20+0000 INFO waiting for rcon readiness...
2022-05-04T23:48:20+0000 ERROR Unable to execute rcon-cli save-on - try 0/5. Retrying in 10s
2022-05-04T23:48:20+0000 ERROR Failure reason: 2022/05/04 23:48:20 Failed to connect to RCON serverdial tcp [::1]:25575: connect: connection refused
2022-05-04T23:48:30+0000 ERROR Unable to execute rcon-cli save-on - try 1/5. Retrying in 10s
2022-05-04T23:48:31+0000 ERROR Failure reason: 2022/05/04 23:48:30 Failed to connect to RCON serverdial tcp [::1]:25575: connect: connection refused
2022-05-04T23:48:41+0000 INFO Command executed successfully rcon-cli save-on
2022-05-04T23:48:41+0000 INFO Command executed successfully rcon-cli save-off
2022-05-04T23:49:00+0000 INFO Command executed successfully rcon-cli save-all flush
2022-05-04T23:49:00+0000 INFO Command executed successfully sync
2022-05-04T23:49:00+0000 INFO Backing up content in /data to /backups/world-20220504-234900.tgz
tar: ./world: file changed as we read it
2022-05-04T23:57:40+0000 WARN tar exited with code 1. Ignoring
2022-05-04T23:57:50+0000 ERROR Unable to execute rcon-cli save-on - try 0/5. Retrying in 10s
2022-05-04T23:57:50+0000 ERROR Failure reason: 2022/05/04 23:57:50 Failed to connect to RCON serverread tcp [::1]:44092->[::1]:25575: i/o timeout
2022-05-04T23:58:10+0000 ERROR Unable to execute rcon-cli save-on - try 1/5. Retrying in 10s
2022-05-04T23:58:10+0000 ERROR Failure reason: 2022/05/04 23:58:10 Failed to connect to RCON serverread tcp [::1]:44094->[::1]:25575: i/o timeout
2022-05-04T23:58:30+0000 ERROR Unable to execute rcon-cli save-on - try 2/5. Retrying in 10s
2022-05-04T23:58:30+0000 ERROR Failure reason: 2022/05/04 23:58:30 Failed to connect to RCON serverread tcp [::1]:44096->[::1]:25575: i/o timeout
2022-05-04T23:58:50+0000 ERROR Unable to execute rcon-cli save-on - try 3/5. Retrying in 10s
2022-05-04T23:58:50+0000 ERROR Failure reason: 2022/05/04 23:58:50 Failed to connect to RCON serverread tcp [::1]:44098->[::1]:25575: i/o timeout
2022-05-04T23:59:10+0000 ERROR Unable to execute rcon-cli save-on - try 4/5. Retrying in 10s
2022-05-04T23:59:10+0000 ERROR Failure reason: 2022/05/04 23:59:10 Failed to connect to RCON serverread tcp [::1]:44100->[::1]:25575: i/o timeout
2022-05-04T23:59:30+0000 ERROR Unable to execute rcon-cli save-on - try 5/5. Retrying in 10s
2022-05-04T23:59:30+0000 ERROR Failure reason: 2022/05/04 23:59:30 Failed to connect to RCON serverread tcp [::1]:44102->[::1]:25575: i/o timeout
2022-05-04T23:59:50+0000 ERROR Unable to execute rcon-cli save-on - try 0/5. Retrying in 5s
2022-05-04T23:59:50+0000 ERROR Failure reason: 2022/05/04 23:59:50 Failed to connect to RCON serverread tcp [::1]:44106->[::1]:25575: i/o timeout
2022-05-05T00:00:05+0000 ERROR Unable to execute rcon-cli save-on - try 1/5. Retrying in 5s
2022-05-05T00:00:05+0000 ERROR Failure reason: 2022/05/05 00:00:05 Failed to connect to RCON serverread tcp [::1]:44108->[::1]:25575: i/o timeout
2022-05-05T00:00:20+0000 ERROR Unable to execute rcon-cli save-on - try 2/5. Retrying in 5s
2022-05-05T00:00:20+0000 ERROR Failure reason: 2022/05/05 00:00:20 Failed to connect to RCON serverread tcp [::1]:44110->[::1]:25575: i/o timeout
2022-05-05T00:00:35+0000 ERROR Unable to execute rcon-cli save-on - try 3/5. Retrying in 5s
2022-05-05T00:00:35+0000 ERROR Failure reason: 2022/05/05 00:00:35 Failed to connect to RCON serverread tcp [::1]:44112->[::1]:25575: i/o timeout
2022-05-05T00:00:50+0000 ERROR Unable to execute rcon-cli save-on - try 4/5. Retrying in 5s
2022-05-05T00:00:50+0000 ERROR Failure reason: 2022/05/05 00:00:50 Failed to connect to RCON serverread tcp [::1]:44114->[::1]:25575: i/o timeout
2022-05-05T00:01:05+0000 ERROR Unable to execute rcon-cli save-on - try 5/5. Retrying in 5s
2022-05-05T00:01:05+0000 ERROR Failure reason: 2022/05/05 00:01:05 Failed to connect to RCON serverread tcp [::1]:44116->[::1]:25575: i/o timeout```
I did some more digging, and it appears the rcon port IS configured for knocking:
root@minecraft-freecrafting-game-deployment-58dc585565-ctgx5:/# cat /tmp/knockd-config.cfg
[options]
logfile = /dev/null
[unpauseMCServer-server]
sequence = 25565
seq_timeout = 1
command = /autopause/resume.sh
tcpflags = syn
[unpauseMCServer-rcon]
sequence = 25575
seq_timeout = 1
command = /autopause/resume.sh
tcpflags = syn
[unpauseMCServer-bedrock]
sequence = 19132:udp
command = /autopause/resume.sh
However the backup container still crashes instead of making the backup :/
Can you provide your container configs, such as compose file? It looks like you went with the option to run the two containers in the same networking namespace. That's the approach I recommend; however, knockd isn't listening on the loopback interface.
Knockd support was a contributed feature so I would need to research if that can be configured to listen to multiple/all interfaces.
Thanks, that was the piece of info I needed to fix my issue, much appreciated! I deployed them in the same networking namespace, however I did it in k8s and not docker compose. Knowing that it does not listen on lo0, I made a k8s service to handle the inter container communications of RCON and then used the internal k8s dns entry for the communication. This resolved the problem for me.
The changes to fix this issue were exposing the RCON port in the load balancer service, and then using the service name in the RCON_HOST env variable.
For reference, here is my working config:
apiVersion: v1
kind: Service
metadata:
name: minecraft-freecrafting-game-service
spec:
type: LoadBalancer
ports:
- name: game
port: 25565
- name: rcon
port: 25575
selector:
app: minecraft-freecrafting-game
---
apiVersion: v1
kind: Service
metadata:
name: minecraft-freecrafting-game-voice-service
spec:
type: LoadBalancer
ports:
- name: voice
port: 24454
protocol: UDP
selector:
app: minecraft-freecrafting-game
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: minecraft-freecrafting-game-deployment
labels:
app: minecraft-freecrafting-game
spec:
replicas: 1
selector:
matchLabels:
app: minecraft-freecrafting-game
template:
metadata:
labels:
app: minecraft-freecrafting-game
spec:
imagePullSecrets:
- name: registry-credentials
# Add the server as an NFS volume for the pod
volumes:
- name: nfs-volume
nfs:
# URL for the NFS server
server: xxxx
path: /k3s/minecraft-freecrafting
- name: nfs-volume-bk
nfs:
server: xxxx
path: /k3s/minecraft-freecrafting-backups
containers:
- name: minecraft-freecrafting-game
image: itzg/minecraft-server:latest
imagePullPolicy: Always
ports:
- containerPort: 25575 # rcon
- containerPort: 25565 # game
- containerPort: 24454 # voicechat
protocol: UDP
# Mount the NFS volume in the container
volumeMounts:
- name: nfs-volume
mountPath: /data
env:
- name: EULA
value: "TRUE"
- name: TYPE
value: "FABRIC"
- name: MEMORY
value: "1500M"
- name: VERSION
value: "1.18.1"
- name: ENABLE_AUTOPAUSE
value: "TRUE"
- name: AUTOPAUSE_TIMEOUT_EST
value: "300"
- name: AUTOPAUSE_TIMEOUT_INIT
value: "300"
- name: minecraft-freecrafting-backup
image: itzg/mc-backup
imagePullPolicy: Always
# Mount the NFS volume in the container
volumeMounts:
- name: nfs-volume
mountPath: /data
- name: nfs-volume-bk
mountPath: /backups
env:
- name: RCON_HOST
value: minecraft-freecrafting-game-service
- name: BACKUP_INTERVAL
value: "1d"
- name: INITIAL_DELAY
value: "5h"
- name: PRUNE_BACKUPS_DAYS
value: "14"
If you'd like, you could add that to the documentation on how to deploy these 2 containers together into a kubernetes server
Thanks. Yes, that would be a good note to add to the docs.
I've been struggling with this on and off now for a bit, and I've finally narrowed the crashes of this container to the fact that the main game container is "paused". Could we add the rcon connection port to the wake up sequence? So the game can be woken up for backups?