search

jammsen / docker-palworld-dedicated-server

Docker container to easily provision and manage Palworld Dedicated Server
https://hub.docker.com/r/jammsen/palworld-dedicated-server
MIT License
898 stars 151 forks source link

Large quantity of backup jobs being generated #204

Closed Schuyler113 closed 4 months ago

Schuyler113 commented 4 months ago

Have you read the Important information text above

Current behavior

I am currently having issues where the backup manager seems to be issuing a never ending list of backup jobs. Today it has generated ~100gb in backup files. Restarting the system had no impact.

Desired behavior

Single backup job per hour

Links to screenshots

time="2024-02-15T14:12:26-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=965 job.schedule="0 * * * *" time="2024-02-15T14:12:26-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=965 job.schedule="0 * * * *" time="2024-02-15T14:12:26-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=611 job.schedule="0 * * * *" time="2024-02-15T14:12:26-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=611 job.schedule="0 * * * *" time="2024-02-15T14:12:26-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=630 job.schedule="0 * * * *" time="2024-02-15T14:12:26-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=630 job.schedule="0 * * * *" time="2024-02-15T14:12:26-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=199 job.schedule="0 * * * *" time="2024-02-15T14:12:26-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=199 job.schedule="0 * * * *" time="2024-02-15T14:12:26-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=223 job.schedule="0 * * * *" time="2024-02-15T14:12:26-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=223 job.schedule="0 * * * *" time="2024-02-15T14:12:26-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=1408 job.schedule="0 * * * *" time="2024-02-15T14:12:26-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=1408 job.schedule="0 * * * *" time="2024-02-15T14:12:26-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=1453 job.schedule="0 * * * *" time="2024-02-15T14:12:26-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=1453 job.schedule="0 * * * *" time="2024-02-15T14:12:26-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=969 job.schedule="0 * * * *" time="2024-02-15T14:12:26-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=969 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=1361 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=1361 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=496 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=496 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=224 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=224 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=403 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=403 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=1304 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=1304 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=598 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=598 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=1215 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=1215 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=1490 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=1490 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=1102 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=1102 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=759 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=759 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=1070 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=1070 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=1429 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=1429 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=1098 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=1098 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=1109 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=1109 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=1260 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=1260 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=1103 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=1103 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=889 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=889 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=343 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=343 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=1157 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=1157 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=190 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=190 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=1090 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=1090 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=34 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=34 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=582 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=582 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=443 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=443 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=1426 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=1426 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=468 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=468 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=1232 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=1232 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=1518 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=1518 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=1068 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=1068 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=981 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=981 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=1065 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=1065 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=152 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=152 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=779 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=779 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=568 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=568 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=32 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=32 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=22 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=22 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=621 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=621 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=1319 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=1319 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=516 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=516 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=947 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=947 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=837 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=837 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg=">>> Done" channel=stdout iteration=3 job.command=/backupmanager.sh job.position=973 job.schedule="0 * * * *" time="2024-02-15T14:12:27-05:00" level=info msg="job succeeded" iteration=3 job.command=/backupmanager.sh job.position=973 job.schedule="0 * * * *"

To Reproduce

Steps to reproduce the behavior: Begins on the next hour after launching the server

Software setup

Hardware setup

Additional context

No response

Schuyler113 commented 4 months ago

I am not incredibly familiar with how cron jobs work, but I did find that the cronlist file had over 2000 lines of "0 /backupmanager.sh".

I've cleared that all out and will see if this resolves the issue. Maybe something got caught in a loop and just queued thousands of backups.

Schuyler113 commented 4 months ago

Seemed to resolve the mass updating, however it seems like it is slowly adding lines back to the cronlist file.

jammsen commented 4 months ago

Hey @Schuyler113 - Please update your image, docker-compose and default.env file, a lot has changed and the bugfix for that bug is included in there. The problem was, we did write a file but only "added" lines, which on restart added another line and another. Now we clear after restart the file and add only 1 line. Sorry about that.