crontab backups not working

[brrrrrrrt]

i installed the crontab:

*/5 * * * * arkmanager backup >> ${SERVER_VOLUME}/log/crontab.log 2>&1

but nothing happened, so i changed it to:

*/5 * * * * arkmanager backup >> /app/log/crontab.log 2>&1

and shortly after i got in /app/log/crontab.log:

Running command 'backup' for instance 'main'
[  WARN  ]      Your ARK server exec could not be found.
mkdir: cannot create directory ‘/server’: Permission denied
[  ERROR  ]     The ARK SavedArks directory is not writable, and saveworld will fail
mkdir: cannot create directory ‘/server’: Permission denied
[  ERROR  ]     The ARK Mods directory is not writable, and installmod will fail
find: ‘/server’: No such file or directory
  [ ERROR  ] Saved arks directory does not exist

when i exec the backup within a docker shell or using docker exec ark_server arkmanager backup it works but the backups are located at /home/steam/ARK-Backups and not within /app/backups

.env:

SESSION_NAME=blabla
SERVER_MAP=CrystalIsles
SERVER_PASSWORD=****************
ADMIN_PASSWORD=****************
MAX_PLAYERS=4
UPDATE_ON_START=true
BACKUP_ON_STOP=false
PRE_UPDATE_BACKUP=true
WARN_ON_STOP=true
GAME_CLIENT_PORT=7777
UDP_SOCKET_PORT=7778
RCON_PORT=27020
SERVER_LIST_PORT=27015
GAME_MOD_IDS=821530042,793605978,895711211,2024410037

docker-compose.yml:

version: '3'

volumes:
  ark-data:
    driver: local
    driver_opts:
      type: "none"
      o: "bind"
      device: "/arkserver"

services:
  server:
    restart: always
    container_name: ark_server
    image: hermsi/ark-server:latest
    volumes:
      - ark-data:/app
    environment:
      - SESSION_NAME=${SESSION_NAME}
      - SERVER_MAP=${SERVER_MAP}
      - SERVER_PASSWORD=${SERVER_PASSWORD}
      - ADMIN_PASSWORD=${ADMIN_PASSWORD}
      - MAX_PLAYERS=${MAX_PLAYERS}
      - UPDATE_ON_START=${UPDATE_ON_START}
      - BACKUP_ON_STOP=${BACKUP_ON_STOP}
      - PRE_UPDATE_BACKUP=${PRE_UPDATE_BACKUP}
      - WARN_ON_STOP=${WARN_ON_STOP}
      - GAME_MOD_IDS=${GAME_MOD_IDS}
      - GAME_CLIENT_PORT=${GAME_CLIENT_PORT}
      - UDP_SOCKET_PORT=${UDP_SOCKET_PORT}
      - RCON_PORT=${RCON_PORT}
      - SERVER_LIST_PORT=${SERVER_LIST_PORT}
    ports:
      # Port for connections from ARK game client
      - "7777:7777/udp"
      # Raw UDP socket port (always Game client port +1)
      - "7778:7778/udp"
      # RCON management port
      - "27020:27020/tcp"
      # Steam's server-list port
      - "27015:27015/udp"
    networks:
      - default

[Hermsi1337]

Ooof - I think the cause of the issue is, that the crontab is not sharing the user's environment.

I will get into this asap!

[jstnmthw]

Hey @brrrrrrrt and @Hermsi1337,

Not sure if you guys ever figured this out but I was having this same problem, and Hermsi was correct. Cron doesn't share the same enviorment variables so arkmanager fails when ran in cron.

I fixed this issue by following this suggested fix.

Edit the /bin/entrypoint.sh and delcare the variables into a file that's placed in ${ARK_SERVER_VOLUME}/environment, you can use this file inside your cronjob to access the same variables. I placed the delcare inside the crontab check.

echo "Loading crontab..."
touch "${ARK_SERVER_VOLUME}/environment"
declare -p | grep -E 'SERVER_MAP|STEAM_HOME|STEAM_USER|ARK_SERVER_VOLUME|GAME_CLIENT_PORT|SERVER_LIST_PORT|RCON_PORT|UPDATE_ON_START|PRE_UPDATE_BACKUP' > "${ARK_SERVER_VOLUME}/environment"
sudo cron -f &
sudo service cron start &
echo "...done"

Rebuild the image and make sure to reference that image in your docker-compose.yml file. Then in the crontab set shell to bash and load the environment variables.

SHELL=/bin/bash
BASH_ENV=/app/environment
0 0 * * * arkmanager update --warn --update-mods >> ${ARK_SERVER_VOLUME}/log/crontab.log 2>&1
*/15 * * * * arkmanager backup >> ${ARK_SERVER_VOLUME}/log/crontab.log 2>&1

Note: The variable for ark server volume in the cronjob looks like it should be ${ARK_SERVER_VOLUME} not ${SERVER_VOLUME}.

This got the cronjob working perfectly.

@Hermsi1337 - I noticed my cron service wasn't starting when the container went up. So I added the following to start the service:

sudo cron -f &
sudo service cron start &
echo "...done"

But I am unsure if cron -f is what's suppose to start it? Running that command in bash, I just get an error similar to it not existing.

cron: can't open or create /var/run/crond.pid: Permission denied

[Hermsi1337]

Hey Folks,

I finally got some time to work on all the issues this image got over the past months.

First of all, many thanks for your research @jstnmthw.
The workaround provided by hackoverflow may work, but IMHO it's quite hacky to inject the environment variables like this.
What do you think about using su in crontab-file? This leaves the env-thing to linux itself.

# Example : backup every day at midnight
0 0 * * * su -c "arkmanager backup" steam 2>&1

I'd appreciate your feedback

[brrrrrrrt]

What do you think about using su in crontab-file? This leaves the env-thing to linux itself.

# Example : backup every day at midnight
0 0 * * * su -c "arkmanager backup" steam 2>&1

not sure if su might be a good solution, su is supposed to run a command with sustitute user and group id.

i am sorry - i moved to another dockerimage for my arkservers, so i cannot test the cronjob using su.

the dockerimage i am using right now also uses cronjobs for backups but instead of su it uses bash -l:

#/etc/cron.d/arkbackup
0 */2 * * * steam bash -l -c 'arkmanager backup >> /ark/log/ark-backup.log 2>&1'

as you see it uses the user steam and bash with the parameter -l:

       -, -l, --login
           Start the shell as a login shell with an environment similar to a real login:
           •   clears all the environment variables except TERM and variables specified by --whitelist-environment
           •   initializes the environment variables HOME, SHELL, USER, LOGNAME, and PATH
           •   changes to the target user’s home directory
           •   sets argv[0] of the shell to '-' in order to make the shell a login shell

maybe using bash in login mode is enough to get the needed enviroment variables, could be worth a try.

--

there are some articles about cron + docker not playing well when it comes to enviroment variables. some recommend to source the enviroment using a file e.g. like this:

0 */2 * * * steam source /etc/container_environment.sh; arkmanager backup >> /ark/log/ark-backup.log 2>&1

some upvoted answer on stackoverflow suggests another approach:

SHELL=/bin/bash
BASH_ENV=/etc/container_environment.sh
0 */2 * * * steam arkmanager backup >> /ark/log/ark-backup.log 2>&1

the file containing the needed enviroment variables can be created like this:

printenv | sed 's/^\(.*\)$/export \1/g' > /etc/container_environment.sh

this should be a reliable way to ensure that the enviroment variables are present.

[jstnmthw]

@Hermsi1337

What do you think about using su in crontab-file? This leaves the env-thing to linux itself.

The crontab is being run with the correct user steam already. It's that crontab does not have access to the same environment variables. Even with your command above, you still need to set enviornment variables for this to work or remove all environment variables from the arkmanager-user.cfg and crontab files for it to work without them.

@brrrrrrrt

For the second part of your reply, that's what I did but automated the environment file in the docker build (see my first reply). I also used declare instead of printenv due to the Stackoverflow fix. Since printenv does not escape the variables but declare does.

[brrrrrrrt]

For the second part of your reply, that's what I did but automated the environment file in the docker build (see my first reply). I also used declare instead of printenv due to the Stackoverflow fix. Since printenv does not escape the variables but declare does.

@jstnmthw ups, sorry, i totally overlooked that... you already posted the proper solution, looks perfect!

regarding your cron error, i think it just means that cron is already running, you could check if there is already a pid-file, before running the command, if this is the case, cron will give an error, because there can only be one running cron process.

cron -f starts cron in foreground mode, and & will send your command to the background, not sure, but does not make sense to me, just running cron without -f and without & should have the same effect. you also do not need to start cron again with service cron start & if it is already running. if the command service is available in the enviroment, i would just to use this and omit the first command. the command service cron start will start cron and then exit, so there is also no need to add a & at the end.

[rustymunkey]

It seems the arkmanager backup command does not require all environment variables to be loaded. I successfully managed to get a cron based backup running by using this crontab entry for the steam user:

# cat /app/crontab
[...]
ARK_SERVER_VOLUME=/app
# Example : backup every 15min
*/15 * * * * arkmanager backup >> /app/log/crontab.log

The backups are then saved to /app/backups.

Hope this helps!

[testkil]

Any more information on how to get this working? rustymunkey's solution didn't really do it for me (except create logs)

[jstnmthw]

@testkil A lot has been updated since this discussion but I posted how I got it working above.

[kumotaicho]

Thank you @jstnmthw, your solution worked right away for me.

I tried many things using su before but none of that was working because the cron job was expected to enter password for user steam.

And thank you very much @Hermsi1337 for this amazing docker image !