search

sameersbn / docker-gitlab

Dockerized GitLab
http://www.damagehead.com/docker-gitlab/
MIT License
7.9k stars 2.14k forks source link

Daily backup with cron #815

Open tetsuo692 opened 8 years ago

tetsuo692 commented 8 years ago

I've recently set up the environnement var GITLAB_BACKUP_DIR.

Now each time I re-start my docker image, there is a new line in the git user crontab. And it's always the same.

root@synology_gitlab:/home/git/gitlab# crontab -l -u git
00 10 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP = RAILS_ENV=production'
00 10 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP = RAILS_ENV=production'

So the backup never start...

solidnerd commented 8 years ago

Hey, could you post your supervisor/cron.log and the value of GITLAB_BACKUP_DIR .

tenken commented 8 years ago

I'm on sameersb/gitlab:8.6.2 and I've noticed for a while I get 2 backups daily. I wonder if it's related to this behavior. I will try to send you the cron.log later today.

GITLAB_BACKUP_EXPIRY       259200
GITLAB_BACKUP_DIR          /home/git/data/backups
GITLAB_BACKUP_TIME         06:00
GITLAB_BACKUP_SCHEDULE     daily
tetsuo692 commented 8 years ago

Hi,

supervisor/cron.log exist but is empty.

GITLAB_BACKUP_DIR /home/git/data/backups/

tenken commented 8 years ago

Is there any further info I can provide?

# uname -a; pwd; ls -lah
Linux synology_gitlab 3.10.77 #7393 SMP Thu Jun 2 19:50:12 CST 2016 x86_64 x86_64 x86_64 GNU/Linux
/var/log/gitlab/supervisor
total 280K
drwxr-xr-x 2 root root 4.0K Jul 21 06:26 .
drwxr-xr-x 6 root root 4.0K May 31 03:58 ..
-rwxr-xr-x 1 root root    0 May 31 03:58 cron.log
-rwxr-xr-x 1 root root    0 May 31 03:58 nginx.log
-rwxr-xr-x 1 root root    0 Jun 12 06:26 sidekiq.log
-rwxr-xr-x 1 root root  760 Jun 12 06:26 sidekiq.log.1
-rwxr-xr-x 1 root root  17K Aug  3 12:07 sshd.log
-rwxr-xr-x 1 root root  30K Jul 21 06:26 sshd.log.1
-rwxr-xr-x 1 root root  863 Jun 12 06:26 sshd.log.2.gz
-rwxr-xr-x 1 root root  49K Aug  3 12:07 supervisord.log
-rwxr-xr-x 1 root root 141K Jul 21 06:26 supervisord.log.1
-rwxr-xr-x 1 root root 6.2K Jun 12 06:26 supervisord.log.2.gz
-rwxr-xr-x 1 root root    0 May 31 03:58 unicorn.log

None of my cron.log files have data in them.

If I look at crontab for user git I see a ton of things; which looks like stuff isn't being cleaned up or set properly:

# crontab -l -u git
15 13 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
30 13 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
30 13 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
30 13 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
30 13 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
30 13 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
00 06 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production'
solidnerd commented 8 years ago

Hey, i tried to investigate this. At the moment I have no clue. I try to reproduce it with your values this takes time until the cron starts. Meanwhile you could check if your cronjob works per minute. For that do the following steps.

  1. Enter in the running gitlab container
docker-compose exec gitlab bash
  1. Add a backup cronjob per minute
echo "*/1 * * * * /bin/bash -l -c 'cd ${GITLAB_INSTALL_DIR} && bundle exec rake gitlab:backup:create SKIP=${GITLAB_BACKUP_SKIP} RAILS_ENV=${RAILS_ENV}'" >> /tmp/cron.${GITLAB_USER} && 
crontab -u ${GITLAB_USER} /tmp/cron.${GITLAB_USER}
  1. Check if cron is installed with

crontab -u git -l

  1. Wait for the result.
tetsuo692 commented 8 years ago

It's very strange

It works, (see screen shot) screen

So i try to change it to run it every 30 minutes... and after that, the entire folder is empty...

solidnerd commented 8 years ago

Did you tried to add your cron (00 10 * * * /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production) command via my instructions . For me it seems that 00 could be a problem because it's not a valid cron value.

tetsuo692 commented 8 years ago

Yes,

I had this line in the git crontab now : /30 * * * \ /bin/bash -l -c 'cd /home/git/gitlab && bundle exec rake gitlab:backup:create SKI P= RAILS_ENV=production' but the directory is empty.

should I set a GITLAB_BACKUP_EXPIRY value ?

tetsuo692 commented 8 years ago

I think that there is a gitlab issue...

when i run bundle exec rake gitlab:backup:create SKIP= RAILS_ENV=production', it finish by deleting all files in backup dir (also tar archive !!!)

tetsuo692 commented 8 years ago

GITLAB_BACKUP_DIR seems to be the temp directory to prepare backup.

I've set up this in gitlab.yml and then it works :

backup:                                                                                     
    upload:                                                                                   
      connection:                                                                             
        provider: local                                                                       
        local_root: '/home/git/data/backup_h'                                                 
      remote_directory: 'gitlab_backups'                                                      
    path: "/home/git/data/backups"
stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had any activity for the last 60 days. It will be closed if no further activity occurs during the next 7 days. Thank you for your contributions.