Backup creation fails?

[cboettig]

I run the backup as described but hit this error:

Creating SSH2 RSA key; this may take some time ...
Creating SSH2 DSA key; this may take some time ...
Creating SSH2 ECDSA key; this may take some time ...
Creating SSH2 ED25519 key; this may take some time ...
invoke-rc.d: policy-rc.d denied execution of restart.
Starting openssh server...
Starting mysql server...
Starting redis server...
Running gitlab rake task...
[deprecated] I18n.enforce_available_locales will default to true in the future. If you really want to skip validation of your locale you can set I18n.enforce_available_locales = false to avoid this message.
Dumping database ... 
Dumping MySQL database gitlabhq_production ... [DONE]
done
Dumping repositories ...
rake aborted!

 Mysql2::Error: Table 'gitlabhq_production.projects' doesn't exist: SELECT  `projects`.* FROM `projects`   ORDER BY `projects`.`id` ASC LIMIT 1000

Any ideas what has gone wrong or how to troubleshoot?

I have run docker stop gitlab to stop the container first, but haven't removed it (don't want to lose my set-up, and removing the container is not mentioned in your backup instructions, though for some reason it is mentioned in your upgrading instructions.

Is it possible to run backup by using nsenter instead?

I also tried this docker backup tool without success.

[sameersbn]

@cboettig As long as you are mounting a data volume and have a volume mounted for the mysql data, in case you are using the internal mysql server, it is perfectly safe to remove the container as the relevant data is store at these mount points. When you recreate the container make sure you specify the same mount points for the data and mysql volumes.

Back to the issue, as long as the container is stopped while taking the backup you should be good. The command you use to take the backup should match the docker run command you use to start the container but with app:rake gitlab:backup:create appended to the command. For example, if your docker run command is:

docker run --name=gitlab -d \
-e 'DB_HOST=192.168.1.100' \
-e 'DB_NAME=gitlabhq_production' -e 'DB_USER=gitlab' -e 'DB_PASS=password' \
-v /opt/gitlab/data:/home/git/data \
sameersbn/gitlab:7.3.2-1

Then the command to create the backup should be:

docker run -it --rm \
-e 'DB_HOST=192.168.1.100' \
-e 'DB_NAME=gitlabhq_production' -e 'DB_USER=gitlab' -e 'DB_PASS=password' \
-v /opt/gitlab/data:/home/git/data \
sameersbn/gitlab:7.3.2-1 app:rake gitlab:backup:create

Both commands should have the exact same environment variables and mount points, except you might want to run the backup command in interactive mode -it and maybe with a different or no --name.

Lastly, yes you can nsenter and execute the rake task to create a backup.

cd /home/git/gitlab
sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production

[cboettig]

@sameersbn Thanks for the detailed reply. As I mention above, I am doing exactly the backup provess you describe by running the same command, except with -it mode instead of -d and ommitting the --name to avoid conflict. As you see in the error log I pasted above, this runs but errors along the way.

Thanks for the nsenter information, that approach worked perfectly for me. It creates a .tar backup file, but it isn't clear to me how I get the tarball off of the container now, or how I would use it to restore my settings later (e.g. after upgrading or launching gitlab docker image on a different machine). I'd greatly appreciate your advice on how to do this!

[sameersbn]

@cboettig are you using the internal mysql server? If yes, have you specified a volume mountpoint for /var/lib/mysql? If not, then there is no way that method to create backups as described in the readme is going to work.

The backups are created in the data volumes backup directory. Have you specified a mountpoint for /home/git/data? If yes, then check the contents of the folder on the host. If not (which is not a good idea), then you can use docker cp to copy the .tar file to the host. Refer docker help for usage.

The restore process is documented in the readme. However for it to work you should have /home/git/data volume mounted.

[cboettig]

Yes, I'm using the internal / default mysql server, and no, I did not specify a volume mountpoint for either /home/git/data or /varl/lib/mysql . Thanks for pointing that out, I'll be sure to do so in future. (In the README QuickStart it didn't show linking either of these volumes).

From docker inspect I can see where /home/git/data is mounted. (/var/lib/docker/vfs/dir/). I can also see that my backup is there, in the backup directory. Very cool, thanks.

In the quickstart, I see you're now linking volumes for docker itself, though I don't see that in other examples:

-v  /var/run/docker.sock:/run/docker.sock \
-v $(which docker):/bin/docker \

What is the reason for this?

Thanks again for a great application. Sounds like in future I should now be linking external redis and postrgresql containers, as the internal ones have been removed? Any reason not to use the Docker 'official' postgres container now that one exists?

On Thu, Oct 16, 2014 at 11:04 AM, Sameer Naik notifications@github.com wrote:

@cboettig are you using the internal mysql server? If yes, have you specified a volume mountpoint for /var/lib/mysql? If not, then there is no way that method to create backups as described in the readme is going to work.

The backups are created in the data volumes backup directory. Have you specified a mountpoint for /home/git/data? If yes, then check the contents of the folder on the host. If not (which is not a good idea), then you can use docker cp to copy the .tar file to the host. Refer docker help for usage.

The restore process is documented in the readme. However for it to work you should have /home/git/data volume mounted.

— Reply to this email directly or view it on GitHub.

Carl Boettiger UC Santa Cruz http://carlboettiger.info/

[sameersbn]

@cboettig well it also says "You should now have the GitLab application up and ready for testing. If you want to use this image in production the please read on."

-v /var/run/docker.sock:/run/docker.sock \ -v $(which docker):/bin/docker \ What is the reason for this?

The reason for this is to get the quick start guide to work. Since the internal mysql and redis components have been removed I wanted to make it easy for new users to take the image for a spin and decide whether they want to use gitlab or not (that is exactly why the quickstart command exists). So now what the image does is, if a redis and/or database connection is not specified (which is the case of the quick start guide) and if the docker binary and socket are volume mounted, then the image will pull a redis and a postgresql image from the docker index, set it all up and you are ready to test gitlab. You can also use fig for this.

You can use any postgresql image. The official postgresql image on the docker index I found is very big. It also lacks the feature to create a user and database at launch. Whatever be the case, you can choose whatever postgresql image. The minimum requirements that a postgresql image should fulfil is that it should expose the postgresql port.

[cboettig]

Wow, that's really clever! thanks again for your help.

[cboettig]

@sameersbn last stupid question: do I need to link both a redis and postgres/mysql container, or is linking the postgresql container sufficient?

[sameersbn]

@cboettig gitlab requires both postgresql/mysql and redis.

[cboettig]

ah, of course I should have seen that in your fig.yml: https://github.com/sameersbn/docker-gitlab/blob/master/fig.yml Like you said, use fig. brilliant! many thanks

On Thu, Oct 16, 2014 at 12:33 PM, Sameer Naik notifications@github.com wrote:

@cboettig https://github.com/cboettig gitlab requires both postgresql/mysql and redis.

— Reply to this email directly or view it on GitHub https://github.com/sameersbn/docker-gitlab/issues/177#issuecomment-59417128 .

Carl Boettiger UC Santa Cruz http://carlboettiger.info/

[sameersbn]

@cboettig please note that the fig config does not mount volumes. It a basic configuration.