search

sameersbn / docker-gitlab

Dockerized GitLab
http://www.damagehead.com/docker-gitlab/
MIT License
7.85k stars 2.14k forks source link

restoring backup leads to error #1655

Open scheiblr opened 6 years ago

scheiblr commented 6 years ago

I tried to restore a backup and get the following error after the database scripts have been processed:

Restoring repositories ...
 * ESID/OSSE-XML-export ... rake aborted!
GRPC::Unavailable: 14:Connect Failed
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/grpc-1.11.0-x86_64-linux/src/ruby/lib/grpc/generic/active_call.rb:31:in `check_status'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/grpc-1.11.0-x86_64-linux/src/ruby/lib/grpc/generic/active_call.rb:180:in `attach_status_results_and_complete_call'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/grpc-1.11.0-x86_64-linux/src/ruby/lib/grpc/generic/active_call.rb:372:in `request_response'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/grpc-1.11.0-x86_64-linux/src/ruby/lib/grpc/generic/client_stub.rb:178:in `block in request_response'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/grpc-1.11.0-x86_64-linux/src/ruby/lib/grpc/generic/interceptors.rb:170:in `intercept!'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/grpc-1.11.0-x86_64-linux/src/ruby/lib/grpc/generic/client_stub.rb:177:in `request_response'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/grpc-1.11.0-x86_64-linux/src/ruby/lib/grpc/generic/service.rb:170:in `block (3 levels) in rpc_stub_class'
/home/git/gitlab/lib/gitlab/gitaly_client.rb:139:in `call'
/home/git/gitlab/lib/gitlab/gitaly_client/namespace_service.rb:35:in `gitaly_client_call'
/home/git/gitlab/lib/gitlab/gitaly_client/namespace_service.rb:17:in `add'
/home/git/gitlab/lib/gitlab/shell.rb:297:in `add_namespace'
/home/git/gitlab/app/models/storage/legacy_project.rb:27:in `ensure_storage_path_exists'
/home/git/gitlab/app/models/project.rb:54:in `ensure_storage_path_exists'
/home/git/gitlab/lib/backup/repository.rb:138:in `block in restore'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.10/lib/active_record/relation/batches.rb:51:in `block (2 levels) in find_each'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.10/lib/active_record/relation/batches.rb:51:in `each'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.10/lib/active_record/relation/batches.rb:51:in `block in find_each'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.10/lib/active_record/relation/batches.rb:124:in `find_in_batches'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.10/lib/active_record/relation/batches.rb:50:in `find_each'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.10/lib/active_record/querying.rb:9:in `find_each'
/home/git/gitlab/lib/backup/repository.rb:135:in `restore'
/home/git/gitlab/lib/tasks/gitlab/backup.rake:87:in `block (4 levels) in <top (required)>'
/home/git/gitlab/lib/tasks/gitlab/backup.rake:60:in `block (3 levels) in <top (required)>'
/home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/rake-12.3.1/exe/rake:27:in `<top (required)>'
Tasks: TOP => gitlab:backup:repo:restore
(See full trace by running task with --trace)
bradleydwyer commented 6 years ago

I'm seeing this too. Did you discover the cause?

EDIT: Ah, I think it was just that I followed some invalid older instructions which turned off too much of the system (instead of just shutting down unicorn/sidekiq).

scheiblr commented 6 years ago

so, how did you do it? could you please give an example of the commands?

bradleydwyer commented 6 years ago

The instructions from https://docs.gitlab.com/ce/raketasks/backup_restore.html#restore-for-omnibus-installations

I had previously done a gitlab-ctl stop which takes down everything and the gRPC error is obvious in that case.

In the end the restore process is:

sudo gitlab-ctl stop unicorn
sudo gitlab-ctl stop sidekiq
sudo gitlab-rake gitlab:backup:restore BACKUP=<your backup file in /var/opt/gitlab/backups>

And manually copying over gitlab-secrets.json in /etc/gitlab if required (which I did).

To be fair, I'm doing all this in a Docker container on Kubernetes hosted on GKE, so I'm not actually using the artefacts of this repo, but the process should be the same/very similar. I launched the new Gitlab instance in a new container/pod and connected to that pod and executed the above commands after copying my backup to /var/opt/gitlab/backups.

mikew commented 6 years ago

There are specific instructions for this in this project's README: https://github.com/sameersbn/docker-gitlab#restoring-backups

@scheiblr what was the command you ran?

serenader2014 commented 6 years ago

I'm having the same issue, follow the backup/restore guide in this project's README always throw the GRPC::Unavailable: 14:Connect Failed error(Gitlab version 11.0.1 and 11.0.3 both have this problem). After some research, I found a way to fix this.

The error is because of some service is not running when you're doing backup or restore, so to fix the problem, just make sure your gitlab is fully running when you are doing backup or restore.

For me, the procedure is:

  1. Starting the gitlab container
  2. Enter the gitlab's bash shell
  3. Execute /sbin/entrypoint.sh app:rake gitlab:backup:restore to restore a backup
  4. When restore finish, just restart gitlab container, and all is done.

For backup procedure, simply execute /sbin/entrypoint.sh app:rake gitlab:backup:create command when you're in gitlab container's shell.

nanquanyuhao commented 5 years ago

I'm having the same issue, follow the backup/restore guide in this project's README always throw the GRPC::Unavailable: 14:Connect Failed error(Gitlab version 11.0.1 and 11.0.3 both have this problem). After some research, I found a way to fix this.

The error is because of some service is not running when you're doing backup or restore, so to fix the problem, just make sure your gitlab is fully running when you are doing backup or restore.

For me, the procedure is:

1. Starting the gitlab container

2. Enter the gitlab's bash shell

3. Execute `/sbin/entrypoint.sh app:rake gitlab:backup:restore` to restore a backup

4. When restore finish, just restart gitlab container, and all is done.

For backup procedure, simply execute /sbin/entrypoint.sh app:rake gitlab:backup:create command when you're in gitlab container's shell.

I will try...

It's very good.., Very 3Q...

stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had any activity for the last 60 days. It will be closed if no further activity occurs during the next 7 days. Thank you for your contributions.