search

offen / docker-volume-backup

Backup Docker volumes locally or to any S3, WebDAV, Azure Blob Storage, Dropbox or SSH compatible storage
https://offen.github.io/docker-volume-backup/
Mozilla Public License 2.0
1.73k stars 80 forks source link

"swarm exec" support #425

Open zibellon opened 1 month ago

zibellon commented 1 month ago

Is your feature request related to a problem? Please describe. Yes. The problem was described in issue 423. previous issue

Main points of problem

  1. Swarm cluster. 2 Nodes. MASTER_1, WORKER_1
  2. Run postgres with docker on WORKER_1
  3. Need to backup postgres. Before backup need to run script: pg_dump ...
  4. Up offen-backup on WORKER_1, add labels to deploy section of postgres stack (service)
  5. Catch the error: no master node. Can exec labels only on master

Describe the solution you'd like Solution is

  1. Run offen-backup service ONLY on master node
  2. For exec labels - run one-time container (docker:25.0.5-cli-alpine) on EACH node when we found tasks (containers) with lables
  3. For volumes - run one-time container (offen-backup, for example) on EACH node when we found tasks (containers) with labels: volume-list (You can see it in my .sh script)
  4. If we want to use different timers for different exec-lables or different volume-list-labels - we can also use your idea of exec-label=database, but add support for exec and for volume-list

Additional context I'm a big fan of docker-swarm. And use swarm cluster in production.

I spent some time and write .sh script for backup WHOLE swarm cluster, depends on labels and use your open-source project swarm-backuper.sh

  1. GET all volumes from all nodes. Start docker dind container on each node and get volume list
  2. GET all services with label volume-list. in this label - we enter volumes, which we want to backup for this service. JOIN this volumes with volumes from step one. Result: MAP<node-name, [volume1, volume2, etc]>
  3. RUN all labels: exec-pre. Run separate container on each node, where we need to run exec-pre and run docker exec ...
  4. STOP all services with label: stop. Important: stop === scale to 0, Stop only services in replicated mode
  5. RUN offen-backup on each node with all volumes from step 2
  6. RESTORE all services with label: stop. Important: we need to use the JSON from step 4. Scale to original replicas number, before step 4
  7. RUN all labels: exec-post

How we can run it ? swarm-backuper-stack.yaml

All of this functions can be write on GoLang and add to ypur project (offen-backup)

m90 commented 1 month ago

I'm still not entirely sure why you're unable to execute commands on the same node (this should work no matter if it's a master or a worker node, albeit setup can be a bit tricky, see https://github.com/moby/moby/issues/27552).

That being said, the described approach of using one-off containers that mount volumes themselves has also been discussed here already https://github.com/offen/docker-volume-backup/issues/329 maybe this use case can be considered if this is going to be implemented.