backube / volsync

Asynchronous data replication for Kubernetes volumes
https://volsync.readthedocs.io
GNU Affero General Public License v3.0
587 stars 68 forks source link

Can' create backup for more than one pvc in same namespace #162

Closed mehdi-wsc closed 2 years ago

mehdi-wsc commented 2 years ago

Describe the bug I'm using volsync with OpenShift cluster, and each time I launch the backup for my pvc , the first job is well passed and the second stuck on pending state. Note: I'm using the same s3 bucket for both pvc ( rcloneDestPath is the same )

Steps to reproduce To reproduce it, you can launch more than one instance of ReplicationSource on namespace that contains more than one pv with same rcolneDestPath. The YAML that I used:

apiVersion: volsync.backube/v1alpha1 kind: ReplicationSource metadata: name: backup-pvc-mysql namespace: wordpress-volsync spec: sourcePVC: data-my-release-mariadb-0 # pvc 1 trigger: schedule: "/10 *" rclone: rcloneConfigSection: "aws-s3-bucket" rcloneDestPath: rcloneConfig: "rclone-secret" copyMethod: Snapshot


apiVersion: volsync.backube/v1alpha1 kind: ReplicationSource metadata: name: backup-pvc-wordpress namespace: wordpress-volsync spec: sourcePVC: my-release-wordpress # pvc 2 trigger: schedule: "/10 *" rclone: rcloneConfigSection: "aws-s3-bucket" rcloneDestPath: rcloneConfig: "rclone-secret" copyMethod: Snapshot

[root@centos ~]# oc get pods
NAME                                               READY   STATUS    RESTARTS   AGE
my-release-mariadb-0                               1/1     Running   0          3h59m
my-release-wordpress-5bb9467848-grpll              1/1     Running   0          3h59m
volsync-rclone-src-backup-pvc-mysql--1-9tfjv       0/1     Pending   0          7m6s

Expected behavior The job runs and the volsync clone my data to S3

Actual results the job keeps in pending state

tesshuflower commented 2 years ago

@mehdi-wsc Are you able to use a different rcloneDestPath for each PVC? This is the path in the s3 bucket where the files would be synced to. If you use the same s3 bucket you will want to use a different rcloneDestPath for each PVC.

As a side note, are you using an empty rcloneDestPath or is it just removed/sanitized in the CR examples above?

mehdi-wsc commented 2 years ago

@tesshuflower No, I'm using a single rcloneDestPath and the bucket was empty. Can you give me an example of using the same bucket for multiple PVC please?

tesshuflower commented 2 years ago

@tesshuflower No, I'm using a single rcloneDestPath and the bucket was empty. Can you give me an example of using the same bucket for multiple PVC please?

Ok, I understand now - you should be able to do something like this:

ReplicationSource1:

...
spec:
  sourcePVC: mypvc1
  trigger:
    schedule: "*/10 * * * *"
  rclone:
    rcloneConfigSection: "aws-s3-bucket"
    rcloneDestPath: "mybucket/mypvc1"
...

ReplicationSource2:

...
spec:
  sourcePVC: mypvc2
  trigger:
    schedule: "*/10 * * * *"
  rclone:
    rcloneConfigSection: "aws-s3-bucket"
    rcloneDestPath: "mybucket/mypvc2"
...

And then there will be a folder (mypvc1 and mypvc2 respectively) created in your mybucket bucket.

mehdi-wsc commented 2 years ago

Hi @tesshuflower I tested it and it works! Thank you