search

k8up-io / k8up

Kubernetes and OpenShift Backup Operator
https://k8up.io/
Apache License 2.0
602 stars 62 forks source link

Archive is Not Created #906

Open chloesoe opened 8 months ago

chloesoe commented 8 months ago

Description

Archive created but empty

Additional Context

In https://github.com/k8up-io/k8up/pull/868 a bug fix for Archive was introduced. Now tested looks like the archive isn't properly created. It exists on the cloud provider but it is empty after backup run.

Logs

kubectl -n company-example-prod logs archive-example-backup-archive-manual-xyz
2023-10-11T15:47:54Z    INFO    k8up    Starting k8up...    {"version": "2.7.2", "date": "2023-10-09T10:13:29Z", "commit": "45d99dd90dbb2a080e6832c34e96b371216a3e0b", "go_os": "linux", "go_arch": "amd64", "go_version": "go1.19.13", "uid": 1000880000, "gid": 0}
2023-10-11T15:47:54Z    INFO    k8up.restic initializing
2023-10-11T15:47:54Z    INFO    k8up.restic setting up a signal handler
2023-10-11T15:47:54Z    INFO    k8up.restic.restic  using the following restic options  {"options": [""]}
2023-10-11T15:47:54Z    INFO    k8up.restic.restic.RepoInit.command restic command  {"path": "/usr/local/bin/restic", "args": ["init", "--option", ""]}
2023-10-11T15:47:54Z    INFO    k8up.restic.restic.RepoInit.command Defining RESTIC_PROGRESS_FPS    {"frequency": 0.016666666666666666}
2023-10-11T15:47:56Z    INFO    k8up.restic.restic.RepoInit.restic.stdout   created restic repository 9bb6c7f16c at s3:https://objects.cloudprovider.tld/erpprod-archive
2023-10-11T15:47:56Z    INFO    k8up.restic.restic.RepoInit.restic.stdout   
2023-10-11T15:47:56Z    INFO    k8up.restic.restic.RepoInit.restic.stdout   Please note that knowledge of your password is required to access
2023-10-11T15:47:56Z    INFO    k8up.restic.restic.RepoInit.restic.stdout   the repository. Losing your password means that your data is
2023-10-11T15:47:56Z    INFO    k8up.restic.restic.RepoInit.restic.stdout   irrecoverably lost.
2023-10-11T15:47:56Z    INFO    k8up.restic.restic.unlock   unlocking repository    {"all": false}
2023-10-11T15:47:56Z    INFO    k8up.restic.restic.unlock.command   restic command  {"path": "/usr/local/bin/restic", "args": ["unlock", "--option", ""]}
2023-10-11T15:47:56Z    INFO    k8up.restic.restic.unlock.command   Defining RESTIC_PROGRESS_FPS    {"frequency": 0.016666666666666666}
2023-10-11T15:47:57Z    INFO    k8up.restic.restic.snapshots    getting list of snapshots
2023-10-11T15:47:57Z    INFO    k8up.restic.restic.snapshots.command    restic command  {"path": "/usr/local/bin/restic", "args": ["snapshots", "--option", "", "--json"]}
2023-10-11T15:47:57Z    INFO    k8up.restic.restic.snapshots.command    Defining RESTIC_PROGRESS_FPS    {"frequency": 0.016666666666666666}
2023-10-11T15:47:58Z    INFO    k8up.restic.restic.snapshots    getting list of snapshots
2023-10-11T15:47:58Z    INFO    k8up.restic.restic.snapshots.command    restic command  {"path": "/usr/local/bin/restic", "args": ["snapshots", "--option", "", "--json"]}
2023-10-11T15:47:58Z    INFO    k8up.restic.restic.snapshots.command    Defining RESTIC_PROGRESS_FPS    {"frequency": 0.016666666666666666}
2023-10-11T15:47:59Z    INFO    k8up.restic.restic.archive  archiving latest snapshots for every host

Expected Behavior

Archive size > 0Bytes

Steps To Reproduce

archive created with schedule:

k -n company-example-prod get schedules.k8up.io example-backup  -oyaml | yq eval '.spec.archive' 
restoreMethod:
  s3:
    accessKeyIDSecretRef:
      key: access_key_id
      name: k8up-credentials
    bucket: example-archive
    endpoint: https://objects.cloudprovider.tld
    secretAccessKeySecretRef:
      key: secret_access_key
      name: k8up-credentials
schedule: 0 0 1 */3 *

Archive could be run manually

Version of K8up

2.7.2

Version of Kubernetes

v1.26.9

Distribution of Kubernetes

OpenShift

poyaz commented 6 months ago

I test on my cluster with my k8up resources (I don't know exactly what your k8up resources are used for to generate an archive).

However, when I didn't have any backup in my cluster, this problem occurred. No archive was created.

P.S: Maybe your problem because of no backup files exist in s3 bucket

Without any backup

2024-01-02T06:02:53Z    INFO    k8up    Starting k8up…    {"version": "2.7.2", "date": "2023-10-09T10:13:29Z", "commit": "45d99dd90dbb2a080e6832c34e96b371216a3e0b", "go_os": "linux", "go_arch": "amd64", "go_versi
2024-01-02T06:02:53Z    INFO    k8up.restic    initializing                                                                                                                                                         
2024-01-02T06:02:53Z    INFO    k8up.restic    setting up a signal handler                                                                                                                                          
2024-01-02T06:02:53Z    INFO    k8up.restic.restic    using the following restic options    {"options": [""]}                                                                                                       
2024-01-02T06:02:53Z    INFO    k8up.restic.restic.RepoInit.command    restic command    {"path": "/usr/local/bin/restic", "args": ["init", "--option", ""]}                                                        
2024-01-02T06:02:53Z    INFO    k8up.restic.restic.RepoInit.command    Defining RESTIC_PROGRESS_FPS    {"frequency": 0.016666666666666666}                                                                          
2024-01-02T06:02:53Z    INFO    k8up.restic.restic.unlock    unlocking repository    {"all": false}                                                                                                                 
2024-01-02T06:02:53Z    INFO    k8up.restic.restic.unlock.command    restic command    {"path": "/usr/local/bin/restic", "args": ["unlock", "--option", ""]}                                                        
2024-01-02T06:02:53Z    INFO    k8up.restic.restic.unlock.command    Defining RESTIC_PROGRESS_FPS    {"frequency": 0.016666666666666666}                                                                            
2024-01-02T06:02:54Z    INFO    k8up.restic.restic.snapshots    getting list of snapshots                                                                                                                           
2024-01-02T06:02:54Z    INFO    k8up.restic.restic.snapshots.command    restic command    {"path": "/usr/local/bin/restic", "args": ["snapshots", "--option", "", "--json"]}                                        
2024-01-02T06:02:54Z    INFO    k8up.restic.restic.snapshots.command    Defining RESTIC_PROGRESS_FPS    {"frequency": 0.016666666666666666}                                                                         
2024-01-02T06:02:54Z    INFO    k8up.restic.restic.snapshots    getting list of snapshots                                                                                                                           
2024-01-02T06:02:54Z    INFO    k8up.restic.restic.snapshots.command    restic command    {"path": "/usr/local/bin/restic", "args": ["snapshots", "--option", "", "--json"]}                                        
2024-01-02T06:02:54Z    INFO    k8up.restic.restic.snapshots.command    Defining RESTIC_PROGRESS_FPS    {"frequency": 0.016666666666666666}                                                                         
2024-01-02T06:02:55Z    INFO    k8up.restic.restic.archive    archiving latest snapshots for every host

With exist at least one backup

2024-01-02T06:02:53Z    INFO    k8up    Starting k8up…    {"version": "2.7.2", "date": "2023-10-09T10:13:29Z", "commit": "45d99dd90dbb2a080e6832c34e96b371216a3e0b", "go_os": "linux", "go_arch": "amd64", "go_versi
2024-01-02T06:02:53Z    INFO    k8up.restic    initializing                                                                                                                                                         
2024-01-02T06:02:53Z    INFO    k8up.restic    setting up a signal handler                                                                                                                                          
2024-01-02T06:02:53Z    INFO    k8up.restic.restic    using the following restic options    {"options": [""]}                                                                                                       
2024-01-02T06:02:53Z    INFO    k8up.restic.restic.RepoInit.command    restic command    {"path": "/usr/local/bin/restic", "args": ["init", "--option", ""]}                                                        
2024-01-02T06:02:53Z    INFO    k8up.restic.restic.RepoInit.command    Defining RESTIC_PROGRESS_FPS    {"frequency": 0.016666666666666666}                                                                          
2024-01-02T06:02:53Z    INFO    k8up.restic.restic.unlock    unlocking repository    {"all": false}                                                                                                                 
2024-01-02T06:02:53Z    INFO    k8up.restic.restic.unlock.command    restic command    {"path": "/usr/local/bin/restic", "args": ["unlock", "--option", ""]}                                                        
2024-01-02T06:02:53Z    INFO    k8up.restic.restic.unlock.command    Defining RESTIC_PROGRESS_FPS    {"frequency": 0.016666666666666666}                                                                            
2024-01-02T06:02:54Z    INFO    k8up.restic.restic.snapshots    getting list of snapshots                                                                                                                           
2024-01-02T06:02:54Z    INFO    k8up.restic.restic.snapshots.command    restic command    {"path": "/usr/local/bin/restic", "args": ["snapshots", "--option", "", "--json"]}                                        
2024-01-02T06:02:54Z    INFO    k8up.restic.restic.snapshots.command    Defining RESTIC_PROGRESS_FPS    {"frequency": 0.016666666666666666}                                                                         
2024-01-02T06:02:54Z    INFO    k8up.restic.restic.snapshots    getting list of snapshots                                                                                                                           
2024-01-02T06:02:54Z    INFO    k8up.restic.restic.snapshots.command    restic command    {"path": "/usr/local/bin/restic", "args": ["snapshots", "--option", "", "--json"]}                                        
2024-01-02T06:02:54Z    INFO    k8up.restic.restic.snapshots.command    Defining RESTIC_PROGRESS_FPS    {"frequency": 0.016666666666666666}                                                                         
2024-01-02T06:02:55Z    INFO    k8up.restic.restic.archive    archiving latest snapshots for every host                                                                                                             
2024-01-02T06:02:55Z    INFO    k8up.restic.restic.archive    starting archival for    {"namespace": "k8up", "pvc": "k8up-nginx"}                                                                                   
2024-01-02T06:02:55Z    INFO    k8up.restic.restic.restore    restore initialised                                                                                                                                   
2024-01-02T06:02:55Z    INFO    k8up.restic.restic.restore    loading all snapshots from repositoy                                                                                                                  
2024-01-02T06:02:55Z    INFO    k8up.restic.restic.snapshots    getting list of snapshots                                                                                                                           
2024-01-02T06:02:55Z    INFO    k8up.restic.restic.snapshots.command    restic command    {"path": "/usr/local/bin/restic", "args": ["snapshots", "--option", "", "--json"]}                                        
2024-01-02T06:02:55Z    INFO    k8up.restic.restic.snapshots.command    Defining RESTIC_PROGRESS_FPS    {"frequency": 0.016666666666666666}                                                                         
2024-01-02T06:02:56Z    INFO    k8up.restic.restic.restore    S3 chosen as restore destination                                                                                                                      
2024-01-02T06:02:56Z    INFO    k8up.restic.restic.restore.command    restic command    {"path": "/usr/local/bin/restic", "args": ["ls", "--option", "", "--json", "1d6cc88e9d1b30071c4af0ebc2e75155f157d89f0f94eac7
2024-01-02T06:02:56Z    INFO    k8up.restic.restic.restore.command    Defining RESTIC_PROGRESS_FPS    {"frequency": 0.016666666666666666}                                                                           
2024-01-02T06:02:56Z    INFO    k8up.restic.restic.restore    starting restore    {"s3 filename": "http://minio.minio:9000/restoremini/backup-k8up-k8up-nginx-2024-01-02T06:01:19Z.tar.gz"}                         
2024-01-02T06:02:56Z    INFO    k8up.restic.restic.restore.command    restic command    {"path": "/usr/local/bin/restic", "args": ["dump", "--option", "", "1d6cc88e9d1b30071c4af0ebc2e75155f157d89f0f94eac7363c2138
2024-01-02T06:02:56Z    INFO    k8up.restic.restic.restore.command    Defining RESTIC_PROGRESS_FPS    {"frequency": 0.016666666666666666}                                                                           
2024-01-02T06:02:57Z    INFO    k8up.restic.restic.restore    restore finishe

Could you please share more about backup and archive resources?