linode / velero-plugin

Verlero plugin that clones Linode CSI volumes for archival
Apache License 2.0
5 stars 3 forks source link

Plugin Panicked error when creating backup #3

Open aleablu opened 10 months ago

aleablu commented 10 months ago

Hi guys, I'm trying to backup my k8s cluster on linode to an aws s3 bucket, I have quite a complex architecture. Here is the output of velero backup describe:

Name:         visidea
Namespace:    velero
Labels:       velero.io/storage-location=default
Annotations:  velero.io/resource-timeout=10m0s
              velero.io/source-cluster-k8s-gitversion=v1.26.9
              velero.io/source-cluster-k8s-major-version=1
              velero.io/source-cluster-k8s-minor-version=26

Phase:  PartiallyFailed (run `velero backup logs visidea` for more information)

Warnings:
  Velero:    error: /the server is currently unable to handle the request
             error: /the server is currently unable to handle the request
  Cluster:    <none>
  Namespaces: <none>

Errors:
  Velero:   
             name: /elasticsearch-es-data-0 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference

             name: /elasticsearch-es-data-1 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference

             name: /elasticsearch-es-data-2 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference

             name: /elasticsearch-es-master-0 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference

             name: /ir-crawler-to-k8s-68558887cb-lzv4p error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference

             name: /mongobackup-859c85666f-snsqw error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference

             name: /mongodb-api-0 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference

             name: /mongodb-api-1 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference
             name: /mongodb-api-2 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference

             name: /mongodb-harness-0 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference

             name: /mongodb-harness-1 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference

             name: /mongodb-harness-2 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference

             name: /recommendify-apish-7574c4c55f-9fdq4 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference

             name: /recommendify-cronjobs-crawl-28334355-47cqd error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference

             name: /data-my-release-etcd-0 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference

             name: /data-my-release-etcd-1 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference

             name: /data-my-release-etcd-2 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference

             name: /datadir-mongodb-ir-0 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference

             name: /datadir-mongodb-ir-1 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference

             name: /datadir-mongodb-ir-2 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference

             name: /mongo-claim0 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference

             name: /mongo-claim1 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference
  Cluster:    <none>
  Namespaces: <none>

Namespaces:
  Included:  default
  Excluded:  <none>

Resources:
  Included:        *
  Excluded:        <none>
  Cluster-scoped:  auto

Label selector:  <none>

Or label selector:  <none>

Storage Location:  default

Velero-Native Snapshot PVs:  auto
Snapshot Move Data:          false
Data Mover:                  velero

TTL:  720h0m0s

CSISnapshotTimeout:    20m0s
ItemOperationTimeout:  4h0m0s

Hooks:  <none>

Backup Format Version:  1.1.0

Started:    2023-11-15 17:47:16 +0100 CET
Completed:  2023-11-15 17:47:30 +0100 CET

Expiration:  2023-12-15 17:47:16 +0100 CET

Total items to be backed up:  427
Items backed up:              427

Velero-Native Snapshots:  0 of 22 snapshots completed successfully (specify --details for more information)

I am currently installing velero via this command:

PLUGIN="velero/velero-plugin-for-aws:v1.0.1,displague/velero-plugin-linode:v0.0.1"

velero install \
    --provider aws \
    --bucket $BUCKET \
    --secret-file $SECRET\
    --backup-location-config region=$REGION,s3ForcePathStyle="true"\
    --use-volume-snapshots=false \
    --plugins $PLUGIN \

I'm using version 1.12 of velero.

Versions of kubernetes are as follows: