search

truecharts / public

Community Helm Chart Repository
https://truecharts.org
GNU Affero General Public License v3.0
1.13k stars 617 forks source link

Velero broken questions in SCALE GUI preventing backups #17080

Closed StevenMcElligott closed 9 months ago

StevenMcElligott commented 9 months ago

App Name

Velero

Operating System

TrueNAS SCALE 23.10.1

App Version

latest_3.1.9

Application Events

2024-01-11 00:09:06
Created container velero
2024-01-11 00:09:06
Started container velero
2024-01-11 00:08:52
Started container velero-plugin-for-aws
2024-01-11 00:08:52
Container image "velero/velero:v1.12.2" already present on machine
2024-01-11 00:08:51
Created container velero-plugin-for-aws
2024-01-11 00:08:45
Successfully pulled image "velero/velero-plugin-for-aws:v1.8.2@sha256:edfb14403fc4ee45ac99f34930125869a06465c44ffacaf57e351e477e2c6b53" in 2.491905207s (2.491921961s including waiting)
2024-01-11 00:08:42
Pulling image "velero/velero-plugin-for-aws:v1.8.2@sha256:edfb14403fc4ee45ac99f34930125869a06465c44ffacaf57e351e477e2c6b53"
2024-01-11 00:08:38
Created container velero-plugin-for-csi
2024-01-11 00:08:38
Started container velero-plugin-for-csi
2024-01-11 00:08:35
Successfully pulled image "velero/velero-plugin-for-csi:v0.6.2@sha256:f0a6f541a24951a21aecbd42d4170e79e138d2f9efdb9e82b6fb14b3effc2d5a" in 3.309061234s (3.309071683s including waiting)
2024-01-11 00:08:32
Pulling image "velero/velero-plugin-for-csi:v0.6.2@sha256:f0a6f541a24951a21aecbd42d4170e79e138d2f9efdb9e82b6fb14b3effc2d5a"
2024-01-11 00:08:28
Created container opeebs-velero-plugin
2024-01-11 00:08:28
Started container opeebs-velero-plugin
2024-01-11 00:08:27
Successfully pulled image "openebs/velero-plugin:3.6.0@sha256:9ea3331d891e436a7239e37e68ca4c8888500cb122be7cdc9d8400f345555c76" in 2.631965404s (2.631975927s including waiting)
2024-01-11 00:08:25
Created container node-agent
2024-01-11 00:08:25
Started container node-agent
2024-01-11 00:08:24
Add eth0 [172.16.24.85/16] from ix-net
2024-01-11 00:08:24
Add eth0 [172.16.24.86/16] from ix-net
2024-01-11 00:08:24
Container image "velero/velero:v1.12.2" already present on machine
2024-01-11 00:08:24
Pulling image "openebs/velero-plugin:3.6.0@sha256:9ea3331d891e436a7239e37e68ca4c8888500cb122be7cdc9d8400f345555c76"
2024-01-11 00:08:23
Job completed
2024-01-11 00:08:23
Scaled up replica set velero-858b754bb9 to 1
2024-01-11 00:08:23
Created pod: velero-858b754bb9-fkmpt
2024-01-11 00:08:23
Created pod: node-agent-vq5cg
2024-01-11 00:08:23
Successfully assigned ix-velero/velero-858b754bb9-fkmpt to ix-truenas
2024-01-11 00:08:23
Successfully assigned ix-velero/node-agent-vq5cg to ix-truenas
2024-01-11 00:08:17
Successfully pulled image "velero/velero:v1.12.2" in 4.741067206s (4.741081538s including waiting)
2024-01-11 00:08:17
Created container velero
2024-01-11 00:08:17
Started container velero
2024-01-11 00:08:12
Pulling image "velero/velero:v1.12.2"
2024-01-11 00:08:07
Created container kubectl
2024-01-11 00:08:07
Started container kubectl
2024-01-11 00:08:06
Successfully pulled image "docker.io/bitnami/kubectl:1.26" in 9.545767749s (9.545791922s including waiting)
2024-01-11 00:07:57
Add eth0 [172.16.24.84/16] from ix-net
2024-01-11 00:07:57
Pulling image "docker.io/bitnami/kubectl:1.26"
2024-01-11 00:07:56
Created pod: velero-upgrade-crds-69r2g
2024-01-11 00:07:56
Successfully assigned ix-velero/velero-upgrade-crds-69r2g to ix-truenas

Application Logs

2024-01-11 00:09:06.666276-05:00time="2024-01-11T05:09:06Z" level=info msg="setting log-level to INFO" logSource="pkg/cmd/server/server.go:193"
2024-01-11 00:09:06.666371-05:00time="2024-01-11T05:09:06Z" level=info msg="Starting Velero server v1.12.2 (06d12dec4030703cb46b2699ca434cdabd9edbb0-dirty)" logSource="pkg/cmd/server/server.go:195"
2024-01-11 00:09:06.666403-05:00time="2024-01-11T05:09:06Z" level=info msg="No feature flags enabled" logSource="pkg/cmd/server/server.go:199"
2024-01-11 00:09:06.722873-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/velero kind=BackupItemAction logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/crd-remap-version
2024-01-11 00:09:06.722976-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/velero kind=BackupItemAction logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/pod
2024-01-11 00:09:06.723041-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/velero kind=BackupItemAction logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/pv
2024-01-11 00:09:06.723081-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/velero kind=BackupItemAction logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/service-account
2024-01-11 00:09:06.723119-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/add-pv-from-pvc
2024-01-11 00:09:06.723164-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/add-pvc-from-pod
2024-01-11 00:09:06.723177-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/admission-webhook-configuration
2024-01-11 00:09:06.723206-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/apiservice
2024-01-11 00:09:06.723219-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/change-image-name
2024-01-11 00:09:06.723243-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/change-pvc-node-selector
2024-01-11 00:09:06.723255-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/change-storage-class
2024-01-11 00:09:06.723273-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/cluster-role-bindings
2024-01-11 00:09:06.723283-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/crd-preserve-fields
2024-01-11 00:09:06.723309-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/dataupload
2024-01-11 00:09:06.723330-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/init-restore-hook
2024-01-11 00:09:06.723342-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/job
2024-01-11 00:09:06.723396-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/pod
2024-01-11 00:09:06.723424-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/pod-volume-restore
2024-01-11 00:09:06.723450-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/role-bindings
2024-01-11 00:09:06.723467-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/secret
2024-01-11 00:09:06.723490-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/service
2024-01-11 00:09:06.723506-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/velero kind=RestoreItemAction logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/service-account
2024-01-11 00:09:06.723528-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/velero kind=DeleteItemAction logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/dataupload-delete
2024-01-11 00:09:06.771524-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/plugins/velero-blockstore-openebs kind=VolumeSnapshotter logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=openebs.io/cstor-blockstore
2024-01-11 00:09:06.771615-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/plugins/velero-blockstore-openebs kind=VolumeSnapshotter logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=openebs.io/zfspv-blockstore
2024-01-11 00:09:06.811518-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/plugins/velero-plugin-for-aws kind=VolumeSnapshotter logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/aws
2024-01-11 00:09:06.811573-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/plugins/velero-plugin-for-aws kind=ObjectStore logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/aws
2024-01-11 00:09:06.872153-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/plugins/velero-plugin-for-csi kind=BackupItemActionV2 logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/csi-pvc-backupper
2024-01-11 00:09:06.872238-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/plugins/velero-plugin-for-csi kind=BackupItemActionV2 logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/csi-volumesnapshot-backupper
2024-01-11 00:09:06.872255-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/plugins/velero-plugin-for-csi kind=BackupItemActionV2 logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/csi-volumesnapshotclass-backupper
2024-01-11 00:09:06.872280-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/plugins/velero-plugin-for-csi kind=BackupItemActionV2 logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/csi-volumesnapshotcontent-backupper
2024-01-11 00:09:06.872312-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/plugins/velero-plugin-for-csi kind=RestoreItemActionV2 logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/csi-pvc-restorer
2024-01-11 00:09:06.872339-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/plugins/velero-plugin-for-csi kind=RestoreItemActionV2 logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/csi-volumesnapshot-restorer
2024-01-11 00:09:06.872354-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/plugins/velero-plugin-for-csi kind=RestoreItemActionV2 logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/csi-volumesnapshotclass-restorer
2024-01-11 00:09:06.872378-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/plugins/velero-plugin-for-csi kind=RestoreItemActionV2 logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/csi-volumesnapshotcontent-restorer
2024-01-11 00:09:06.872393-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/plugins/velero-plugin-for-csi kind=DeleteItemAction logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/csi-volumesnapshot-delete
2024-01-11 00:09:06.872426-05:00time="2024-01-11T05:09:06Z" level=info msg="registering plugin" command=/plugins/velero-plugin-for-csi kind=DeleteItemAction logSource="pkg/plugin/clientmgmt/process/registry.go:101" name=velero.io/csi-volumesnapshotcontent-delete
2024-01-11 00:09:07.583394-05:00time="2024-01-11T05:09:07Z" level=info msg="Metrics server is starting to listen" addr=":8080" logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" logger=controller-runtime.metrics
2024-01-11 00:09:07.584413-05:00time="2024-01-11T05:09:07Z" level=info msg="Checking existence of namespace." logSource="pkg/cmd/server/server.go:449" namespace=ix-velero
2024-01-11 00:09:07.588064-05:00time="2024-01-11T05:09:07Z" level=info msg="Namespace exists" logSource="pkg/cmd/server/server.go:455" namespace=ix-velero
2024-01-11 00:09:09.438895-05:00I0111 05:09:09.438693       1 request.go:690] Waited for 1.047272164s due to client-side throttling, not priority and fairness, request: GET:https://172.17.0.1:443/apis/velero.io/v2alpha1?timeout=32s
2024-01-11 00:09:10.640119-05:00time="2024-01-11T05:09:10Z" level=info msg="Checking existence of Velero custom resource definitions" logSource="pkg/cmd/server/server.go:484"
2024-01-11 00:09:15.190064-05:00time="2024-01-11T05:09:15Z" level=info msg="Found custom resource" kind=ServerStatusRequest logSource="pkg/cmd/server/server.go:495"
2024-01-11 00:09:15.190200-05:00time="2024-01-11T05:09:15Z" level=info msg="Found custom resource" kind=PodVolumeBackup logSource="pkg/cmd/server/server.go:495"
2024-01-11 00:09:15.190219-05:00time="2024-01-11T05:09:15Z" level=info msg="Found custom resource" kind=Restore logSource="pkg/cmd/server/server.go:495"
2024-01-11 00:09:15.190257-05:00time="2024-01-11T05:09:15Z" level=info msg="Found custom resource" kind=Schedule logSource="pkg/cmd/server/server.go:495"
2024-01-11 00:09:15.190336-05:00time="2024-01-11T05:09:15Z" level=info msg="Found custom resource" kind=VolumeSnapshotLocation logSource="pkg/cmd/server/server.go:495"
2024-01-11 00:09:15.190354-05:00time="2024-01-11T05:09:15Z" level=info msg="Found custom resource" kind=DeleteBackupRequest logSource="pkg/cmd/server/server.go:495"
2024-01-11 00:09:15.190383-05:00time="2024-01-11T05:09:15Z" level=info msg="Found custom resource" kind=BackupStorageLocation logSource="pkg/cmd/server/server.go:495"
2024-01-11 00:09:15.190436-05:00time="2024-01-11T05:09:15Z" level=info msg="Found custom resource" kind=DownloadRequest logSource="pkg/cmd/server/server.go:495"
2024-01-11 00:09:15.190467-05:00time="2024-01-11T05:09:15Z" level=info msg="Found custom resource" kind=Backup logSource="pkg/cmd/server/server.go:495"
2024-01-11 00:09:15.190512-05:00time="2024-01-11T05:09:15Z" level=info msg="Found custom resource" kind=PodVolumeRestore logSource="pkg/cmd/server/server.go:495"
2024-01-11 00:09:15.190544-05:00time="2024-01-11T05:09:15Z" level=info msg="Found custom resource" kind=BackupRepository logSource="pkg/cmd/server/server.go:495"
2024-01-11 00:09:15.190574-05:00time="2024-01-11T05:09:15Z" level=info msg="Found custom resource" kind=DataDownload logSource="pkg/cmd/server/server.go:495"
2024-01-11 00:09:15.190626-05:00time="2024-01-11T05:09:15Z" level=info msg="Found custom resource" kind=DataUpload logSource="pkg/cmd/server/server.go:495"
2024-01-11 00:09:15.190687-05:00time="2024-01-11T05:09:15Z" level=info msg="All Velero custom resource definitions exist" logSource="pkg/cmd/server/server.go:513"
2024-01-11 00:09:15.930650-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting controllers" logSource="pkg/cmd/server/server.go:640"
2024-01-11 00:09:15.931209-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting metric server at address [:8085]" logSource="pkg/cmd/server/server.go:645"
2024-01-11 00:09:15.932210-05:00time="2024-01-11T05:09:15Z" level=info msg="Server starting..." logSource="pkg/cmd/server/server.go:970"
2024-01-11 00:09:15.932582-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting server" addr="{\"IP\":\"::\",\"Port\":8080,\"Zone\":\"\"}" kind=metrics logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" path=/metrics
2024-01-11 00:09:15.932701-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting EventSource" controller=backupstoragelocation controllerGroup=velero.io controllerKind=BackupStorageLocation logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" source="kind source: *v1.BackupStorageLocation"
2024-01-11 00:09:15.932780-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting EventSource" controller=backup controllerGroup=velero.io controllerKind=Backup logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" source="kind source: *v1.Backup"
2024-01-11 00:09:15.932852-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting Controller" controller=backup controllerGroup=velero.io controllerKind=Backup logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108"
2024-01-11 00:09:15.932914-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting EventSource" controller=backup controllerGroup=velero.io controllerKind=Backup logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" source="kind source: *v1.Backup"
2024-01-11 00:09:15.932985-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting EventSource" controller=schedule controllerGroup=velero.io controllerKind=Schedule logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" source="kind source: *v1.Schedule"
2024-01-11 00:09:15.933083-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting EventSource" controller=deletebackuprequest controllerGroup=velero.io controllerKind=DeleteBackupRequest logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" source="kind source: *v1.DeleteBackupRequest"
2024-01-11 00:09:15.933180-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting EventSource" controller=downloadrequest controllerGroup=velero.io controllerKind=DownloadRequest logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" source="kind source: *v1.DownloadRequest"
2024-01-11 00:09:15.933234-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting EventSource" controller=backupstoragelocation controllerGroup=velero.io controllerKind=BackupStorageLocation logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" source="kind source: *v1.BackupStorageLocation"
2024-01-11 00:09:15.933292-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting EventSource" controller=restore controllerGroup=velero.io controllerKind=Restore logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" source="kind source: *v1.Restore"
2024-01-11 00:09:15.933408-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting EventSource" controller=restore controllerGroup=velero.io controllerKind=Restore logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" source="kind source: *v1.RestoreList"
2024-01-11 00:09:15.933470-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting Controller" controller=restore controllerGroup=velero.io controllerKind=Restore logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108"
2024-01-11 00:09:15.933524-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting EventSource" controller=backup controllerGroup=velero.io controllerKind=Backup logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" source="kind source: *v1.Backup"
2024-01-11 00:09:15.933616-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting EventSource" controller=restore controllerGroup=velero.io controllerKind=Restore logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" source="kind source: *v1.Restore"
2024-01-11 00:09:15.933683-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting Controller" controller=restore controllerGroup=velero.io controllerKind=Restore logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108"
2024-01-11 00:09:15.933731-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting EventSource" controller=backup controllerGroup=velero.io controllerKind=Backup logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" source="kind source: *v1.BackupList"
2024-01-11 00:09:15.933769-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting EventSource" controller=backuprepository controllerGroup=velero.io controllerKind=BackupRepository logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" source="kind source: *v1.BackupRepository"
2024-01-11 00:09:15.933843-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting EventSource" controller=schedule controllerGroup=velero.io controllerKind=Schedule logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" source="kind source: *v1.ScheduleList"
2024-01-11 00:09:15.933876-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting Controller" controller=backup controllerGroup=velero.io controllerKind=Backup logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108"
2024-01-11 00:09:15.933916-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting EventSource" controller=backuprepository controllerGroup=velero.io controllerKind=BackupRepository logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" source="kind source: *v1.BackupRepositoryList"
2024-01-11 00:09:15.933959-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting EventSource" controller=backuprepository controllerGroup=velero.io controllerKind=BackupRepository logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" source="kind source: *v1.BackupStorageLocation"
2024-01-11 00:09:15.933997-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting EventSource" controller=backupstoragelocation controllerGroup=velero.io controllerKind=BackupStorageLocation logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" source="kind source: *v1.BackupStorageLocationList"
2024-01-11 00:09:15.934041-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting EventSource" controller=serverstatusrequest controllerGroup=velero.io controllerKind=ServerStatusRequest logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" source="kind source: *v1.ServerStatusRequest"
2024-01-11 00:09:15.934088-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting EventSource" controller=downloadrequest controllerGroup=velero.io controllerKind=DownloadRequest logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" source="kind source: *v1.DownloadRequestList"
2024-01-11 00:09:15.934172-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting EventSource" controller=backupstoragelocation controllerGroup=velero.io controllerKind=BackupStorageLocation logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" source="kind source: *v1.BackupStorageLocationList"
2024-01-11 00:09:15.934299-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting Controller" controller=backupstoragelocation controllerGroup=velero.io controllerKind=BackupStorageLocation logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108"
2024-01-11 00:09:15.934377-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting EventSource" controller=backup controllerGroup=velero.io controllerKind=Backup logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" source="kind source: *v1.Backup"
2024-01-11 00:09:15.934469-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting EventSource" controller=deletebackuprequest controllerGroup=velero.io controllerKind=DeleteBackupRequest logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" source="kind source: *v1.DeleteBackupRequestList"
2024-01-11 00:09:15.934536-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting Controller" controller=deletebackuprequest controllerGroup=velero.io controllerKind=DeleteBackupRequest logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108"
2024-01-11 00:09:15.934568-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting Controller" controller=schedule controllerGroup=velero.io controllerKind=Schedule logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108"
2024-01-11 00:09:15.934610-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting Controller" controller=backuprepository controllerGroup=velero.io controllerKind=BackupRepository logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108"
2024-01-11 00:09:15.934637-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting Controller" controller=backupstoragelocation controllerGroup=velero.io controllerKind=BackupStorageLocation logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108"
2024-01-11 00:09:15.934682-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting Controller" controller=serverstatusrequest controllerGroup=velero.io controllerKind=ServerStatusRequest logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108"
2024-01-11 00:09:15.934775-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting Controller" controller=downloadrequest controllerGroup=velero.io controllerKind=DownloadRequest logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108"
2024-01-11 00:09:15.934879-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting EventSource" controller=backup controllerGroup=velero.io controllerKind=Backup logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" source="kind source: *v1.BackupList"
2024-01-11 00:09:15.934912-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting Controller" controller=backup controllerGroup=velero.io controllerKind=Backup logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108"
2024-01-11 00:09:15.934955-05:00time="2024-01-11T05:09:15Z" level=info msg="Starting Controller" controller=backup controllerGroup=velero.io controllerKind=Backup logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108"
2024-01-11 00:09:16.034637-05:00time="2024-01-11T05:09:16Z" level=info msg="Starting workers" controller=backupstoragelocation controllerGroup=velero.io controllerKind=BackupStorageLocation logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" worker count=1
2024-01-11 00:09:16.034761-05:00time="2024-01-11T05:09:16Z" level=info msg="Starting workers" controller=backup controllerGroup=velero.io controllerKind=Backup logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" worker count=1
2024-01-11 00:09:16.034806-05:00time="2024-01-11T05:09:16Z" level=info msg="Starting workers" controller=backup controllerGroup=velero.io controllerKind=Backup logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" worker count=1
2024-01-11 00:09:16.034873-05:00time="2024-01-11T05:09:16Z" level=info msg="Starting workers" controller=backupstoragelocation controllerGroup=velero.io controllerKind=BackupStorageLocation logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" worker count=1
2024-01-11 00:09:16.034918-05:00time="2024-01-11T05:09:16Z" level=info msg="Starting workers" controller=backup controllerGroup=velero.io controllerKind=Backup logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" worker count=1
2024-01-11 00:09:16.034945-05:00time="2024-01-11T05:09:16Z" level=info msg="Starting workers" controller=backuprepository controllerGroup=velero.io controllerKind=BackupRepository logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" worker count=1
2024-01-11 00:09:16.034999-05:00time="2024-01-11T05:09:16Z" level=info msg="Starting workers" controller=schedule controllerGroup=velero.io controllerKind=Schedule logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" worker count=1
2024-01-11 00:09:16.035021-05:00time="2024-01-11T05:09:16Z" level=info msg="Starting workers" controller=downloadrequest controllerGroup=velero.io controllerKind=DownloadRequest logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" worker count=1
2024-01-11 00:09:16.035075-05:00time="2024-01-11T05:09:16Z" level=info msg="Starting workers" controller=deletebackuprequest controllerGroup=velero.io controllerKind=DeleteBackupRequest logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" worker count=1
2024-01-11 00:09:16.035094-05:00time="2024-01-11T05:09:16Z" level=info msg="Starting workers" controller=backup controllerGroup=velero.io controllerKind=Backup logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" worker count=1
2024-01-11 00:09:16.035133-05:00time="2024-01-11T05:09:16Z" level=info msg="Starting workers" controller=restore controllerGroup=velero.io controllerKind=Restore logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" worker count=1
2024-01-11 00:09:16.135784-05:00time="2024-01-11T05:09:16Z" level=info msg="Starting workers" controller=restore controllerGroup=velero.io controllerKind=Restore logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" worker count=1
2024-01-11 00:09:16.135879-05:00time="2024-01-11T05:09:16Z" level=info msg="Starting workers" controller=serverstatusrequest controllerGroup=velero.io controllerKind=ServerStatusRequest logSource="/go/pkg/mod/github.com/bombsimon/logrusr/v3@v3.0.0/logrusr.go:108" worker count=10
2024-01-11 00:09:17.198675-05:00time="2024-01-11T05:09:17Z" level=info msg="Validating BackupStorageLocation" backup-storage-location=ix-velero/velero-default controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:152"
2024-01-11 00:09:17.360742-05:00time="2024-01-11T05:09:17Z" level=error msg="fail to validate backup store" backup-storage-location=ix-velero/velero-default controller=backup-storage-location error="rpc error: code = Unknown desc = InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.\n\tstatus code: 403, request id: ZVV0Y57640H8VKXX, host id: M5exonhq7MjOEJ11RuhpeEsHUDVxJKjZ97TRqrq5mmw8QRCMFADCTDDIzUYHA0DLj6Xlc6tZZF8=" error.file="/go/src/github.com/vmware-tanzu/velero/pkg/persistence/object_store.go:198" error.function="github.com/vmware-tanzu/velero/pkg/persistence.(*objectBackupStore).IsValid" logSource="pkg/controller/backup_storage_location_controller.go:155"
2024-01-11 00:09:17.360848-05:00time="2024-01-11T05:09:17Z" level=info msg="BackupStorageLocation is invalid, marking as unavailable" backup-storage-location=ix-velero/velero-default controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:131"
2024-01-11 00:09:17.369459-05:00time="2024-01-11T05:09:17Z" level=error msg="Current BackupStorageLocations available/unavailable/unknown: 0/0/1, BackupStorageLocation \"velero-default\" is unavailable: rpc error: code = Unknown desc = InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.\n\tstatus code: 403, request id: ZVV0Y57640H8VKXX, host id: M5exonhq7MjOEJ11RuhpeEsHUDVxJKjZ97TRqrq5mmw8QRCMFADCTDDIzUYHA0DLj6Xlc6tZZF8=)" controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:192"
2024-01-11 00:09:17.369562-05:00time="2024-01-11T05:09:17Z" level=warning msg="There is no existing BackupStorageLocation set as default. Please see `velero backup-location -h` for options." controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:201"
2024-01-11 00:09:17.396534-05:00time="2024-01-11T05:09:17Z" level=error msg="Error listing backups in backup store" backupLocation=ix-velero/velero-default controller=backup-sync error="rpc error: code = Unknown desc = InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.\n\tstatus code: 403, request id: ZVV6TKKYWFM5XBTG, host id: OBzMR3rh6SFwSUAhPlFvcHJofHXbQJd607nOaRIQl9iEUx0DpNcNDXyRStLAahOz893EY2Xit4U=" error.file="/go/src/velero-plugin-for-aws/velero-plugin-for-aws/object_store.go:440" error.function="main.(*ObjectStore).ListCommonPrefixes" logSource="pkg/controller/backup_sync_controller.go:107"
2024-01-11 00:10:17.354878-05:00time="2024-01-11T05:10:17Z" level=error msg="Error listing backups in backup store" backupLocation=ix-velero/velero-default controller=backup-sync error="rpc error: code = Unknown desc = InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.\n\tstatus code: 403, request id: CZ815J5KXHNJ8059, host id: Kz8Qbv4dcPGl239V34xhBwM+SQ1ds+nCoP5RgX1dUzoaO2BjkcOeAXgv+JutAVgV6OUK0yihPIw=" error.file="/go/src/velero-plugin-for-aws/velero-plugin-for-aws/object_store.go:440" error.function="main.(*ObjectStore).ListCommonPrefixes" logSource="pkg/controller/backup_sync_controller.go:107"
2024-01-11 00:10:26.989183-05:00time="2024-01-11T05:10:26Z" level=info msg="Validating BackupStorageLocation" backup-storage-location=ix-velero/velero-default controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:152"
2024-01-11 00:10:27.154073-05:00time="2024-01-11T05:10:27Z" level=error msg="fail to validate backup store" backup-storage-location=ix-velero/velero-default controller=backup-storage-location error="rpc error: code = Unknown desc = InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.\n\tstatus code: 403, request id: 62JV73T41RDCS6CV, host id: b1aBHX9Yq85b0N7aa8eKw8IIcklpFtDRrFUewjMoUalrot3Ja4n7Qdcp87L/zfqkCqbaSzsC9xk=" error.file="/go/src/github.com/vmware-tanzu/velero/pkg/persistence/object_store.go:198" error.function="github.com/vmware-tanzu/velero/pkg/persistence.(*objectBackupStore).IsValid" logSource="pkg/controller/backup_storage_location_controller.go:155"
2024-01-11 00:10:27.154195-05:00time="2024-01-11T05:10:27Z" level=info msg="BackupStorageLocation is invalid, marking as unavailable" backup-storage-location=ix-velero/velero-default controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:131"
2024-01-11 00:10:27.169592-05:00time="2024-01-11T05:10:27Z" level=error msg="Current BackupStorageLocations available/unavailable/unknown: 0/1/0, BackupStorageLocation \"velero-default\" is unavailable: rpc error: code = Unknown desc = InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.\n\tstatus code: 403, request id: 62JV73T41RDCS6CV, host id: b1aBHX9Yq85b0N7aa8eKw8IIcklpFtDRrFUewjMoUalrot3Ja4n7Qdcp87L/zfqkCqbaSzsC9xk=)" controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:192"
2024-01-11 00:10:27.169666-05:00time="2024-01-11T05:10:27Z" level=warning msg="There is no existing BackupStorageLocation set as default. Please see `velero backup-location -h` for options." controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:201"
2024-01-11 00:11:16.810931-05:00time="2024-01-11T05:11:16Z" level=error msg="Error listing backups in backup store" backupLocation=ix-velero/velero-default controller=backup-sync error="rpc error: code = Unknown desc = InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.\n\tstatus code: 403, request id: NRWCHH98BXKY5T2X, host id: zFBGPLSH91PNcIsGyBvjebguE7wn7Luf8oBlEednaCasarhVLFz1y1MGqXU5g+5iDoIcdu42JVw=" error.file="/go/src/velero-plugin-for-aws/velero-plugin-for-aws/object_store.go:440" error.function="main.(*ObjectStore).ListCommonPrefixes" logSource="pkg/controller/backup_sync_controller.go:107"
2024-01-11 00:11:37.212759-05:00time="2024-01-11T05:11:37Z" level=info msg="Validating BackupStorageLocation" backup-storage-location=ix-velero/velero-default controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:152"
2024-01-11 00:11:37.371931-05:00time="2024-01-11T05:11:37Z" level=error msg="fail to validate backup store" backup-storage-location=ix-velero/velero-default controller=backup-storage-location error="rpc error: code = Unknown desc = InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.\n\tstatus code: 403, request id: T4CZFN8NCHPK6QFD, host id: ugnYsUQDzN+h3XeM+hVwvXMAQoOSDOF5czNyA1vW3D1oflt3Y8+PDO6NDhyW8znRef3mNh3KuzE=" error.file="/go/src/github.com/vmware-tanzu/velero/pkg/persistence/object_store.go:198" error.function="github.com/vmware-tanzu/velero/pkg/persistence.(*objectBackupStore).IsValid" logSource="pkg/controller/backup_storage_location_controller.go:155"
2024-01-11 00:11:37.372024-05:00time="2024-01-11T05:11:37Z" level=info msg="BackupStorageLocation is invalid, marking as unavailable" backup-storage-location=ix-velero/velero-default controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:131"
2024-01-11 00:11:37.381515-05:00time="2024-01-11T05:11:37Z" level=error msg="Current BackupStorageLocations available/unavailable/unknown: 0/1/0, BackupStorageLocation \"velero-default\" is unavailable: rpc error: code = Unknown desc = InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.\n\tstatus code: 403, request id: T4CZFN8NCHPK6QFD, host id: ugnYsUQDzN+h3XeM+hVwvXMAQoOSDOF5czNyA1vW3D1oflt3Y8+PDO6NDhyW8znRef3mNh3KuzE=)" controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:192"
2024-01-11 00:11:37.381595-05:00time="2024-01-11T05:11:37Z" level=warning msg="There is no existing BackupStorageLocation set as default. Please see `velero backup-location -h` for options." controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:201"
2024-01-11 00:12:17.196037-05:00time="2024-01-11T05:12:17Z" level=error msg="Error listing backups in backup store" backupLocation=ix-velero/velero-default controller=backup-sync error="rpc error: code = Unknown desc = InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.\n\tstatus code: 403, request id: YJ8P3A32F23XKY08, host id: JkQT3j5Of1Fam0LEqeBnRLwoh5hiyCQU/FxZwmewaFBaM3gEe5xwoWnu7ktmkx1jxRNiBjmAJEQ=" error.file="/go/src/velero-plugin-for-aws/velero-plugin-for-aws/object_store.go:440" error.function="main.(*ObjectStore).ListCommonPrefixes" logSource="pkg/controller/backup_sync_controller.go:107"
2024-01-11 00:12:46.586582-05:00time="2024-01-11T05:12:46Z" level=info msg="Validating BackupStorageLocation" backup-storage-location=ix-velero/velero-default controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:152"
2024-01-11 00:12:46.763452-05:00time="2024-01-11T05:12:46Z" level=error msg="fail to validate backup store" backup-storage-location=ix-velero/velero-default controller=backup-storage-location error="rpc error: code = Unknown desc = InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.\n\tstatus code: 403, request id: YZSNHXNJ56K4GDG9, host id: G196GukqfT4qRLYqtjdkW+DAZHwJ0kLDwxZJmMKcV/rZ03KRL7TqMkrHV6Tb+3tzWVTTbKtfKZQ=" error.file="/go/src/github.com/vmware-tanzu/velero/pkg/persistence/object_store.go:198" error.function="github.com/vmware-tanzu/velero/pkg/persistence.(*objectBackupStore).IsValid" logSource="pkg/controller/backup_storage_location_controller.go:155"
2024-01-11 00:12:46.763567-05:00time="2024-01-11T05:12:46Z" level=info msg="BackupStorageLocation is invalid, marking as unavailable" backup-storage-location=ix-velero/velero-default controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:131"
2024-01-11 00:12:46.776071-05:00time="2024-01-11T05:12:46Z" level=error msg="Current BackupStorageLocations available/unavailable/unknown: 0/1/0, BackupStorageLocation \"velero-default\" is unavailable: rpc error: code = Unknown desc = InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.\n\tstatus code: 403, request id: YZSNHXNJ56K4GDG9, host id: G196GukqfT4qRLYqtjdkW+DAZHwJ0kLDwxZJmMKcV/rZ03KRL7TqMkrHV6Tb+3tzWVTTbKtfKZQ=)" controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:192"
2024-01-11 00:12:46.776191-05:00time="2024-01-11T05:12:46Z" level=warning msg="There is no existing BackupStorageLocation set as default. Please see `velero backup-location -h` for options." controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:201"
2024-01-11 00:13:16.620293-05:00time="2024-01-11T05:13:16Z" level=error msg="Error listing backups in backup store" backupLocation=ix-velero/velero-default controller=backup-sync error="rpc error: code = Unknown desc = InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.\n\tstatus code: 403, request id: G520JHVNWFMTEZD2, host id: sSGhRR17gCVcrlHAKU2L3K8rp4cZ1OpICQkB2CMr+Pip4IcTvr5C136qiot2d1KMV2c8xHZJHvo=" error.file="/go/src/velero-plugin-for-aws/velero-plugin-for-aws/object_store.go:440" error.function="main.(*ObjectStore).ListCommonPrefixes" logSource="pkg/controller/backup_sync_controller.go:107"
2024-01-11 00:13:47.053652-05:00time="2024-01-11T05:13:47Z" level=info msg="Validating BackupStorageLocation" backup-storage-location=ix-velero/velero-default controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:152"
2024-01-11 00:13:47.230058-05:00time="2024-01-11T05:13:47Z" level=error msg="fail to validate backup store" backup-storage-location=ix-velero/velero-default controller=backup-storage-location error="rpc error: code = Unknown desc = InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.\n\tstatus code: 403, request id: MM798B6RF754XMJ4, host id: I+V0u6Y4K8SuCw/6q15kJlzTh1FQYYNuvaF1pEHk429y1SkfmKpM90oiHw+n6d03XUu3ONRzob8=" error.file="/go/src/github.com/vmware-tanzu/velero/pkg/persistence/object_store.go:198" error.function="github.com/vmware-tanzu/velero/pkg/persistence.(*objectBackupStore).IsValid" logSource="pkg/controller/backup_storage_location_controller.go:155"
2024-01-11 00:13:47.230165-05:00time="2024-01-11T05:13:47Z" level=info msg="BackupStorageLocation is invalid, marking as unavailable" backup-storage-location=ix-velero/velero-default controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:131"
2024-01-11 00:13:47.240065-05:00time="2024-01-11T05:13:47Z" level=error msg="Current BackupStorageLocations available/unavailable/unknown: 0/1/0, BackupStorageLocation \"velero-default\" is unavailable: rpc error: code = Unknown desc = InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.\n\tstatus code: 403, request id: MM798B6RF754XMJ4, host id: I+V0u6Y4K8SuCw/6q15kJlzTh1FQYYNuvaF1pEHk429y1SkfmKpM90oiHw+n6d03XUu3ONRzob8=)" controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:192"
2024-01-11 00:13:47.240151-05:00time="2024-01-11T05:13:47Z" level=warning msg="There is no existing BackupStorageLocation set as default. Please see `velero backup-location -h` for options." controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:201"
2024-01-11 00:14:16.660005-05:00time="2024-01-11T05:14:16Z" level=error msg="Error listing backups in backup store" backupLocation=ix-velero/velero-default controller=backup-sync error="rpc error: code = Unknown desc = InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.\n\tstatus code: 403, request id: F6S3XBS6H5BCQCMN, host id: oNgSHUSNSeA63jDFABPDQNv0u2yX8DmdlMlvIsv8UDak2P1Bwk2LWgMa/ghuorBKt7z1MiwvpII=" error.file="/go/src/velero-plugin-for-aws/velero-plugin-for-aws/object_store.go:440" error.function="main.(*ObjectStore).ListCommonPrefixes" logSource="pkg/controller/backup_sync_controller.go:107"
2024-01-11 00:14:17.041638-05:00I0111 05:14:17.041478       1 request.go:690] Waited for 1.046771408s due to client-side throttling, not priority and fairness, request: GET:https://172.17.0.1:443/apis/velero.io/v1?timeout=32s
2024-01-11 00:14:56.601700-05:00time="2024-01-11T05:14:56Z" level=info msg="Validating BackupStorageLocation" backup-storage-location=ix-velero/velero-default controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:152"
2024-01-11 00:14:56.764887-05:00time="2024-01-11T05:14:56Z" level=error msg="fail to validate backup store" backup-storage-location=ix-velero/velero-default controller=backup-storage-location error="rpc error: code = Unknown desc = InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.\n\tstatus code: 403, request id: M0RR3B5SNPTEYB96, host id: YMBoyMdd+XRAl/zeI6toTe419vcvA/Cm1XQn4OgiAoqe18yzSWc/hbKRyMYfMNEtJew+DsFujRw=" error.file="/go/src/github.com/vmware-tanzu/velero/pkg/persistence/object_store.go:198" error.function="github.com/vmware-tanzu/velero/pkg/persistence.(*objectBackupStore).IsValid" logSource="pkg/controller/backup_storage_location_controller.go:155"
2024-01-11 00:14:56.765070-05:00time="2024-01-11T05:14:56Z" level=info msg="BackupStorageLocation is invalid, marking as unavailable" backup-storage-location=ix-velero/velero-default controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:131"
2024-01-11 00:14:56.778414-05:00time="2024-01-11T05:14:56Z" level=error msg="Current BackupStorageLocations available/unavailable/unknown: 0/1/0, BackupStorageLocation \"velero-default\" is unavailable: rpc error: code = Unknown desc = InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.\n\tstatus code: 403, request id: M0RR3B5SNPTEYB96, host id: YMBoyMdd+XRAl/zeI6toTe419vcvA/Cm1XQn4OgiAoqe18yzSWc/hbKRyMYfMNEtJew+DsFujRw=)" controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:192"
2024-01-11 00:14:56.778508-05:00time="2024-01-11T05:14:56Z" level=warning msg="There is no existing BackupStorageLocation set as default. Please see `velero backup-location -h` for options." controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:201"
2024-01-11 00:15:16.665346-05:00time="2024-01-11T05:15:16Z" level=error msg="Error listing backups in backup store" backupLocation=ix-velero/velero-default controller=backup-sync error="rpc error: code = Unknown desc = InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.\n\tstatus code: 403, request id: 4CX7YMZ4THTBYVB1, host id: 9syuMABbrCS1g6ZUVkFNTTN+YQGNCahl1TkHVowZAh6fwD6J+VFZsiJU2uFONskDHubS6IAKtCw=" error.file="/go/src/velero-plugin-for-aws/velero-plugin-for-aws/object_store.go:440" error.function="main.(*ObjectStore).ListCommonPrefixes" logSource="pkg/controller/backup_sync_controller.go:107"
2024-01-11 00:15:57.321011-05:00time="2024-01-11T05:15:57Z" level=info msg="Validating BackupStorageLocation" backup-storage-location=ix-velero/velero-default controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:152"
2024-01-11 00:15:57.498523-05:00time="2024-01-11T05:15:57Z" level=error msg="fail to validate backup store" backup-storage-location=ix-velero/velero-default controller=backup-storage-location error="rpc error: code = Unknown desc = InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.\n\tstatus code: 403, request id: D6NP5G5H1PZ2S4XV, host id: OOf5/2moGT4gFVtLHOs5rTRSXW9wkO1lJHe5eZm34P43bnvw4FP3o4GKNs76VrKGv/y7g5F17qU=" error.file="/go/src/github.com/vmware-tanzu/velero/pkg/persistence/object_store.go:198" error.function="github.com/vmware-tanzu/velero/pkg/persistence.(*objectBackupStore).IsValid" logSource="pkg/controller/backup_storage_location_controller.go:155"
2024-01-11 00:15:57.498627-05:00time="2024-01-11T05:15:57Z" level=info msg="BackupStorageLocation is invalid, marking as unavailable" backup-storage-location=ix-velero/velero-default controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:131"
2024-01-11 00:15:57.507192-05:00time="2024-01-11T05:15:57Z" level=error msg="Current BackupStorageLocations available/unavailable/unknown: 0/1/0, BackupStorageLocation \"velero-default\" is unavailable: rpc error: code = Unknown desc = InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.\n\tstatus code: 403, request id: D6NP5G5H1PZ2S4XV, host id: OOf5/2moGT4gFVtLHOs5rTRSXW9wkO1lJHe5eZm34P43bnvw4FP3o4GKNs76VrKGv/y7g5F17qU=)" controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:192"
2024-01-11 00:15:57.507287-05:00time="2024-01-11T05:15:57Z" level=warning msg="There is no existing BackupStorageLocation set as default. Please see `velero backup-location -h` for options." controller=backup-storage-location logSource="pkg/controller/backup_storage_location_controller.go:201"

Application Configuration

image

Describe the bug

GUI defaults to show all providers and they need to be fixed on SCALE

To Reproduce

Install, enter S3 and still see AWS

Expected Behavior

Runs if I can get more testing

Screenshots

Same thing inside volume snapshot locations

image

Additional Context

Putting this here so we can fix later otherwise I'll forget

I've read and agree with the following

PrivatePuffin commented 9 months ago

We cannot change this. AWS/AZure etc sections will show when s3 is selected and should just be ignored.

Not a bug.

PrivatePuffin commented 9 months ago

Also, its not preventing backups.

PrivatePuffin commented 9 months ago

can we please keep alpha testing on the discord?