vmware-tanzu / velero

Backup and migrate Kubernetes applications and their persistent volumes
https://velero.io
Apache License 2.0
8.42k stars 1.37k forks source link

Enable google api_endpoint_overrides for storage in velero-plugin-for-gcp #7731

Open salla2 opened 2 months ago

salla2 commented 2 months ago

Describe the problem/challenge you have

We have different storage endpoint for gcs bucket and we cannot set that today, usually using gcloud cli we can set this through gcloud config set api_endpoint_overrides/storage https://storage-test1.p.googleapis.com/storage/v1/ and with sdk through below based on docs [https://cloud.google.com/storage/docs/request-endpoints#storage-set-client-endpoint-go]

Describe the solution you'd like

It would be great if I can pass api_endpoint_overrides/storage through BackupStorageLocation , something like additional key under bucket. objectStorage: bucket: velero-backups prefix: private/stage/cluster api_endpoint_overrides/storage: https://storage-test1.p.googleapis.com/storage/v1/

Anything else you would like to add:

Environment:

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project's top voted issues listed here.
Use the "reaction smiley face" up to the right of this comment to vote.

blackpiglet commented 2 months ago

Does your environment try to connect to the GCP API in the on-premise infrastructure? https://cloud.google.com/vpc/docs/configure-private-service-connect-apis

If the OSS API uses the specified DNS, the snapshot API may also need the same configuration.

chrislinan commented 2 months ago

I have the same request, We would like to use regional endpoints of the GCP service. And I have a potential solution:

  1. add a new config in GCP BSL, such as "endpoint", example:
    spec:
    accessMode: ReadWrite
    credential:
    key: key
    name: velero
    default: true
    endpoint: https://storage-test1.p.googleapis.com/storage/v1
    objectStorage:
    bucket: bucket_name
    prefix: prefix_example
    provider: gcp
  2. enhance the GCP plugin here: Use WithEndPoint for the clientOptions.

I can prepare the PR to implement this feature if that is acceptable.

blackpiglet commented 2 months ago

Thanks for your interest in contribution. If a new configuration is needed, I prefer to add it in the BSL.spec.config.

chrislinan commented 2 months ago

OK, I will prepare a PR for this feature.

chrislinan commented 2 months ago

PR opened for this feature: https://github.com/vmware-tanzu/velero-plugin-for-gcp/pull/184