hashicorp / terraform-provider-google

Terraform Provider for Google Cloud Platform
https://registry.terraform.io/providers/hashicorp/google/latest/docs
Mozilla Public License 2.0
2.36k stars 1.75k forks source link

Support for Cloudfront Domain on AWS to GCP Storage Transfer Jobs #20299

Open qe-wlgoh opened 1 week ago

qe-wlgoh commented 1 week ago

Community Note

Description

Currently, when creating a google_storage_transfer_job resource, it supports specifying a aws_s3_data_source to migrate files from S3 to GCS.

However, even though there are documented means to do it via the REST API, there is currently no documented way for us to pass the CloudFront distribution domain name to the provider. Supporting this feature will definitely ease any data migration to GCP as it will help to minimize data transfer costs incurred on AWS and potentially give us quicker data transfers.

New or Affected Resource(s)

Potential Terraform Configuration

resource "google_storage_transfer_job" "..." {
  project = "..."
  description = "..."

  transfer_spec {
    aws_s3_data_source {
      bucket_name = each.key
      role_arn    = "..."
      cloudfront_domain = "abcdefghijkl.cloudfront.net"
    }
    transfer_options {
        ...
    }
    gcs_data_sink {
        ...
    }
  }
}

References

https://cloud.google.com/storage-transfer/docs/s3-cloudfront#google-cloud-console

BBBmau commented 4 days ago

Confirming that we currently do not support cloudFrontDomain field which can be found in the API Reference here: https://cloud.google.com/storage-transfer/docs/reference/rest/v1/TransferSpec#AwsS3Data