Outdated code example for R2 with aws-sdk-go-v2

[urgn]

Which Cloudflare product does this pertain to?

R2

Existing documentation URL(s)

The code from this example does not really work. It produces an error:

operation error S3: ListObjectsV2, resolve auth scheme: resolve endpoint: endpoint rule error, Invalid region: region was not a valid DNS name.

What changes are you suggesting?

Update the code listing with working example.

Additional information

No response

[ITheCorgi]

Its just requieres adding WithRegion setting:

        var bucketName = "sdk-example"
    var accountId = "<accountid>"
    var accessKeyId = "<access_key_id>"
    var accessKeySecret = "<access_key_secret>"

    r2Resolver := aws.EndpointResolverWithOptionsFunc(func(service, region string, options ...interface{}) (aws.Endpoint, error) {
        return aws.Endpoint{
            URL: fmt.Sprintf("https://%s.r2.cloudflarestorage.com", accountId),
        }, nil
    })

    cfg, err := config.LoadDefaultConfig(context.TODO(),
        config.WithEndpointResolverWithOptions(r2Resolver),
        config.WithCredentialsProvider(credentials.NewStaticCredentialsProvider(accessKeyId, accessKeySecret, "")),
                config.WithRegion("auto"),
    )
    if err != nil {
        log.Fatal(err)
    }

    client := s3.NewFromConfig(cfg)

[rhaist]

Both aws.EndpointResolverWithOptionsFunc and aws. Endpoint are now deprecated by AWS and the code should be updated.

It seems like AWS makes it harder to use non-AWS services with their SDK and it could be better to use another S3 client that is compatible with R2 for the example section.

Reference: https://aws.github.io/aws-sdk-go-v2/docs/configuring-sdk/endpoints/

[plu]

    cfg, err := config.LoadDefaultConfig(context.TODO(),
        config.WithCredentialsProvider(credentials.NewStaticCredentialsProvider(accessKeyId, accessKeySecret, "")),
        config.WithRegion("auto"),
    )
    if err != nil {
        log.Fatal(err)
    }

    client := s3.NewFromConfig(cfg, func(o *s3.Options) {
        o.BaseEndpoint = aws.String("https://myaccountid.eu.r2.cloudflarestorage.com")
    })

    listObjectsOutput, err := client.ListObjectsV2(context.TODO(), &s3.ListObjectsV2Input{
        Bucket: &bucketName,
    })
    if err != nil {
        log.Fatal(err)
    }

Also stumbled upon the deprecation warnings, then found this solution ☝️.

// Edit

Sorry, to give some more context: AWS has deprecated the global endpoint configuration in favour of a per-service configuration. So only V1 of the endpoint resolver API is deprecated, but there's a successor V2. More details here: https://aws.github.io/aws-sdk-go-v2/docs/configuring-sdk/endpoints/

[zaydek]

This is @plu's code that worked for me in case anyone is struggling:

package main

import (
  "context"
  "fmt"
  "log"

  "github.com/aws/aws-sdk-go-v2/aws"
  "github.com/aws/aws-sdk-go-v2/config"
  "github.com/aws/aws-sdk-go-v2/credentials"
  "github.com/aws/aws-sdk-go-v2/service/s3"
  "github.com/sirupsen/logrus"
)

// https://github.com/cloudflare/cloudflare-docs/issues/12043#issuecomment-2198148921
func ConnectToS3(endpoint, accessKey, secretAccessKey string) (*s3.Client, error) {
  cfg, err := config.LoadDefaultConfig(context.TODO(),
    config.WithCredentialsProvider(credentials.NewStaticCredentialsProvider(
      accessKey,
      secretAccessKey,
      "",
    )),
    config.WithRegion("auto"),
  )
  if err != nil {
    return nil, fmt.Errorf("failed to load config, %w", err)
  }
  client := s3.NewFromConfig(cfg, func(o *s3.Options) {
    o.BaseEndpoint = aws.String(endpoint)
  })
  return client, nil
}

func main() {
  logrus.Info("connecting to S3")
  r2, err := ConnectToS3(
    fmt.Sprintf("https://%s.r2.cloudflarestorage.com", CLOUDFLARE_R2_ACCOUNT_ID),
    CLOUDFLARE_R2_ACCESS_KEY,
    CLOUDFLARE_R2_SECRET_KEY,
  )
  if err != nil {
    logrus.Fatalf("failed to connect to S3: %v", err)
  }
  logrus.Info("connected to S3")