Cannot update AWS SDK to support EKS Pod Identity due to other dependencies failing

[JoeNorth]

What happened: I am attempting to update the AWS SDK for Go to the latest version (v1.48.9) to add support for EKS Pod Identity. However, due to a chain of broken dependencies related to github.com/datawire/ambassador I am unable to do so.

$ go get -u github.com/aws/aws-sdk-go
go: github.com/datawire/ambassador@v1.12.4 requires
        k8s.io/kubectl@v0.18.4 requires
        vbom.ml/util@v0.0.0-20160121211510-db5cfe13f5cc: unrecognized import path "vbom.ml/util": https fetch: Get "https://vbom.ml/util?go-get=1": dial tcp: lookup vbom.ml on 127.0.0.1:53: no such host

Adding a replace directive in go.mod as such does not fix the issue either:

replace (
    vbom.ml/util v0.0.0-20160121211510-db5cfe13f5cc => github.com/fvbommel/util v0.0.0-20160121211510-db5cfe13f5cc
)

$ go get -u github.com/aws/aws-sdk-go
go: github.com/datawire/ambassador@v1.12.4 requires
        github.com/rubenv/sql-migrate@v0.0.0-20200616145509-8d140a17f351 requires
        github.com/godror/godror@v0.13.3 requires
        github.com/go-kit/kit@v0.10.0 requires
        sourcegraph.com/sourcegraph/appdash@v0.0.0-20190731080439-ebfcffb1b5c0: unrecognized import path "sourcegraph.com/sourcegraph/appdash": reading https://sourcegraph.com/sourcegraph/appdash?go-get=1: 404 Not Found

Attempting to update to the latest version (v1.14.4) then runs into other issues:

$ go get -u github.com/aws/aws-sdk-go
go: github.com/datawire/ambassador@v1.14.4 requires
        helm.sh/helm/v3@v3.5.3 requires
        github.com/deislabs/oras@v0.10.0 requires
        github.com/docker/distribution@v0.0.0-00010101000000-000000000000: invalid version: unknown revision 000000000000

This was fixed upstream in ORAS almost 3 years ago in https://github.com/oras-project/oras/pull/218

Given this string of issues we're unable to update the AWS SDK for Go.

What you expected to happen: The dependency on github.com/datawire/ambassador@v1.12.4 will need to be resolved. Likely updating this to the 2.x or 3.x release.

How to reproduce it (as minimally and precisely as possible): Clone the repo and run go get -u github.com/aws/aws-sdk-go

Anything else we need to know?:

Environment:

• External-DNS version (use external-dns --version):
• DNS provider:
• Others:

[symtor]

@JoeNorth Hope you found some workaround in the 2 weeks since your post!

In case you're still stuck: I've encountered the appdash error in a different context (which led me to your post). The appdash package from 2019 appears to no longer be available online. This popped up some time in November 2023. Adding a

replace github.com/go-kit/kit v0.10.0 => github.com/go-kit/kit v0.11.0

into go.mod fixes that step.

In any case, the "good" solution would indeed be to update datawire/ambassador to a more recent major version, so that you no longer depend on 4-year-old packages.

[hungran]

hmm not sure if I am wrong but seem we could update AWS SDK for Go to 1.49.7 with out issue, also tried ko to build image local then test with my end seem ok if following this docs https://docs.aws.amazon.com/eks/latest/userguide/pod-identities.html

• The EKS cluster must setup Amazon EKS Pod Identity Agent Add-ons (done by UI for this check)

• AWS CLI latest

  aws-cli/2.15.4 Python/3.11.6 Darwin/23.1.0 exe/x86_64 prompt/off

• An IAM Role that have trust policy to allow pod. and must associate with SA following RBAC setup

this turn me out that need to be update tutorials for Pod Identity Usage 🤔, if so I'd like to work in this issue

log when using v0.14.0

2023/12/23 17:06:34 Ignoring, HTTP credential provider invalid endpoint host, "169.254.170.23", only loopback hosts are allowed. <nil>
time="2023-12-23T17:06:34Z" level=fatal msg="records retrieval failed: failed to list hosted zones: NoCredentialProviders: no valid providers in chain. Deprecated.\n\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors"

diff --git a/go.mod b/go.mod
index 46659b40..8fe2047f 100644
--- a/go.mod
+++ b/go.mod
@@ -16,7 +16,7 @@ require (
        github.com/alecthomas/kingpin/v2 v2.4.0
        github.com/aliyun/alibaba-cloud-sdk-go v1.62.619
        github.com/ans-group/sdk-go v1.17.0
-       github.com/aws/aws-sdk-go v1.48.9
+       github.com/aws/aws-sdk-go v1.49.7

How with aws-sdk-go v1.49.7

diff --git a/go.sum b/go.sum
index 98a368b8..1b29f007 100644
--- a/go.sum
+++ b/go.sum
@@ -153,6 +153,8 @@ github.com/aws/aws-sdk-go v1.15.11/go.mod h1:mFuSZ37Z9YOHbQEwBWztmVzqXrEkub65tZo
 github.com/aws/aws-sdk-go v1.27.0/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo=
 github.com/aws/aws-sdk-go v1.48.9 h1:vqzjg5FCi/QDWTEenBs65gu57GJdvkqZ0+5steFb44g=
 github.com/aws/aws-sdk-go v1.48.9/go.mod h1:LF8svs817+Nz+DmiMQKTO3ubZ/6IaTpq3TjupRn3Eqk=
+github.com/aws/aws-sdk-go v1.49.7 h1:qQAOWYajSCRQQUFt+OZZ4pgDg2Uf3h4bBQmYzPyyka8=
+github.com/aws/aws-sdk-go v1.49.7/go.mod h1:LF8svs817+Nz+DmiMQKTO3ubZ/6IaTpq3TjupRn3Eqk=
 github.com/aws/aws-sdk-go-v2 v0.18.0/go.mod h1:JWVYvqSMppoMJC0x5wdwiImzgXTI9FuZwxzkQq9wy+g=
 github.com/benbjohnson/clock v1.3.0 h1:ip6w0uFQkncKQ979AypyG0ER7mqUSBdKLOgAle/AT8A=
 github.com/benbjohnson/clock v1.3.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA=

[Raffo]

Look like dependabot was able to update the aws sdk https://github.com/kubernetes-sigs/external-dns/pull/4136. Do you think this update is enough to fix this issue? Can you verify that for us?

[JoeNorth]

Apologies for the delay. Can confirm that it is working with Pod Identity as of the latest commit. Closing issue.

[coro]

Is there a rough ETA on when this might make it to a release? ExternalDNS is the last component in our infra that we've yet to migrate to Pod Identity. Thanks for the PR!

[Indresh2410]

Hi team. Any ETA on the release of this feature? We're eagerly waiting for this

[hegerdes]

A new version with support for pod identity came out 4 days ago. But there is no new helm chart yet. So you have to set the app/image version yourself in your values:

image:
  repository: registry.k8s.io/external-dns/external-dns
  tag: v0.14.1

This works for my