route53: aws-sdk-go-v2 no longer allows empty region

jspdown commented 7 months ago

Welcome

[X] Yes, I'm using a binary release within 2 latest releases.
[X] Yes, I've searched similar issues on GitHub and didn't find any.
[X] Yes, I've included all information below (version, config, etc).

What did you expect to see?

Since I upgraded from Lego v4.13.3 to v4.14.0 I can no longer use the route-53 provider without explicitly providing an AWS region (e.g. AWS_REGION env variable).

What did you see instead?

I would expect to still be able to not provide an AWS region.

How do you use lego?

Through Traefik

Reproduction steps

Run on v4.14.0:

AWS_ACCESS_KEY_ID=<access-key> \
AWS_SECRET_ACCESS_KEY=<secret-key> \
lego --domains test.example.com --email test@example.com --dns route53 --accept-tos=true run

You get the following error:

error presenting token: route53: failed to determine hosted zone ID: operation error Route 53: ListHostedZonesByName, failed to resolve service endpoint, an AWS region is required, but was not found
Run the same command on v4.13.3
No issue

Version of lego

lego version 7186ebb6f194c55781432162a47ff62a0bb21023 linux/amd64

Logs

```console Your account credentials have been saved in your Let's Encrypt configuration directory at "/home/jspdown/Oss/lego/.lego/accounts". You should make a secure backup of this folder now. This configuration directory will also contain certificates and private keys obtained from Let's Encrypt so making regular backups of this folder is ideal. 2023/12/07 17:33:40 [INFO] [test.example.com] acme: Obtaining bundled SAN certificate 2023/12/07 17:33:41 [INFO] [test.example.com] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/291348424436 2023/12/07 17:33:41 [INFO] [test.example.com] acme: Could not find solver for: tls-alpn-01 2023/12/07 17:33:41 [INFO] [test.example.com] acme: Could not find solver for: http-01 2023/12/07 17:33:41 [INFO] [test.example.com] acme: use dns-01 solver 2023/12/07 17:33:41 [INFO] [test.example.com] acme: Preparing to solve DNS-01 2023/12/07 17:33:41 [INFO] [test.example.com] acme: Cleaning DNS-01 challenge 2023/12/07 17:33:41 [WARN] [test.example.com] acme: cleaning up failed: failed to determine Route 53 hosted zone ID: operation error Route 53: ListHostedZonesByName, failed to resolve service endpoint, an AWS region is required, but was not found 2023/12/07 17:33:41 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/291348424436 2023/12/07 17:33:41 Could not obtain certificates: error: one or more domains had a problem: [test.example.com [test.example.com] acme: error presenting token: route53: failed to determine hosted zone ID: operation error Route 53: ListHostedZonesByName, failed to resolve service endpoint, an AWS region is required, but was not found ```

Go environment (if applicable)

```console $ go version && go env # paste output here ```

jspdown commented 7 months ago

After some digging with the help of Traefik community in https://github.com/traefik/traefik/issues/10195, we've located a regression since the upgrade from aws-sdk-go v1.39.0 to aws-sdk-go-v2 v1.19.0 introduce in that commit https://github.com/go-acme/lego/commit/fc47c35e89562ca8606534f53a650b74ed1ab6eb

aws-sdk-go didn't force the user to specify an AWS region on some service. You can find the list of services here: https://github.com/aws/aws-sdk-go/blob/7087ed2d41a8462e40c00a473fa884d79124b660/aws/endpoints/v3model.go#L183-L200

But the new version aws-sdk-go-v2 is much more strict and doesn't allow it. The fact that we didn't see it coming in the unit tests is because it's a check done in the endpoint Resolver that we mock: https://github.com/aws/aws-sdk-go-v2/blob/57a201c7bb214737f78cf871fbeafaa929bf1d8d/service/route53/internal/endpoints/endpoints.go#L66-L77

ldez commented 7 months ago

Hello,

You said:

I would expect to still be able to not provide an AWS region.

and

But the new version aws-sdk-go-v2 is much more strict and doesn't allow it.

We will not revert the migration to the aws-sdk-go-v2, so what is your suggestion?

jspdown commented 7 months ago

Just reporting the issue, from the outside it's a regression.

Now I understand that there's no much we can do. If we provided a default region directly in Lego like it's done in lightsail, it would create even more breaking change for those using AWS_SDK_LOAD_CONFIG.

So yes, I agree with you. Can't be fixed.

ldez commented 7 months ago

So we agree on the conclusion: it can't be fixed, the region is now a requirement but it depends on the type of credentials used to authenticate.

go-acme / lego