search

williamjacksn / docker-caddy-route53

A Docker image for caddyserver/caddy with the dns.providers.route53 module included
11 stars 7 forks source link

dns-01 failing after updating to 2.8.4-1.3.3 #97

Closed chrisshearing closed 2 months ago

chrisshearing commented 2 months ago

New docker instance with fresh caddy image

No record of the api key being used on aws and dns-01 challenge instantly fails after updating to 2.8.4-1.3.3, downgrade to 2.7.6-1.3.3 and the same config worked immediately

logs included, personalised info removed

happy to provide more information if required!

chrisshearing commented 2 months ago 
caddy-1       | {"level":"info","ts":1718379481.3727453,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
caddy-1       | {"level":"warn","ts":1718379481.3734121,"msg":"The 'tls_trusted_ca_certs' field is deprecated. Use the 'tls_trust_pool' field instead."}
caddy-1       | {"level":"warn","ts":1718379481.3739336,"msg":"The 'tls_trusted_ca_certs' field is deprecated. Use the 'tls_trust_pool' field instead."}
caddy-1       | {"level":"warn","ts":1718379481.3740823,"msg":"The 'tls_trusted_ca_certs' field is deprecated. Use the 'tls_trust_pool' field instead."}
caddy-1       | {"level":"warn","ts":1718379481.3742251,"msg":"The 'tls_trusted_ca_certs' field is deprecated. Use the 'tls_trust_pool' field instead."}
caddy-1       | {"level":"warn","ts":1718379481.3743658,"msg":"The 'tls_trusted_ca_certs' field is deprecated. Use the 'tls_trust_pool' field instead."}
caddy-1       | {"level":"info","ts":1718379481.3750556,"msg":"adapted config to JSON","adapter":"caddyfile"}
caddy-1       | {"level":"warn","ts":1718379481.3750799,"msg":"Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies","adapter":"caddyfile","file":"/etc/caddy/Caddyfile","line":2}
caddy-1       | {"level":"info","ts":1718379481.3758266,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
caddy-1       | {"level":"info","ts":1718379481.3760822,"logger":"http.auto_https","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
caddy-1       | {"level":"info","ts":1718379481.3761654,"logger":"http.auto_https","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
caddy-1       | {"level":"info","ts":1718379481.3761685,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0003d3e00"}
caddy-1       | {"level":"warn","ts":1718379481.3763952,"logger":"http.reverse_proxy.transport.http","msg":"root_ca_pool and root_ca_pem_files are deprecated. Use one of the tls.ca_pool.source modules instead"}
caddy-1       | {"level":"warn","ts":1718379481.3766341,"logger":"http.reverse_proxy.transport.http","msg":"root_ca_pool and root_ca_pem_files are deprecated. Use one of the tls.ca_pool.source modules instead"}
caddy-1       | {"level":"warn","ts":1718379481.376763,"logger":"http.reverse_proxy.transport.http","msg":"root_ca_pool and root_ca_pem_files are deprecated. Use one of the tls.ca_pool.source modules instead"}
caddy-1       | {"level":"warn","ts":1718379481.3769174,"logger":"http.reverse_proxy.transport.http","msg":"root_ca_pool and root_ca_pem_files are deprecated. Use one of the tls.ca_pool.source modules instead"}
caddy-1       | {"level":"warn","ts":1718379481.3770738,"logger":"http.reverse_proxy.transport.http","msg":"root_ca_pool and root_ca_pem_files are deprecated. Use one of the tls.ca_pool.source modules instead"}
caddy-1       | {"level":"info","ts":1718379481.3772483,"logger":"http","msg":"enabling HTTP/3 listener","addr":":443"}
caddy-1       | {"level":"info","ts":1718379481.3775344,"msg":"failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 7168 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Buffer-Sizes for details."}
caddy-1       | {"level":"info","ts":1718379481.377687,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
caddy-1       | {"level":"info","ts":1718379481.3777697,"logger":"http.log","msg":"server running","name":"remaining_auto_https_redirects","protocols":["h1","h2","h3"]}
caddy-1       | {"level":"info","ts":1718379481.3777943,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["site1.example.com","site2.example.com","sub.site3.example.com","api.site1.example.com","api.site2.example.com"]}
caddy-1       | {"level":"info","ts":1718379481.378466,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
caddy-1       | {"level":"info","ts":1718379481.37855,"msg":"serving initial configuration"}
caddy-1       | {"level":"info","ts":1718379481.3787599,"logger":"tls.obtain","msg":"acquiring lock","identifier":"site1.example.com"}
caddy-1       | {"level":"info","ts":1718379481.3789825,"logger":"tls","msg":"storage cleaning happened too recently; skipping for now","storage":"FileStorage:/data/caddy","instance":"e171236f-ef06-4ef4-818e-d29e03feb926","try_again":1718465881.3789818,"try_again_in":86399.999999715}
caddy-1       | {"level":"info","ts":1718379481.3790884,"logger":"tls","msg":"finished cleaning storage units"}
caddy-1       | {"level":"info","ts":1718379481.3793228,"logger":"tls.obtain","msg":"acquiring lock","identifier":"site2.example.com"}
caddy-1       | {"level":"info","ts":1718379481.3803155,"logger":"tls.obtain","msg":"lock acquired","identifier":"site1.example.com"}
caddy-1       | {"level":"info","ts":1718379481.3804283,"logger":"tls.obtain","msg":"obtaining certificate","identifier":"site1.example.com"}
caddy-1       | {"level":"info","ts":1718379481.3808823,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["site1.example.com"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"email@example.com"}
caddy-1       | {"level":"info","ts":1718379481.3808935,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["site1.example.com"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"email@example.com"}
caddy-1       | {"level":"info","ts":1718379481.3809023,"logger":"tls.issuance.acme","msg":"using ACME account","account_id":"https://acme-v02.api.letsencrypt.org/acme/acct/1782215137","account_contact":["mailto:email@example.com"]}
caddy-1       | {"level":"info","ts":1718379481.3811173,"logger":"tls.obtain","msg":"lock acquired","identifier":"site2.example.com"}
caddy-1       | {"level":"info","ts":1718379481.3812017,"logger":"tls.obtain","msg":"obtaining certificate","identifier":"site2.example.com"}
caddy-1       | {"level":"info","ts":1718379481.3815465,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["site2.example.com"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"email@example.com"}
caddy-1       | {"level":"info","ts":1718379481.3817232,"logger":"tls.obtain","msg":"acquiring lock","identifier":"sub.site3.example.com"}
caddy-1       | {"level":"info","ts":1718379481.3817484,"logger":"tls.obtain","msg":"acquiring lock","identifier":"api.site1.example.com"}
caddy-1       | {"level":"info","ts":1718379481.3833723,"logger":"tls.obtain","msg":"lock acquired","identifier":"sub.site3.example.com"}
caddy-1       | {"level":"info","ts":1718379481.3833756,"logger":"tls.obtain","msg":"lock acquired","identifier":"api.site1.example.com"}
caddy-1       | {"level":"info","ts":1718379481.3834925,"logger":"tls.obtain","msg":"obtaining certificate","identifier":"sub.site3.example.com"}
caddy-1       | {"level":"info","ts":1718379481.383549,"logger":"tls.obtain","msg":"obtaining certificate","identifier":"api.site1.example.com"}
caddy-1       | {"level":"info","ts":1718379481.3838038,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["api.site1.example.com"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"email@example.com"}
caddy-1       | {"level":"info","ts":1718379481.3839145,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["sub.site3.example.com"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"email@example.com"}
caddy-1       | {"level":"info","ts":1718379481.3839238,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["sub.site3.example.com"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"email@example.com"}
caddy-1       | {"level":"info","ts":1718379481.3839307,"logger":"tls.issuance.acme","msg":"using ACME account","account_id":"https://acme-v02.api.letsencrypt.org/acme/acct/1782215137","account_contact":["mailto:email@example.com"]}
caddy-1       | {"level":"info","ts":1718379481.3840573,"logger":"tls.obtain","msg":"acquiring lock","identifier":"api.site2.example.com"}
caddy-1       | {"level":"info","ts":1718379481.3842926,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["site2.example.com"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"email@example.com"}
caddy-1       | {"level":"info","ts":1718379481.3843098,"logger":"tls.issuance.acme","msg":"using ACME account","account_id":"https://acme-v02.api.letsencrypt.org/acme/acct/1782215137","account_contact":["mailto:email@example.com"]}
caddy-1       | {"level":"info","ts":1718379481.3844,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["api.site1.example.com"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"email@example.com"}
caddy-1       | {"level":"info","ts":1718379481.3844113,"logger":"tls.issuance.acme","msg":"using ACME account","account_id":"https://acme-v02.api.letsencrypt.org/acme/acct/1782215137","account_contact":["mailto:email@example.com"]}
caddy-1       | {"level":"info","ts":1718379481.385587,"logger":"tls.obtain","msg":"lock acquired","identifier":"api.site2.example.com"}
caddy-1       | {"level":"info","ts":1718379481.3856633,"logger":"tls.obtain","msg":"obtaining certificate","identifier":"api.site2.example.com"}
caddy-1       | {"level":"info","ts":1718379481.3859046,"logger":"tls.issuance.acme","msg":"waiting on internal rate limiter","identifiers":["api.site2.example.com"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"email@example.com"}
caddy-1       | {"level":"info","ts":1718379481.3859334,"logger":"tls.issuance.acme","msg":"done waiting on internal rate limiter","identifiers":["api.site2.example.com"],"ca":"https://acme-v02.api.letsencrypt.org/directory","account":"email@example.com"}
caddy-1       | {"level":"info","ts":1718379481.385943,"logger":"tls.issuance.acme","msg":"using ACME account","account_id":"https://acme-v02.api.letsencrypt.org/acme/acct/1782215137","account_contact":["mailto:email@example.com"]}
caddy-1       | {"level":"info","ts":1718379482.2657802,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"site1.example.com","challenge_type":"dns-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}
caddy-1       | {"level":"info","ts":1718379482.2727945,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"sub.site3.example.com","challenge_type":"dns-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}
caddy-1       | {"level":"info","ts":1718379482.2825663,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"site2.example.com","challenge_type":"dns-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}
caddy-1       | {"level":"info","ts":1718379482.3011637,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"api.site2.example.com","challenge_type":"dns-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}
watchtower-1  | time="2024-06-14T15:38:02Z" level=info msg="Watchtower 1.7.1"
watchtower-1  | time="2024-06-14T15:38:02Z" level=info msg="Using no notifications"
watchtower-1  | time="2024-06-14T15:38:02Z" level=info msg="Checking all containers (except explicitly disabled with label)"
watchtower-1  | time="2024-06-14T15:38:02Z" level=info msg="Scheduling first run: 2024-06-15 15:38:02 +0000 UTC"
watchtower-1  | time="2024-06-14T15:38:02Z" level=info msg="Note that the first check will be performed in 23 hours, 59 minutes, 59 seconds"
caddy-1       | {"level":"info","ts":1718379482.331834,"logger":"tls.issuance.acme.acme_client","msg":"trying to solve challenge","identifier":"api.site1.example.com","challenge_type":"dns-01","ca":"https://acme-v02.api.letsencrypt.org/directory"}
caddy-1       | {"level":"error","ts":1718379482.3453333,"logger":"tls.issuance.acme.acme_client","msg":"cleaning up solver","identifier":"site1.example.com","challenge_type":"dns-01","error":"no memory of presenting a DNS record for \"_acme-challenge.site1.example.com\" (usually OK if presenting also failed)"}
caddy-1       | {"level":"error","ts":1718379482.4677951,"logger":"tls.issuance.acme.acme_client","msg":"cleaning up solver","identifier":"sub.site3.example.com","challenge_type":"dns-01","error":"no memory of presenting a DNS record for \"_acme-challenge.sub.site3.example.com\" (usually OK if presenting also failed)"}
caddy-1       | {"level":"error","ts":1718379482.4846554,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"site1.example.com","issuer":"acme-v02.api.letsencrypt.org-directory","error":"[site1.example.com] solving challenges: presenting for challenge: adding temporary record for zone \"example.com.\": not found, ResolveEndpointV2 (order=https://acme-v02.api.letsencrypt.org/acme/order/1782215137/278352309167) (ca=https://acme-v02.api.letsencrypt.org/directory)"}
caddy-1       | {"level":"error","ts":1718379482.4847696,"logger":"tls.obtain","msg":"will retry","error":"[site1.example.com] Obtain: [site1.example.com] solving challenges: presenting for challenge: adding temporary record for zone \"example.com.\": not found, ResolveEndpointV2 (order=https://acme-v02.api.letsencrypt.org/acme/order/1782215137/278352309167) (ca=https://acme-v02.api.letsencrypt.org/directory)","attempt":1,"retrying_in":60,"elapsed":1.104443195,"max_duration":2592000}
caddy-1       | {"level":"error","ts":1718379482.5560853,"logger":"tls.issuance.acme.acme_client","msg":"cleaning up solver","identifier":"site2.example.com","challenge_type":"dns-01","error":"no memory of presenting a DNS record for \"_acme-challenge.site2.example.com\" (usually OK if presenting also failed)"}
caddy-1       | {"level":"error","ts":1718379482.6065476,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"sub.site3.example.com","issuer":"acme-v02.api.letsencrypt.org-directory","error":"[sub.site3.example.com] solving challenges: presenting for challenge: adding temporary record for zone \"example.com.\": not found, ResolveEndpointV2 (order=https://acme-v02.api.letsencrypt.org/acme/order/1782215137/278352309197) (ca=https://acme-v02.api.letsencrypt.org/directory)"}
caddy-1       | {"level":"error","ts":1718379482.6066809,"logger":"tls.obtain","msg":"will retry","error":"[sub.site3.example.com] Obtain: [sub.site3.example.com] solving challenges: presenting for challenge: adding temporary record for zone \"example.com.\": not found, ResolveEndpointV2 (order=https://acme-v02.api.letsencrypt.org/acme/order/1782215137/278352309197) (ca=https://acme-v02.api.letsencrypt.org/directory)","attempt":1,"retrying_in":60,"elapsed":1.223301066,"max_duration":2592000}
caddy-1       | {"level":"error","ts":1718379482.6726089,"logger":"tls.issuance.acme.acme_client","msg":"cleaning up solver","identifier":"api.site2.example.com","challenge_type":"dns-01","error":"no memory of presenting a DNS record for \"_acme-challenge.api.site2.example.com\" (usually OK if presenting also failed)"}
caddy-1       | {"level":"error","ts":1718379482.7073889,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"site2.example.com","issuer":"acme-v02.api.letsencrypt.org-directory","error":"[site2.example.com] solving challenges: presenting for challenge: adding temporary record for zone \"example.com.\": not found, ResolveEndpointV2 (order=https://acme-v02.api.letsencrypt.org/acme/order/1782215137/278352309257) (ca=https://acme-v02.api.letsencrypt.org/directory)"}
caddy-1       | {"level":"error","ts":1718379482.7075138,"logger":"tls.obtain","msg":"will retry","error":"[site2.example.com] Obtain: [site2.example.com] solving challenges: presenting for challenge: adding temporary record for zone \"example.com.\": not found, ResolveEndpointV2 (order=https://acme-v02.api.letsencrypt.org/acme/order/1782215137/278352309257) (ca=https://acme-v02.api.letsencrypt.org/directory)","attempt":1,"retrying_in":60,"elapsed":1.326361434,"max_duration":2592000}
caddy-1       | {"level":"error","ts":1718379482.7572443,"logger":"tls.issuance.acme.acme_client","msg":"cleaning up solver","identifier":"api.site1.example.com","challenge_type":"dns-01","error":"no memory of presenting a DNS record for \"_acme-challenge.api.site1.example.com\" (usually OK if presenting also failed)"}
caddy-1       | {"level":"error","ts":1718379482.828149,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"api.site2.example.com","issuer":"acme-v02.api.letsencrypt.org-directory","error":"[api.site2.example.com] solving challenges: presenting for challenge: adding temporary record for zone \"example.com.\": not found, ResolveEndpointV2 (order=https://acme-v02.api.letsencrypt.org/acme/order/1782215137/278352309287) (ca=https://acme-v02.api.letsencrypt.org/directory)"}
caddy-1       | {"level":"error","ts":1718379482.8281736,"logger":"tls.obtain","msg":"will retry","error":"[api.site2.example.com] Obtain: [api.site2.example.com] solving challenges: presenting for challenge: adding temporary record for zone \"example.com.\": not found, ResolveEndpointV2 (order=https://acme-v02.api.letsencrypt.org/acme/order/1782215137/278352309287) (ca=https://acme-v02.api.letsencrypt.org/directory)","attempt":1,"retrying_in":60,"elapsed":1.442567232,"max_duration":2592000}
caddy-1       | {"level":"error","ts":1718379482.897798,"logger":"tls.obtain","msg":"could not get certificate from issuer","identifier":"api.site1.example.com","issuer":"acme-v02.api.letsencrypt.org-directory","error":"[api.site1.example.com] solving challenges: presenting for challenge: adding temporary record for zone \"example.com.\": not found, ResolveEndpointV2 (order=https://acme-v02.api.letsencrypt.org/acme/order/1782215137/278352309477) (ca=https://acme-v02.api.letsencrypt.org/directory)"}
caddy-1       | {"level":"error","ts":1718379482.8978376,"logger":"tls.obtain","msg":"will retry","error":"[api.site1.example.com] Obtain: [api.site1.example.com] solving challenges: presenting for challenge: adding temporary record for zone \"example.com.\": not found, ResolveEndpointV2 (order=https://acme-v02.api.letsencrypt.org/acme/order/1782215137/278352309477) (ca=https://acme-v02.api.letsencrypt.org/directory)","attempt":1,"retrying_in":60,"elapsed":1.514364193,"max_duration":2592000}
williamjacksn commented 2 months ago

Looks like this is reported upstream at https://github.com/caddy-dns/route53/issues/42

chrisshearing commented 2 months ago

Excellent news, thanks for finding that! It looks like it can be fixed after the pull requests bumping the aws sdk version have been accepted,

Thanks for maintaining this!

williamjacksn commented 2 months ago

I just released version 2.8.4-1.4.0 of this image.

I had to specify an AWS region, which I did not have to do before, but it is working for me now.

williamjacksn commented 2 months ago

I used an environment variable in the container, AWS_REGION: us-east-1