search

desec-io / desec-stack

Backbone of the deSEC Free Secure DNS Hosting Service
https://desec.io/
MIT License
392 stars 50 forks source link

DynDNS Update API fails for sub-domains <sub>.<main>.dedyn.io #411

Closed hcc23 closed 4 years ago

hcc23 commented 4 years ago

What I did:

  1. registered with desec.io and got myself a domain: hcc.dedyn.io
  2. used the API to generate a subdomain: sub.hcc.dedyn.io
  3. checked success via curl: 
    curl -X GET https://desec.io/api/v1/domains/hcc.dedyn.io/rrsets/\?type\=A \      
--header "Authorization: Token <my access token>" | python -m json.tool
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                             Dload  Upload   Total   Spent    Left  Speed
100   196  100   196    0     0   2000      0 --:--:-- --:--:-- --:--:--  2000
[
{
    "created": "2020-06-07T13:41:48.026493Z",
    "domain": "hcc.dedyn.io",
    "subname": "sub",
    "name": "sub.hcc.dedyn.io.",
    "records": [
        "127.0.0.1"
    ],
    "ttl": 3600,
    "type": "A",
    "touched": "2020-06-07T13:48:23.303495Z"
}
]
  4. tried to update the A record via the DynDNS API via curl: 
    curl -v -u hcc.dedyn.io:<my access token> \                           
https://update.dedyn.io/?hostname=sub.hcc.dedyn.io&myipv4=127.0.0.2&myipv6=::2
*   Trying 2a01:4f8:10a:1044:deec:642:ac10:80:443...
* Connected to update.dedyn.io (2a01:4f8:10a:1044:deec:642:ac10:80) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=dedyn.io
*  start date: Apr  9 13:50:41 2020 GMT
*  expire date: Jul  8 13:50:41 2020 GMT
*  subjectAltName: host "update.dedyn.io" matched cert's "update.dedyn.io"
*  issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Server auth using Basic with user 'hcc.dedyn.io'
* Using Stream ID: 1 (easy handle 0x55c641ae3d90)
> GET /?hostname=sub.hcc.dedyn.io&myipv4=127.0.0.2&myipv6=::2 HTTP/2
> Host: update.dedyn.io
> authorization: Basic <base64 of my login credentials>
> user-agent: curl/7.70.0
> accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 404 
< server: nginx
< date: Sun, 07 Jun 2020 13:51:44 GMT
< content-type: text/plain
< content-length: 6
< allow: GET, HEAD, OPTIONS
< vary: Origin
< strict-transport-security: max-age=31536000; includeSubdomains; preload
< 
* Connection #0 to host update.dedyn.io left intact
nohost
  5. got a 404 nohost

Expected Behavior:

Observed Behavior:

Context: I am trying to get a wildcard DNS record ( *.hcc.dedyn.io) auto-updated by a Fritz!Box in order to start playing with various server containers behind a Traefik reverse proxy (which can use letsencrypt DNS challenge to issue wildcard certificates for all those servers)

peterthomassen commented 4 years ago

Thank you for your report!

This behavior is expected: The update endpoint currently only works with the zone apex (not with subdomains). In your opinion, is this unclear in the documentation? If so, feel free to open a pull request to improve it!

Regarding your subdomain: You can create a CNAME on the * subdomain with our API, pointing to the domain name itself. This way, IP updates also effectively work for subdomains. Does that help?

hcc23 commented 4 years ago

Hi @peterthomassen

Thanks for the feedback :)

With respect to the docu:

The fact that I have completely missed the part that I can only update the apex domain via the dyndns mechanics (at least in my eyes) would indicate that there is potential for improvement ;)

-> Would you prefer a separate PR or shall I simply extend #403 ?

With respect to the actual problem:

Did I get this right? "All" I need to do is:

  1. add a CNAME which points *.<main>.dedyn.io to <main>.dedyn.io
  2. regularly update <main>.dedyn.io via the rugular dyndns service

But then the question then remains: how would I then use the DNS-01 Challenge from letsencrypt? My plan would be to use the goacme/lego client (as part of a Traefik setup) as outlined in https://go-acme.github.io/lego/dns/desec/ .... Using the CNAME setup, would I then simply do a 

DESEC_TOKEN=x-xxxxxxxxxxxxxxxxxxxxxxxxxx \
lego --dns desec --domains hcc.dedyn.io --email postmaster@myemail.de run

?

PS: I'll try it anyways and would incorporate my lessons learned in the docu PR. (new or #403 ?)

hcc23 commented 4 years ago

Summary

Admittedly, I am neither a DNS nor a let's-encrypt guru, but to me it seems that the proposed CNAME-based workaround with the CNAME entry pointing *.<main>.dedyn.io to <main>.dedyn.io does not work towards having the goal to have a letsencrypt wildcard certificate for a domain whose IP is auto-updated by my FritzBox :(

The long read

Here is what I did:

Step 1: Prepare the domain by adding an A, AAAA, and CNAME record:

curl -X PATCH https://desec.io/api/v1/domains/c0.dedyn.io/rrsets/ \
    --header "Authorization: <my access token>" \
    --header "Content-Type: application/json" --data @- <<EOF
    [
      {"subname": "", "type": "A", "ttl": 3600, "records": ["127.0.0.1"]},
      {"subname": "", "type": "AAAA", "ttl": 3600, "records": ["::1"]},
      {"subname": "*", "type": "CNAME", "ttl": 3600, "records": ["c0.dedyn.io."]},
    ]
EOF

Step 2: Testing that the DynDNS API is working

curl -u c0.dedyn.io:<my access token> https://update.dedyn.io/?myipv4=127.0.0.2&myipv6=::2

Step 3: Setup the FritzBox to update the desec DynDNS using the URL https://update.dedyn.io/?myipv4=<ipaddr>&myipv6=<ip6addr>

Step 4: Checking the results

curl -X GET https://desec.io/api/v1/domains/c0.dedyn.io/rrsets/ \
    --header "Authorization: Token <my access token>" | python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1065  100  1065    0     0   8949      0 --:--:-- --:--:-- --:--:--  8949
[
    {
        "created": "2020-06-08T11:50:05.488666Z",
        "domain": "c0.dedyn.io",
        "subname": "",
        "name": "c0.dedyn.io.",
        "records": [
            "2a01:c22:d200:28a:e228:6dff:fe49:3e54"
        ],
        "ttl": 60,
        "type": "AAAA",
        "touched": "2020-06-08T12:23:59.901734Z"
    },
    {
        "created": "2020-06-08T11:50:05.481158Z",
        "domain": "c0.dedyn.io",
        "subname": "",
        "name": "c0.dedyn.io.",
        "records": [
            "77.189.127.190"
        ],
        "ttl": 60,
        "type": "A",
        "touched": "2020-06-08T12:23:59.896156Z"
    },
    {
        "created": "2020-06-08T11:50:05.470050Z",
        "domain": "c0.dedyn.io",
        "subname": "*",
        "name": "*.c0.dedyn.io.",
        "records": [
            "c0.dedyn.io."
        ],
        "ttl": 3600,
        "type": "CNAME",
        "touched": "2020-06-08T11:50:05.478375Z"
    },
    {
        "created": "2020-05-09T19:52:23.548996Z",
        "domain": "c0.dedyn.io",
        "subname": "",
        "name": "c0.dedyn.io.",
        "records": [
            "ns1.desec.io.",
            "ns2.desec.org."
        ],
        "ttl": 3600,
        "type": "NS",
        "touched": "2020-05-09T19:52:23.550359Z"
    }
]

So far, so good :)

Step 5: Testing DNS-01 challenge for the domain apex (using goacme/lego )

docker run -it \
-e DESEC_TOKEN=<my access token> \
goacme/lego --accept-tos --dns desec --domains c0.dedyn.io --email postmaster@dadac0.de run

2020/06/08 12:42:24 No key found for account postmaster@dadac0.de. Generating a P384 key.
2020/06/08 12:42:24 Saved key to /.lego/accounts/acme-v02.api.letsencrypt.org/postmaster@dadac0.de/keys/postmaster@dadac0.de.key
2020/06/08 12:42:25 [INFO] acme: Registering account for postmaster@dadac0.de
!!!! HEADS UP !!!!

Your account credentials have been saved in your Let's Encrypt
configuration directory at "/.lego/accounts".

You should make a secure backup of this folder now. This
configuration directory will also contain certificates and
private keys obtained from Let's Encrypt so making regular
backups of this folder is ideal.
2020/06/08 12:42:25 [INFO] [c0.dedyn.io] acme: Obtaining bundled SAN certificate
2020/06/08 12:42:26 [INFO] [c0.dedyn.io] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/5099816795
2020/06/08 12:42:26 [INFO] [c0.dedyn.io] acme: Could not find solver for: tls-alpn-01
2020/06/08 12:42:26 [INFO] [c0.dedyn.io] acme: Could not find solver for: http-01
2020/06/08 12:42:26 [INFO] [c0.dedyn.io] acme: use dns-01 solver
2020/06/08 12:42:26 [INFO] [c0.dedyn.io] acme: Preparing to solve DNS-01
2020/06/08 12:42:26 [INFO] [c0.dedyn.io] acme: Trying to solve DNS-01
2020/06/08 12:42:26 [INFO] [c0.dedyn.io] acme: Checking DNS record propagation using [192.168.71.2:53]
2020/06/08 12:42:26 [INFO] Wait for propagation [timeout: 1m0s, interval: 2s]
2020/06/08 12:42:26 [INFO] [c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 12:42:28 [INFO] [c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 12:42:35 [INFO] [c0.dedyn.io] The server validated our request
2020/06/08 12:42:35 [INFO] [c0.dedyn.io] acme: Cleaning DNS-01 challenge
2020/06/08 12:42:35 [INFO] [c0.dedyn.io] acme: Validations succeeded; requesting certificates
2020/06/08 12:42:36 [INFO] [c0.dedyn.io] Server responded with a certificate.

Yeah, success :)

Step 6: Attempting the DNS-01 against the CNAME-ed wildcard domain *.<main>.dedyn.io:

docker run -it \
-e DESEC_TOKEN=<my access token> \
goacme/lego --accept-tos --dns desec --domains *.c0.dedyn.io --email postmaster@dadac0.de run

2020/06/08 12:43:25 No key found for account postmaster@dadac0.de. Generating a P384 key.
2020/06/08 12:43:25 Saved key to /.lego/accounts/acme-v02.api.letsencrypt.org/postmaster@dadac0.de/keys/postmaster@dadac0.de.key
2020/06/08 12:43:26 [INFO] acme: Registering account for postmaster@dadac0.de
!!!! HEADS UP !!!!

Your account credentials have been saved in your Let's Encrypt
configuration directory at "/.lego/accounts".

You should make a secure backup of this folder now. This
configuration directory will also contain certificates and
private keys obtained from Let's Encrypt so making regular
backups of this folder is ideal.
2020/06/08 12:43:26 [INFO] [*.c0.dedyn.io] acme: Obtaining bundled SAN certificate
2020/06/08 12:43:26 [INFO] [*.c0.dedyn.io] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/5099830813
2020/06/08 12:43:26 [INFO] [*.c0.dedyn.io] acme: use dns-01 solver
2020/06/08 12:43:26 [INFO] [*.c0.dedyn.io] acme: Preparing to solve DNS-01
2020/06/08 12:43:27 [INFO] [*.c0.dedyn.io] acme: Trying to solve DNS-01
2020/06/08 12:43:27 [INFO] [*.c0.dedyn.io] acme: Checking DNS record propagation using [192.168.71.2:53]
2020/06/08 12:43:27 [INFO] Wait for propagation [timeout: 1m0s, interval: 2s]
2020/06/08 12:43:27 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 12:43:34 [INFO] [*.c0.dedyn.io] acme: Cleaning DNS-01 challenge
2020/06/08 12:43:35 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/5099830813
2020/06/08 12:43:35 [INFO] Unable to deactivate the authorization: https://acme-v02.api.letsencrypt.org/acme/authz-v3/5099830813
2020/06/08 12:43:35 Could not obtain certificates:
        error: one or more domains had a problem:
[*.c0.dedyn.io] acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: Incorrect TXT record "TjPRxgeBsf-WcnIZu59SwRq290llCK-kPRnWvGNmytg" found at _acme-challenge.c0.dedyn.io, url:

:'( that didn't work: acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: Incorrect TXT record "TjPRxgeBsf-WcnIZu59SwRq290llCK-kPRnWvGNmytg" found at _acme-challenge.c0.dedyn.io

Step 7: Clear the mismatching TXT record:

curl -X PATCH https://desec.io/api/v1/domains/c0.dedyn.io/rrsets/ \             
    --header "Authorization: Token <my access token>" \
    --header "Content-Type: application/json" --data @- <<EOF
    [
      {"subname": "_acme-challenge", "type": "TXT", "ttl": 3600, "records": []}        
    ]
EOF

Step 8: Attempting the DNS-01 against the CNAME-ed wildcard domain *.<main>.dedyn.io again:

docker run -it \                                                 
-e DESEC_TOKEN=<my access token> \                                         
goacme/lego --accept-tos --dns desec --domains \*.c0.dedyn.io --email postmaster@dadac0.de run

2020/06/08 13:03:54 No key found for account postmaster@dadac0.de. Generating a P384 key.
2020/06/08 13:03:54 Saved key to /.lego/accounts/acme-v02.api.letsencrypt.org/postmaster@dadac0.de/keys/postmaster@dadac0.de.key
2020/06/08 13:03:54 [INFO] acme: Registering account for postmaster@dadac0.de
!!!! HEADS UP !!!!

Your account credentials have been saved in your Let's Encrypt
configuration directory at "/.lego/accounts".

You should make a secure backup of this folder now. This
configuration directory will also contain certificates and
private keys obtained from Let's Encrypt so making regular
backups of this folder is ideal.
2020/06/08 13:03:55 [INFO] [*.c0.dedyn.io] acme: Obtaining bundled SAN certificate
2020/06/08 13:03:55 [INFO] [*.c0.dedyn.io] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/5100134982
2020/06/08 13:03:55 [INFO] [*.c0.dedyn.io] acme: use dns-01 solver
2020/06/08 13:03:55 [INFO] [*.c0.dedyn.io] acme: Preparing to solve DNS-01
2020/06/08 13:03:55 [INFO] [*.c0.dedyn.io] acme: Trying to solve DNS-01
2020/06/08 13:03:55 [INFO] [*.c0.dedyn.io] acme: Checking DNS record propagation using [192.168.71.2:53]
2020/06/08 13:03:55 [INFO] Wait for propagation [timeout: 1m0s, interval: 2s]
2020/06/08 13:03:55 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:03:57 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:03:59 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:02 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:04 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:06 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:08 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:10 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:12 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:14 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:16 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:18 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:20 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:22 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:24 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:26 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:28 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:31 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:33 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:35 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:37 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:39 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:41 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:43 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:45 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:47 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:49 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:51 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:53 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:55 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/08 13:04:57 [INFO] [*.c0.dedyn.io] acme: Cleaning DNS-01 challenge
2020/06/08 13:04:58 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/5100134982
2020/06/08 13:04:58 Could not obtain certificates:
        error: one or more domains had a problem:
[*.c0.dedyn.io] time limit exceeded: last error: NS ns1.desec.io. did not return the expected TXT record [fqdn: c0.dedyn.io., value: 0DLfq8Hj1YV7WZwhE-dE0oIFQjx4iZY99OS_9jzfLLY]:

That didn't work :'(

hcc23 commented 4 years ago

So, is it possible to change the expected behavior that only the apex domain can be updated via the DynDNS API, i.e. to make this issue #411 a feature request?

nils-wisiol commented 4 years ago

This behavior is expected: The update endpoint currently only works with the zone apex (not with subdomains).

Can we change this behavior? Given a token and a hostname, if the token is valid for the nearest known ancestor of the given hostname, update this zone so that the corresponding A/AAAA record is set to the requested IP. If I'm not overlooking something, this could be a relatively easy change with a big impact on usability and flexibility.

peterthomassen commented 4 years ago

That seems feasible, yes.

However, it's not clear to me yet why @hcc23 is observing problems with the ACME challenge when using a wildcard CNAME. The ACME challenge DNS name for *.example.com is _acme-challenge.example.com. In other words, if for the c0.dedyn.io domain itself and for the wildcard, a TXT record needs to be placed under _acme-challenge.c0.dedyn.io -- and as it seems, that record worked correctly to issue a certificate for the main domain. It should also work in order to request a wildcard certificate, and the presence of the wildcard CNAME should not play any role (Let's Encrypt only looks at the TXT record at _acme-challenge.<apex>).

So, something else must be going on. Maybe a caching issue? What's the TTL you were using for you TXT records? Maybe reduce the TTL of the CNAME to prevent it from getting cached?

hcc23 commented 4 years ago

Good morning.

Q: What was the TTL (of the TXT record)? A: Sorry, I don't recall. To check that the non-wildcard TXT wasn't a problem I tried again just now (without having attempted to get a DNS-01 challenge to the non-wildcard doamain first) - time out again.... :cry: 

➜  ~ curl -X GET https://desec.io/api/v1/domains/c0.dedyn.io/rrsets/ \
    --header "Authorization: Token <my access token>" | python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   809  100   809    0     0   5152      0 --:--:-- --:--:-- --:--:--  5152
[
    {
        "created": "2020-06-08T11:50:05.488666Z",
        "domain": "c0.dedyn.io",
        "name": "c0.dedyn.io.",
        "records": [
            "2a01:c23:7e00:55a2:e228:6dff:fe49:3e54"
        ],
        "subname": "",
        "touched": "2020-06-09T01:11:14.914074Z",
        "ttl": 60,
        "type": "AAAA"
    },
    {
        "created": "2020-06-08T11:50:05.481158Z",
        "domain": "c0.dedyn.io",
        "name": "c0.dedyn.io.",
        "records": [
            "77.180.11.103"
        ],
        "subname": "",
        "touched": "2020-06-09T01:11:14.907670Z",
        "ttl": 60,
        "type": "A"
    },
    {
        "created": "2020-06-08T11:50:05.470050Z",
        "domain": "c0.dedyn.io",
        "name": "*.c0.dedyn.io.",
        "records": [
            "c0.dedyn.io."
        ],
        "subname": "*",
        "touched": "2020-06-08T11:50:05.478375Z",
        "ttl": 3600,
        "type": "CNAME"
    },
    {
        "created": "2020-05-09T19:52:23.548996Z",
        "domain": "c0.dedyn.io",
        "name": "c0.dedyn.io.",
        "records": [
            "ns1.desec.io.",
            "ns2.desec.org."
        ],
        "subname": "",
        "touched": "2020-05-09T19:52:23.550359Z",
        "ttl": 3600,
        "type": "NS"
    }
]
➜  ~ docker run -it \                                                 
-e DESEC_TOKEN=<my access token> \
goacme/lego --accept-tos --dns desec --domains \*.c0.dedyn.io --email postmaster@dadac0.de run

2020/06/09 08:47:38 No key found for account postmaster@dadac0.de. Generating a P384 key.
2020/06/09 08:47:38 Saved key to /.lego/accounts/acme-v02.api.letsencrypt.org/postmaster@dadac0.de/keys/postmaster@dadac0.de.key
2020/06/09 08:47:39 [INFO] acme: Registering account for postmaster@dadac0.de
!!!! HEADS UP !!!!

Your account credentials have been saved in your Let's Encrypt
configuration directory at "/.lego/accounts".

You should make a secure backup of this folder now. This
configuration directory will also contain certificates and
private keys obtained from Let's Encrypt so making regular
backups of this folder is ideal.
2020/06/09 08:47:39 [INFO] [*.c0.dedyn.io] acme: Obtaining bundled SAN certificate
2020/06/09 08:47:40 [INFO] [*.c0.dedyn.io] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/5117326411
2020/06/09 08:47:40 [INFO] [*.c0.dedyn.io] acme: use dns-01 solver
2020/06/09 08:47:40 [INFO] [*.c0.dedyn.io] acme: Preparing to solve DNS-01
2020/06/09 08:47:40 [INFO] [*.c0.dedyn.io] acme: Trying to solve DNS-01
2020/06/09 08:47:40 [INFO] [*.c0.dedyn.io] acme: Checking DNS record propagation using [192.168.71.2:53]
2020/06/09 08:47:40 [INFO] Wait for propagation [timeout: 1m0s, interval: 2s]
2020/06/09 08:47:40 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:47:42 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:47:44 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:47:46 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:47:48 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:47:50 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:47:52 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:47:55 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:47:57 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:47:59 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:48:01 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:48:03 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:48:05 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:48:07 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:48:09 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:48:11 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:48:13 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:48:15 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:48:17 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:48:19 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:48:21 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:48:24 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:48:26 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:48:28 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:48:30 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:48:32 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:48:34 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:48:36 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:48:38 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 08:48:40 [INFO] [*.c0.dedyn.io] acme: Cleaning DNS-01 challenge
2020/06/09 08:48:40 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/5117326411
2020/06/09 08:48:41 Could not obtain certificates:
        error: one or more domains had a problem:
[*.c0.dedyn.io] time limit exceeded: last error: NS ns1.desec.io. did not return the expected TXT record [fqdn: c0.dedyn.io., value: Z9nl8zUEYa1CCBl_FhfUcdOHmeLAPTwIEehwf3n9hzA]: 
➜  ~

Q: Maybe reduce the TTL on the CNAME to prevent it from getting cached? A: Sorry, I don't have the background to make any educated decision here, but I'll try a TTL of 100 and see what happens... :man_shrugging:

@nils-wisiol :

If I'm not overlooking something, this could be a relatively easy change with a big impact on usability and flexibility.

Obviously I'd like that :+1: :smiley:

hcc23 commented 4 years ago

Good news: after changing the CNAME TTL to 100 (and waiting a bit) I tried the DNS-01 challenge again - and it worked :)

Question: Is there any harm in a TTL of 100?

However, goacme/lego still was apparently waiting a lot (not certain if that is to be expected, though):

➜  ~ curl -X GET https://desec.io/api/v1/domains/c0.dedyn.io/rrsets/\?type\=CNAME \
    --header "Authorization: Token <my access token>" | python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   196  100   196    0     0   1351      0 --:--:-- --:--:-- --:--:--  1351
[
    {
        "created": "2020-06-08T11:50:05.470050Z",
        "domain": "c0.dedyn.io",
        "name": "*.c0.dedyn.io.",
        "records": [
            "c0.dedyn.io."
        ],
        "subname": "*",
        "touched": "2020-06-09T09:19:15.707353Z",
        "ttl": 100,
        "type": "CNAME"
    }
]
➜  ~ docker run -it \
-e DESEC_TOKEN=<my access token> \
goacme/lego --accept-tos --dns desec --domains \*.c0.dedyn.io --email postmaster@dadac0.de run

2020/06/09 09:20:58 No key found for account postmaster@dadac0.de. Generating a P384 key.
2020/06/09 09:20:58 Saved key to /.lego/accounts/acme-v02.api.letsencrypt.org/postmaster@dadac0.de/keys/postmaster@dadac0.de.key
2020/06/09 09:20:59 [INFO] acme: Registering account for postmaster@dadac0.de
!!!! HEADS UP !!!!

Your account credentials have been saved in your Let's Encrypt
configuration directory at "/.lego/accounts".

You should make a secure backup of this folder now. This
configuration directory will also contain certificates and
private keys obtained from Let's Encrypt so making regular
backups of this folder is ideal.
2020/06/09 09:20:59 [INFO] [*.c0.dedyn.io] acme: Obtaining bundled SAN certificate
2020/06/09 09:21:00 [INFO] [*.c0.dedyn.io] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/5117795554
2020/06/09 09:21:00 [INFO] [*.c0.dedyn.io] acme: use dns-01 solver
2020/06/09 09:21:00 [INFO] [*.c0.dedyn.io] acme: Preparing to solve DNS-01
2020/06/09 09:21:00 [INFO] [*.c0.dedyn.io] acme: Trying to solve DNS-01
2020/06/09 09:21:00 [INFO] [*.c0.dedyn.io] acme: Checking DNS record propagation using [192.168.71.2:53]
2020/06/09 09:21:00 [INFO] Wait for propagation [timeout: 1m0s, interval: 2s]
2020/06/09 09:21:00 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 09:21:02 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 09:21:04 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 09:21:06 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 09:21:08 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 09:21:11 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 09:21:13 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 09:21:15 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 09:21:17 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 09:21:19 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 09:21:21 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 09:21:23 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 09:21:25 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 09:21:27 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 09:21:29 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 09:21:31 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 09:21:33 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 09:21:35 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 09:21:37 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 09:21:40 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 09:21:42 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 09:21:44 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 09:21:46 [INFO] [*.c0.dedyn.io] acme: Waiting for DNS record propagation.
2020/06/09 09:21:56 [INFO] [*.c0.dedyn.io] The server validated our request
2020/06/09 09:21:56 [INFO] [*.c0.dedyn.io] acme: Cleaning DNS-01 challenge
2020/06/09 09:21:56 [INFO] [*.c0.dedyn.io] acme: Validations succeeded; requesting certificates
2020/06/09 09:21:57 [INFO] [*.c0.dedyn.io] Server responded with a certificate.
nils-wisiol commented 4 years ago

@hcc23 Here is my best guess on what happend:

  1. You set up CNAME *.c0.dedyn.io c0.dedyn.io., let's assume propagation of these changes already happened.
  2. The acme client set up TXT _acme-challenge.c0.dedyn.io, but changes do not get propagated immediately.
  3. The acme client queries TXT _acme-challenge.c0.dedyn.io, and receives as an answer that this is a CNAME for c0.dedyn.io, hence TXT c0.dedyn.io. should be queried (but there is no TXT).
  4. Changes propagate
  5. The acme client now queries TXT c0.dedyn.io, due to the cached CNAME response.

In the old setup, acme client would eventually give up, delete TXT _acme-challenge.c0.dedyn.io ("cleaning up" in log). Afterwards, Let's Encrypt would see an outdated challenge respones (we just deleted the correct one).

In the new setup, the CNAME in the resolver cache of the acme client would eventually expire, and the query would be answered by actually querying TXT _acme-challenge.c0.dedyn.io. Afterwards, Let's Encrypt would see the correct result.

Bottom line, the acme client should not cache CNAME responses when waiting for DNS record propagation, and this is not an issue with deSEC.

The feature request for dyn updates of subdomains should best be handled in a separate issue or PR.

hcc23 commented 4 years ago

Shall I create the feature request issue for the update of subdomains?

nils-wisiol commented 4 years ago

Sure, thanks!