TSL Certificate Provisioning Issue (since v0.54)

[cmasterdelphi]

Since July (2021) in am experience issues with the TSL certificate provisioning. I constantly get the messages, that the certificates were not able to be updated (automatically) because the DNS/IP was not found. If I update them manually, it works but that's not really a solution.

The issues affects all domains on my MiaB (including the primary domain, which is used to host MiaB) I get always following messages: (note: mydomain.tld = one example of my domains)

Provisioning TLS certificates for mydomain.tld, autoconfig.mydomain.tld, autodiscover.mydomain.tld, mta-sts.mydomain.tld, www.mydomain.tld.
error: mydomain.tld, autoconfig.mydomain.tld, autodiscover.mydomain.tld, mta-sts.mydomain.tld, www.mydomain.tld:
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Performing the following challenges:
http-01 challenge for autoconfig.mydomain.tld
http-01 challenge for autodiscover.mydomain.tld
http-01 challenge for mta-sts.mydomain.tld
http-01 challenge for mydomain.tld
http-01 challenge for www.mydomain.tld
Using the webroot path /home/user-data/ssl/lets_encrypt/webroot for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. autodiscover.mydomain.tld (http-01): urn:ietf:params:acme:error:dns :: During secondary validation: DNS problem: query timed out looking up CAA for tld, www.mydomain.tld (http-01): urn:ietf:params:acme:error:dns :: During secondary validation: DNS problem: SERVFAIL looking up CAA for mydomain.tld - the domain's nameservers may be malfunctioning, mydomain.tld (http-01): urn:ietf:params:acme:error:dns :: During secondary validation: DNS problem: query timed out looking up A for mydomain.tld, autoconfig.mydomain.tld (http-01): urn:ietf:params:acme:error:dns :: During secondary validation: No valid IP addresses found for autoconfig.mydomain.tld
IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: autodiscover.mydomain.tld
   Type:   None
   Detail: During secondary validation: DNS problem: query timed out
   looking up CAA for tld

   Domain: www.mydomain.tld
   Type:   None
   Detail: During secondary validation: DNS problem: SERVFAIL looking
   up CAA for mydomain.tld - the domain's nameservers may be
   malfunctioning

   Domain: mydomain.tld
   Type:   None
   Detail: During secondary validation: DNS problem: query timed out
   looking up A for mydomain.tld

   Domain: autoconfig.mydomain.tld
   Type:   None
   Detail: During secondary validation: No valid IP addresses found
   for autoconfig.mydomain.tld

The log does not provide any additional information. Here an excerpt:

2021-08-22 03:03:04,616:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/authz-v3/23978598960 HTTP/1.1" 200 1017
2021-08-22 03:03:04,616:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Sun, 22 Aug 2021 01:03:05 GMT
Content-Type: application/json
Content-Length: 1017
Connection: keep-alive
Boulder-Requester: 73605419
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: 0001dDQdeu_7DtH696UpfigaB6QMAnpUjwWuqJv74B_sXmA
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "identifier": {
    "type": "dns",
    "value": "autoconfig.mydomain.tld"
  },
  "status": "invalid",
  "expires": "2021-08-29T01:02:32Z",
  "challenges": [
    {
      "type": "http-01",
      "status": "invalid",
      "error": {
        "type": "urn:ietf:params:acme:error:dns",
        "detail": "During secondary validation: DNS problem: query timed out looking up A for autoconfig.mydomain.tld",
        "status": 400
      },
      "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/23978598960/j4yq5Q",
      "token": "_sometokenhashiremovednow_",
      "validationRecord": [
        {
          "url": "http://autoconfig.mydomain.tld/.well-known/acme-challenge/_sometokenhashiremovednow_",
          "hostname": "autoconfig.mydomain.tld",
          "port": "80",
          "addressesResolved": [
            "_0.0.0.0 (my-actual-ip was stated here)_"
          ],
          "addressUsed": "_0.0.0.0 (my-actual-ip was stated here)_"
          }
      ],
      "validated": "2021-08-22T01:02:33Z"
    }
  ]
}

Anyway, the only change between the last successful provisioning in March and the next failed attempt were as following: (There had not been any other changes besides that. No change in the IP/ReverseDNS, no change in the nameservers or at the side of the domain providers. I even use multiple domain providers and this issue occurs for all domains.)

UpDated MiaB Version: 07.07.2021

   0.53 -> 0.54

Installed/Updated packages:

06.06.2021

   python-apt-common (1.6.5ubuntu0.6)
   python3-apt (1.6.5ubuntu0.6)
   grub-efi-amd64 (2.04-1ubuntu44.1)
   grub-efi-amd64-signed (1.167~18.04.3+2.04-1ubuntu44.1)
   grub-efi-amd64-bin (2.04-1ubuntu44.1)
   linux-libc-dev (4.15.0-144.148)

26.06.2021

   libnss-systemd (237-3ubuntu10.48)
   libsystemd0 (237-3ubuntu10.48)
   libpam-systemd (237-3ubuntu10.48)
   systemd (237-3ubuntu10.48)
   udev (237-3ubuntu10.48)
   libudev1 (237-3ubuntu10.48)
   systemd-sysv (237-3ubuntu10.48)
   ubuntu-advantage-tools (27.1~18.04.1)
   linux-libc-dev (4.15.0-147.151)
   cloud-init (21.2-3-g899bfaa9-0ubuntu2~18.04.1)

10.07.2021

   [24 standard security updates] ()
   libavahi-common-data (0.7-3.1ubuntu1.3)
   libavahi-common3 (0.7-3.1ubuntu1.3)
   libavahi-client3 (0.7-3.1ubuntu1.3)
   php7.2-gd (7.2.24-0ubuntu0.18.04.8)
   php7.2-gmp (7.2.24-0ubuntu0.18.04.8)
   php7.2-opcache (7.2.24-0ubuntu0.18.04.8)
   php7.2-intl (7.2.24-0ubuntu0.18.04.8)
   php7.2-json (7.2.24-0ubuntu0.18.04.8)
   php7.2-bcmath (7.2.24-0ubuntu0.18.04.8)
   php7.2-readline (7.2.24-0ubuntu0.18.04.8)
   php7.2-soap (7.2.24-0ubuntu0.18.04.8)
   php7.2-mbstring (7.2.24-0ubuntu0.18.04.8)
   php7.2-curl (7.2.24-0ubuntu0.18.04.8)
   php7.2-imap (7.2.24-0ubuntu0.18.04.8)
   php7.2-ldap (7.2.24-0ubuntu0.18.04.8)
   php7.2-sqlite3 (7.2.24-0ubuntu0.18.04.8)
   php7.2-zip (7.2.24-0ubuntu0.18.04.8)
   php7.2-pspell (7.2.24-0ubuntu0.18.04.8)
   php7.2-xml (7.2.24-0ubuntu0.18.04.8)
   php7.2-dev (7.2.24-0ubuntu0.18.04.8)
   php7.2-cli (7.2.24-0ubuntu0.18.04.8)
   php7.2-fpm (7.2.24-0ubuntu0.18.04.8)
   php7.2-common (7.2.24-0ubuntu0.18.04.8)
   php7.2 (7.2.24-0ubuntu0.18.04.8)

[clonejo]

Keep getting the same problem, though also sometimes with this error:

Detail: During secondary validation: DNS problem: SERVFAIL looking up CAA for www.example.com - the domain's nameservers may be malfunctioning

Weirdly enough, it only fails for the first 4-5 certs. The later certs also being renewed in the same nightly job get processed just fine.

Manually running cd /root/mailinabox && management/daily_tasks.sh (as root) renewed all certs successfully.

[cmasterdelphi]

@clonejo are you still experience these problems. Mine somehow seem to have vanished by themselves at some point in time. I'm regularly checking in, but so far, there have not been any new certification issues.

[clonejo]

@cmasterdelphi No problems, recent renewals have been fine.