DNS challenge with DuckDNS does not work

[Rihan9]

Checklist

• Have you pulled and found the error with jc21/nginx-proxy-manager:latest docker image?
  • Yes
• Are you sure you're not using someone else's docker image?
  • Yes
• Have you searched for similar issues (both open and closed)?
  • Yes

Describe the bug

This isseus is some sort of "reopening" of #2092. We can't request a SSL certificate for a domain hosted on DuckDNS. In the log we found the following message: "The DNS response does not contain an answer to the question: mydomain.duckdns.org. IN TXT"

Nginx Proxy Manager Version

v2.10.3

To Reproduce Steps to reproduce the behavior:

1. create a hostname on duckDNS and make sure is pointed to the nginx proxy manager installation
2. on tab ssl certificates --> add ssl certificates --> let's encrypt --> compelte the form using dns challange: yes and the required token (see screenshot)
3. the error should pop-up

Screenshots

[holocronology]

Did you delete a cert prior to this? I think there is an error in NPM that the database is not updating when certs are deleted. So NPM starting looking for the folder of the deleted cert, but it is not there. SO it ends up in an error loop looking for these deleted files.

[Rihan9]

I don't think so. The only think that come to my mind is the renew certifica process. Can it cause the deletion of the old certificate?

[devedse]

I'm running into exactly the same issue. I've tried to clean everything up an recreate the certificate but no matter what I tried I keep having this issue.

[devedse]

Ok I hacked the solution to work.

I found out here: https://github.com/infinityofspace/certbot_dns_duckdns/issues/21 That apparently you need to add --dns-duckdns-no-txt-restore to the certbot command

So I added that in the certificate.js file.

To fix it on your local docker instance:

Download this file: https://gist.github.com/devedse/b63f79851c44aac27892c95009090325 And mount it in the docker container as a volume:

volumes:
      - /home/devedse/dockercomposers/nginxproxymanager/certificate.js:/app/internal/certificate.js:ro

My full compose file:

services:
  nginx-proxy-manager:
    image: 'jc21/nginx-proxy-manager:latest'
    container_name: nginx-proxy-manager
    restart: unless-stopped
    volumes:
      - /home/devedse/dockercomposers/nginxproxymanager/data:/data
      - /home/devedse/dockercomposers/nginxproxymanager/letsencrypt:/etc/letsencrypt
      - /home/devedse/dockercomposers/nginxproxymanager/snippets:/config/nginx/snippets:ro
      - /home/devedse/dockercomposers/nginxproxymanager/certificate.js:/app/internal/certificate.js:ro
    network_mode: "host"

[devedse]

I'm honestly not sure why and how this works but I would advice the maintainers of nginx-proxy-manager to take a look at this and possibly include this fix in the solution.

@jc21 :)

[devedse]

For renewing certificates you need to update the renewal config. In my case here (only required if they were generated before you applied this fix):

/home/devedse/dockercomposers/nginxproxymanager/letsencrypt/renewal/npm-18.conf

Then add this line:

dns_duckdns_no_txt_restore = True

The complete file should look like this:

# renew_before_expiry = 30 days
version = 2.5.0
archive_dir = /etc/letsencrypt/archive/npm-18
cert = /etc/letsencrypt/live/npm-18/cert.pem
privkey = /etc/letsencrypt/live/npm-18/privkey.pem
chain = /etc/letsencrypt/live/npm-18/chain.pem
fullchain = /etc/letsencrypt/live/npm-18/fullchain.pem

# Options used in the renewal process
[renewalparams]
account = *******************
key_type = ecdsa
elliptic_curve = secp384r1
preferred_chain = ISRG Root X1
work_dir = /tmp/letsencrypt-lib
logs_dir = /tmp/letsencrypt-log
authenticator = dns-duckdns
dns_duckdns_propagation_seconds = 30
dns_duckdns_credentials = /etc/letsencrypt/credentials/credentials-18
dns_duckdns_no_txt_restore = True
webroot_path = /data/letsencrypt-acme-challenge,
server = https://acme-v02.api.letsencrypt.org/directory
[[webroot_map]]

After this renewal's should work again for DuckDNS

[Rihan9]

@devedse please, let me bow to you.

I can confirm that the fix work like a charm. (I didn't use the binding volumes: i direclty used curl on the container's console to download the new file and replaced the old).

Do I need to touch the config of my cert if it was created after your change? Because if it is needed, I think we need an additional change on the code to avoid it.

[devedse]

Nope then you don't need to change it as far as I understand. The disadvantage of your way is that if you restart/update the container you need to re-apply the change.

[Rihan9]

Yeah, I know, but I'm lazy and I didn't want to access throght ssh, so I used the console via Portainer. I also don't see why your fix should be posponed in an update other than the next, given that we have 2 different positive response and your fix seems to me quite clean. So I'm positive that there will be no need to re-apply the fix

[devedse]

The main thing is that I don't understand "why" my fix is needed. I don't understand DNS well enough for that. The setting we've added should keep the TXT on duckdns after we've proven that we are the owner of said domain. However I don't understand why it needs to remain there. It would make sense to only need to remain there for the duration of the certbot renew.

[Rihan9]

I don't have an answer for you. I know almost nothing about it. My knowledge as a developer is related to CRM software and atomation systems thought python, I'm not a network manager. I'm using this software because is the most efficent and easy solution that I have found to manage my services related to my smart home.

[kruechten]

I can confirm this bug is still present and the fix to the container.js from devedse above works. To fix without mounting the file inside your container, you can also just overwrite /app/internal/certificate.js with the contents of his version. Did that through a shell in portainer and now renewal and new certificates work again via DuckDNS challenge.

[Dark-Noir]

Hmm, i can't get it to work, i used @devedse solution here: https://github.com/NginxProxyManager/nginx-proxy-manager/issues/3138#issuecomment-1691447683 The file is correctly linked in to the container.

CommandError: Saving debug log to /tmp/letsencrypt-log/letsencrypt.log
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /tmp/letsencrypt-log/letsencrypt.log or re-run Certbot with -v for more details.

    at /app/lib/utils.js:16:13
    at ChildProcess.exithandler (node:child_process:430:5)
    at ChildProcess.emit (node:events:519:28)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5)

Certbot failed to authenticate some domains (authenticator: dns-duckdns). The Certificate Authority reported these problems:
  Domain: ****.duckdns.org
  Type:   dns
  Detail: DNS problem: SERVFAIL looking up TXT for _acme-challenge.****.duckdns.org - the domain's nameservers may be malfunctioning

Hint: The Certificate Authority failed to verify the DNS TXT records created by --dns-duckdns. Ensure the above domains are hosted by this DNS provider, or try increasing --dns-duckdns-propagation-se>

2024-06-22 21:16:03,583:DEBUG:certbot._internal.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/auth_handler.py", line 108, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, max_time_mins, best_effort)
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/auth_handler.py", line 212, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.

2024-06-22 21:16:03,584:DEBUG:certbot._internal.error_handler:Calling registered functions
2024-06-22 21:16:03,584:INFO:certbot._internal.auth_handler:Cleaning up challenges
2024-06-22 21:16:04,166:DEBUG:charset_normalizer:Encoding detection: ascii is most likely the one.
2024-06-22 21:16:04,169:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "/opt/certbot/bin/certbot", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/certbot/lib/python3.11/site-packages/certbot/main.py", line 19, in main
    return internal_main.main(cli_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/main.py", line 1894, in main
    return config.func(config, plugins)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/main.py", line 1600, in certonly
    lineage = _get_and_save_cert(le_client, config, domains, certname, lineage)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/main.py", line 143, in _get_and_save_cert
    lineage = le_client.obtain_and_enroll_certificate(domains, certname)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/client.py", line 517, in obtain_and_enroll_certificate
    cert, chain, key, _ = self.obtain_certificate(domains)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/client.py", line 428, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/client.py", line 496, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, self.config, best_effort)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/auth_handler.py", line 108, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, max_time_mins, best_effort)
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/auth_handler.py", line 212, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.
2024-06-22 21:16:04,175:ERROR:certbot._internal.log:Some challenges have failed.