Certbot issues with DNS challenges

[GDowning14]

Checklist

• Have you pulled and found the error with jc21/nginx-proxy-manager:latest docker image? Yes
• Are you sure you're not using someone else's docker image? Yes
• Have you searched for similar issues (both open and closed)? Yes

Describe the bug Certbot certificate renewal when using DNS challenge for GoDaddy fails with the following error:

ImportError: cannot import name 'ClientBase' from 'acme.client'

The entire stack trace is: Error: Command failed: certbot certonly --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --cert-name "npm-1" --agree-tos --email "myemail@mydomain" --domains "my.domain" --authenticator dns-godaddy --dns-godaddy-credentials "/etc/letsencrypt/credentials/credentials-1" Traceback (most recent call last): File "/usr/bin/certbot", line 5, in from certbot.main import main File "/opt/certbot/lib/python3.7/site-packages/certbot/main.py", line 6, in from certbot._internal import main as internal_main File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/main.py", line 28, in from certbot import crypto_util File "/opt/certbot/lib/python3.7/site-packages/certbot/crypto_util.py", line 42, in from certbot import interfaces File "/opt/certbot/lib/python3.7/site-packages/certbot/interfaces.py", line 21, in from acme.client import ClientBase ImportError: cannot import name 'ClientBase' from 'acme.client' (/opt/certbot/lib/python3.7/site-packages/acme/client.py)

at ChildProcess.exithandler (node:child_process:402:12)
at ChildProcess.emit (node:events:513:28)

Nginx Proxy Manager Version 2.10.4

To Reproduce Steps to reproduce the behavior:

1. Go to '](https://nginxproxymanager.com/guide/#hosting-your-home-network)'
2. Follow steps as describe
3. Nginx Proxy Manager comntainer is created, starts, and login is accessible at port 81.
4. Navigate to SSL Certificates tab and click on Add SSL Certificate
5. Fill in form, selecting DNS Challenge, GoDaddy Provider, and provide Key and Secret
6. Click submit
7. Receive error

Expected behavior Expected to receive a Let's Encrypt certificate for my domain

Screenshots

Operating System Raspberry PI OS 64-bit on RPi 4 model B 8GB. (Debian 11.7 for arm64)

Additional context I have another instance of NPM (release 2.10.2) running on an amd64 host (Linux Mint LMDE 5 - Debian 11.2). This instance has been running for a couple of years now with no issues. I am attempting to move NPM from the LMDE host to a RPI host. I also tried pulling the 2.10.2 version that is running fine on LMDE and bring it up on the RPI but received a different error (below).

Both the issue with 2.10.4 above, and the issue with 2.10.2 below appear to be issues with Certbot. I did find posts that seem to indicate that the version of Certbot packaged with these release is somewhat dated. Are there known issues with Certbot on RPI?

** Error noted with NPM 2.10.2 on RPi 4 B 8GB - Debian 11.7 ARM64 Error: Command failed: . /opt/certbot/bin/activate && pip install --no-cache-dir --user certbot-dns-godaddy~=0.2.0 && deactivate ERROR: Will not install to the user site because it will lack sys.path precedence to certbot in /opt/certbot/lib/python3.7/site-packages

[notice] A new release of pip is available: 23.0.1 -> 23.2.1 [notice] To update, run: pip install --upgrade pip

at ChildProcess.exithandler (node:child_process:402:12)
at ChildProcess.emit (node:events:513:28)
at maybeClose (node:internal/child_process:1100:16)
at Socket. (node:internal/child_process:458:11)
at Socket.emit (node:events:513:28)
at Pipe. (node:net:301:12)

[tiiraluoto]

Same problem here with ClouDNS. The output I get is the same as yours but I have this at the end of it: `ImportError: cannot import name 'ClientBase' from 'acme.client' (/opt/certbot/lib/python3.7/site-packages/acme/client.py)

at ChildProcess.exithandler (node:child_process:402:12)
at ChildProcess.emit (node:events:513:28)
at maybeClose (node:internal/child_process:1100:16)
at Process.ChildProcess._handle.onexit (node:internal/child_process:304:5)`

[Atredies]

Same issue here with Route53. I get the same error.

Failed to renew certificate npm-2 with error: Unable to locate credentials
To use certbot-dns-route53, configure credentials as described at https://boto3.readthedocs.io/en/latest/guide/configuration.html#best-practices-for-configuring-credentials and add the necessary permissions for Route53 access.
All renewals failed. The following certificates could not be renewed:
  /etc/letsencrypt/live/npm-2/fullchain.pem (failure)
1 renew failure(s), 0 parse failure(s)

    at ChildProcess.exithandler (node:child_process:402:12)
    at ChildProcess.emit (node:events:513:28)
    at maybeClose (node:internal/child_process:1100:16)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:304:5)

Additionally I've seen that the CPU gets overloaded on my sever since it's spawning multiple processes for:

node --abort_on_uncaught_exception --max_old_space_size=250 index.js

It's also really weird, as long as I don't touch the web interface or try to renew the certificates or anything like that, the nginx part works just fine and the applications are online and working.

[GDowning14]

UPDATE: It appears this is a version/dependency issue - perhaps in the docker-nginx-full base docker image? Opening a shell into the container and running the certbot functions from the command line also reports the same errors as noted in the NPM certificates window. A 'pip list' shows the following Python packages installed:

Package             Version
------------------- ---------
acme                2.6.0
certbot             1.32.0
certbot-dns-godaddy 0.2.4

Version 2.6.0 of the "acme" package appears to require a more recent version of the "certbot" packages.

After running

pip install --upgrade certbot certbot-dns-godaddy

the "pip list" now shows

Package             Version
------------------- ---------
acme                2.6.0
certbot             2.6.0
certbot-dns-godaddy 2.6.0

Running certbot from the command line now succeeds and no errors are reported.

Unfortunately, this does not seem to resolve the original problem. When again attempting to obtain a certificate via NPM, the same error is reported. Using the shell and "pip list" shows that the certbot package upgrades have been reverted and are back to the original, incompatible, versions.

Being an enterprise developer, I am new to both Python and Docker and am unable to determine how/why this downgrade is happening when using NPM to launch the certbot scripts.

[GDowning14]

The following resolved the issue for me. Use the docker shell, updated /app/global/certbot-dns-providers.js to set the required version of "certbot-dns-godaddy" to "~=2.6.0". Submitted pull request #3147

[jerimiah797]

I resolved all this by combining several steps I saw in this (and other) threads to build the latest certbot/acme in the docker container. I'm now running with everything on 'latest', and did not have to change the godaddy version anywhere. :-)

Shell into your nginx docker container as root. I used portainer so it was very simple. Install dependencies:

apt update
apt install python3-dev libffi-dev pkg-config libssl-dev

Install Rust - the apt version of rustc is too old (1.41.1, >1.56.0 is required)

curl https://sh.rustup.rs -sSf | sh
source "$HOME/.cargo/env"

Activate python venv source /opt/certbot/bin/activate

Build and install certbot/acme

pip install certbot
pip install acme
pip install certbot-dns-godaddy

After all this, I still got an error trying to renew my godaddy domain. Evidently it now requires domain validation. If you see this problem in the logs, go to your DNS settings at the GoDaddy website and add a TXT entry called _acme-challenge and put the key that you see in the error log as the value. Then the DNS challenge will succeed.

[djmixman]

Add another one for ClouDNS. Was trying to setup NPM for the first time and ran into this problem.

Same problem here with ClouDNS. The output I get is the same as yours but I have this at the end of it: `ImportError: cannot import name 'ClientBase' from 'acme.client' (/opt/certbot/lib/python3.7/site-packages/acme/client.py)

at ChildProcess.exithandler (node:child_process:402:12)
at ChildProcess.emit (node:events:513:28)
at maybeClose (node:internal/child_process:1100:16)
at Process.ChildProcess._handle.onexit (node:internal/child_process:304:5)`

[github-actions[bot]]

Issue is now considered stale. If you want to keep it open, please comment :+1: