search

NginxProxyManager / nginx-proxy-manager

Docker container for managing Nginx proxy hosts with a simple, powerful interface
https://nginxproxymanager.com
MIT License
21.92k stars 2.53k forks source link

Add ClouDNS LetsEncrypt certificate #2844

Open phillipmcmahon opened 1 year ago

phillipmcmahon commented 1 year ago

Checklist

Describe the bug Error on cert generation. Error text below.

Nginx Proxy Manager Version

Error: Command failed: . /opt/certbot/bin/activate && pip install --no-cache-dir --user certbot-dns-cloudns~=0.4.0 && deactivate ERROR: Will not install to the user site because it will lack sys.path precedence to certbot in /opt/certbot/lib/python3.7/site-packages

[notice] A new release of pip is available: 23.0.1 -> 23.1 [notice] To update, run: pip install --upgrade pip

at ChildProcess.exithandler (node:child_process:402:12)
at ChildProcess.emit (node:events:513:28)
at maybeClose (node:internal/child_process:1100:16)
at Process.ChildProcess._handle.onexit (node:internal/child_process:304:5)

Screenshots

image

vpont commented 1 year ago

I got the same error after upgrading:

[notice] A new release of pip is available: 23.0.1 -> 23.1
[notice] To update, run: pip install --upgrade pip

[4/21/2023] [10:24:33 AM] [Migrate  ] › ℹ  info      Current database version: none
[4/21/2023] [10:24:36 AM] [Global   ] › ✖  error     Command failed: . /opt/certbot/bin/activate && pip install --no-cache-dir --user certbot-dns-cloudns~=0.4.0  && deactivate
ERROR: Will not install to the user site because it will lack sys.path precedence to certbot in /opt/certbot/lib/python3.7/site-packages
vpont commented 1 year ago

Related to: https://github.com/NginxProxyManager/nginx-proxy-manager/issues/2776

phillipmcmahon commented 1 year ago

Upgraded to 2.10.3 and still getting the same error when attempting to generate a cert using Cloudns plugin.

Help! :)

@jc21

mashb1t commented 1 year ago

i encountered the same issue and would appreciate a fix

jeffshead commented 1 year ago

I also encountered the same issue and got the same error with:latest (2.10.3).

So I pulled the 2971 pr image (https://github.com/NginxProxyManager/nginx-proxy-manager/pull/2971) but it did not solve the issue. Now, I get the error below which is different:

Error: Command failed: certbot certonly --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --cert-name "npm-21" --agree-tos --email "user@example.com" --domains "*.example.com,example.com" --authenticator dns-cloudns --dns-cloudns-credentials "/etc/letsencrypt/credentials/credentials-21"
Traceback (most recent call last):
  File "/usr/bin/certbot", line 5, in 
    from certbot.main import main
  File "/opt/certbot/lib/python3.7/site-packages/certbot/main.py", line 6, in 
    from certbot._internal import main as internal_main
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/main.py", line 28, in 
    from certbot import crypto_util
  File "/opt/certbot/lib/python3.7/site-packages/certbot/crypto_util.py", line 42, in 
    from certbot import interfaces
  File "/opt/certbot/lib/python3.7/site-packages/certbot/interfaces.py", line 21, in 
    from acme.client import ClientBase
ImportError: cannot import name 'ClientBase' from 'acme.client' (/opt/certbot/lib/python3.7/site-packages/acme/client.py)

    at ChildProcess.exithandler (node:child_process:402:12)
    at ChildProcess.emit (node:events:513:28)
    at maybeClose (node:internal/child_process:1100:16)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:304:5)
wolviex commented 1 year ago

@jeffshead

File "/opt/certbot/lib/python3.7/site-packages/certbot/interfaces.py", line 21, in from acme.client import ClientBase

I would have expected this to be 

from acme.client import ClientV2

It sounds like you somehow have conflicting versions of python modules acme/certbot/certbot-nginx. Are you able to confirm the version of the certbot module? I'm seeing 2.5.0 You could try pruning all related images and builds and pulling/building fresh.

@phillipmcmahon @vpont @mashb1t - is this the same as #2921 ?

jeffshead commented 1 year ago

You could try pruning all related images and builds and pulling/building fresh.

I did pull new images and I ran docker image prune -a and docker system prune -a. I have no idea how there could be conflicting versions. I've not modified either image. Could there be something static in one of the NPM Docker volumes (letsencrypt or data) that needs to be deleted/edited?

Are you able to confirm the version of the certbot module?

certbot --version returns 2.5.0

I don't have all that much experience with Docker. What other steps do I need to perform when switching from :latest to the :github-pr-2971 image? I used Portainer to recreate the container, re-pull the image and rebooted the server.

Also, I have a CanaryTokens container on the same server. Could this be interfering in someway since it also has nginx and certbot-nginx? But it was working with DNSMadeEasy challenge before I updated NPM to 2.10.3.

phillipmcmahon commented 1 year ago

@jeffshead

File "/opt/certbot/lib/python3.7/site-packages/certbot/interfaces.py", line 21, in from acme.client import ClientBase

I would have expected this to be

from acme.client import ClientV2

It sounds like you somehow have conflicting versions of python modules acme/certbot/certbot-nginx. Are you able to confirm the version of the certbot module? I'm seeing 2.5.0 You could try pruning all related images and builds and pulling/building fresh.

@phillipmcmahon @vpont @mashb1t - is this the same as #2921 ?

Not too sure, I pulled the version referenced in that issue. Prior to that did a docker system prune -a, and when trying to generate a cert get a similar looking error.

Internal Error
Error: Command failed: certbot certonly --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --cert-name "npm-15" --agree-tos --email "phillip.mcmahon@gmail.com" --domains "cloud.phillipmcmahon.com" --authenticator dns-cloudns --dns-cloudns-credentials "/etc/letsencrypt/credentials/credentials-15" --dns-cloudns-propagation-seconds 180
Traceback (most recent call last):
  File "/usr/bin/certbot", line 5, in 
    from certbot.main import main
  File "/opt/certbot/lib/python3.7/site-packages/certbot/main.py", line 6, in 
    from certbot._internal import main as internal_main
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/main.py", line 28, in 
    from certbot import crypto_util
  File "/opt/certbot/lib/python3.7/site-packages/certbot/crypto_util.py", line 42, in 
    from certbot import interfaces
  File "/opt/certbot/lib/python3.7/site-packages/certbot/interfaces.py", line 21, in 
    from acme.client import ClientBase
ImportError: cannot import name 'ClientBase' from 'acme.client' (/opt/certbot/lib/python3.7/site-packages/acme/client.py)

    at ChildProcess.exithandler (node:child_process:402:12)
    at ChildProcess.emit (node:events:513:28)
    at maybeClose (node:internal/child_process:1100:16)
    at Process.ChildProcess._handle.onexit (node:internal/child
wolviex commented 1 year ago

ImportError: cannot import name 'ClientBase' from 'acme.client' (/opt/certbot/lib/python3.7/site-packages/acme/client.py)

is quite different from this issue's

ERROR: Will not install to the user site because it will lack sys.path precedence to certbot in /opt/certbot/lib/python3.7/site-packages

It might make the most sense to open a new issue for that.

What you're seeing looks like a mismatch between the certbot and acme.client modules - but I'm really not sure how you're getting there. Try attaching a console to the container and checking their versions.

/opt/certbot/bin/pip show acme certbot
jeffshead commented 1 year ago

@wolviex - Thank you for taking the time.

/opt/certbot/bin/pip show acme certbot:

Name: acme
Version: 2.5.0
Summary: ACME protocol implementation in Python
Home-page: https://github.com/letsencrypt/letsencrypt
Author: Certbot Project
Author-email: certbot-dev@eff.org
License: Apache License 2.0
Location: /opt/certbot/lib/python3.7/site-packages
Requires: cryptography, josepy, PyOpenSSL, pyrfc3339, pytz, requests, setuptools
Required-by: certbot, certbot-dns-dnsmadeeasy
---
Name: certbot
Version: 1.32.0
Summary: ACME client
Home-page: https://github.com/letsencrypt/letsencrypt
Author: Certbot Project
Author-email: certbot-dev@eff.org
License: Apache License 2.0
Location: /opt/certbot/lib/python3.7/site-packages
Requires: acme, ConfigArgParse, configobj, cryptography, distro, josepy, parsedatetime, pyrfc3339, pytz, setuptools, zope.component, zope.interface
Required-by: certbot-dns-cloudns, certbot-dns-dnsmadeeasy
wolviex commented 1 year ago

I still have no idea how you have that old version of certbot in this container.

/opt/certbot/bin/pip install certbot==2.5.0

Will sort you out, at least temporarily.

jeffshead commented 1 year ago

Unfortunatly, that command did not fix the issue.

/opt/certbot/bin/pip install certbot==2.5.0 output:

Requirement already satisfied: certbot==2.5.0 in /opt/certbot/lib/python3.7/site-packages (2.5.0)
Requirement already satisfied: acme>=2.5.0 in /opt/certbot/lib/python3.7/site-packages (from certbot==2.5.0) (2.5.0)
Requirement already satisfied: ConfigArgParse>=0.9.3 in /opt/certbot/lib/python3.7/site-packages (from certbot==2.5.0) (1.5.3)
Requirement already satisfied: configobj>=5.0.6 in /opt/certbot/lib/python3.7/site-packages (from certbot==2.5.0) (5.0.8)
Requirement already satisfied: cryptography>=2.5.0 in /opt/certbot/lib/python3.7/site-packages (from certbot==2.5.0) (2.8)
Requirement already satisfied: distro>=1.0.1 in /opt/certbot/lib/python3.7/site-packages (from certbot==2.5.0) (1.8.0)
Requirement already satisfied: josepy>=1.13.0 in /opt/certbot/lib/python3.7/site-packages (from certbot==2.5.0) (1.13.0)
Requirement already satisfied: parsedatetime>=2.4 in /opt/certbot/lib/python3.7/site-packages (from certbot==2.5.0) (2.6)
Requirement already satisfied: pyrfc3339 in /opt/certbot/lib/python3.7/site-packages (from certbot==2.5.0) (1.1)
Requirement already satisfied: pytz>=2019.3 in /opt/certbot/lib/python3.7/site-packages (from certbot==2.5.0) (2023.3)
Requirement already satisfied: setuptools>=41.6.0 in /opt/certbot/lib/python3.7/site-packages (from certbot==2.5.0) (67.8.0)
Requirement already satisfied: PyOpenSSL!=23.1.0,>=17.5.0 in /opt/certbot/lib/python3.7/site-packages (from acme>=2.5.0->certbot==2.5.0) (19.1.0)
Requirement already satisfied: requests>=2.20.0 in /opt/certbot/lib/python3.7/site-packages (from acme>=2.5.0->certbot==2.5.0) (2.31.0)
Requirement already satisfied: six in /opt/certbot/lib/python3.7/site-packages (from configobj>=5.0.6->certbot==2.5.0) (1.16.0)
Requirement already satisfied: cffi!=1.11.3,>=1.8 in /opt/certbot/lib/python3.7/site-packages (from cryptography>=2.5.0->certbot==2.5.0) (1.15.1)
Requirement already satisfied: pycparser in /opt/certbot/lib/python3.7/site-packages (from cffi!=1.11.3,>=1.8->cryptography>=2.5.0->certbot==2.5.0) (2.21)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/certbot/lib/python3.7/site-packages (from requests>=2.20.0->acme>=2.5.0->certbot==2.5.0) (3.1.0)
Requirement already satisfied: idna<4,>=2.5 in /opt/certbot/lib/python3.7/site-packages (from requests>=2.20.0->acme>=2.5.0->certbot==2.5.0) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/certbot/lib/python3.7/site-packages (from requests>=2.20.0->acme>=2.5.0->certbot==2.5.0) (2.0.2)
Requirement already satisfied: certifi>=2017.4.17 in /opt/certbot/lib/python3.7/site-packages (from requests>=2.20.0->acme>=2.5.0->certbot==2.5.0) (2023.5.7)

I also tried to update just to see what would happen: /opt/certbot/bin/pip install --upgrade pip output:

Requirement already satisfied: pip in /opt/certbot/lib/python3.7/site-packages (23.1.2)

Initial output when accessing console:

 _   _       _            ____                      __  __                                   
| \ | | __ _(_)_ __ __  _|  _ \ _ __ _____  ___   _|  \/  | __ _ _ __   __ _  __ _  ___ _ __ 
|  \| |/ _` | | '_ \\ \/ / |_) | '__/ _ \ \/ / | | | |\/| |/ _` | '_ \ / _` |/ _` |/ _ \ '__|
| |\  | (_| | | | | |>  <|  __/| | | (_) >  <| |_| | |  | | (_| | | | | (_| | (_| |  __/ |   
|_| \_|\__, |_|_| |_/_/\_\_|   |_|  \___/_/\_\\__, |_|  |_|\__,_|_| |_|\__,_|\__, |\___|_|   
       |___/                                  |___/                          |___/           
Traceback (most recent call last):
  File "/usr/bin/certbot", line 5, in <module>
    from certbot.main import main
  File "/opt/certbot/lib/python3.7/site-packages/certbot/main.py", line 6, in <module>
    from certbot._internal import main as internal_main
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/main.py", line 28, in <module>
    from certbot import crypto_util
  File "/opt/certbot/lib/python3.7/site-packages/certbot/crypto_util.py", line 42, in <module>
    from certbot import interfaces
  File "/opt/certbot/lib/python3.7/site-packages/certbot/interfaces.py", line 21, in <module>
    from acme.client import ClientBase
ImportError: cannot import name 'ClientBase' from 'acme.client' (/opt/certbot/lib/python3.7/site-packages/acme/client.py)
Version 2.10.3 (f39e527) 2023-06-01 18:12:13 UTC, OpenResty 1.21.4.1, debian 10 (buster), Certbot 
Base: debian:buster-slim, linux/amd64
Certbot: jc21/nginx-full:latest, linux/amd64
Node: jc21/nginx-full:certbot, linux/amd64

[root@docker-225315d7801d:/app]#
jeffshead commented 1 year ago

I tried a couple more things.

[root@docker-225315d7801d:/app]# /opt/certbot/bin/pip uninstall certbot
Found existing installation: certbot 1.32.0
Uninstalling certbot-1.32.0:
  Would remove:
    /opt/certbot/bin/certbot
    /opt/certbot/lib/python3.7/site-packages/certbot-1.32.0.dist-info/*
    /opt/certbot/lib/python3.7/site-packages/certbot/*
Proceed (Y/n)? y
  Successfully uninstalled certbot-1.32.0
[root@docker-225315d7801d:/app]# /opt/certbot/bin/pip install certbot==2.5.0
Collecting certbot==2.5.0
  Downloading certbot-2.5.0-py3-none-any.whl (398 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 398.4/398.4 kB 24.1 MB/s eta 0:00:00
Requirement already satisfied: acme>=2.5.0 in /opt/certbot/lib/python3.7/site-packages (from certbot==2.5.0) (2.5.0)
Requirement already satisfied: ConfigArgParse>=0.9.3 in /opt/certbot/lib/python3.7/site-packages (from certbot==2.5.0) (1.5.3)
Requirement already satisfied: configobj>=5.0.6 in /opt/certbot/lib/python3.7/site-packages (from certbot==2.5.0) (5.0.8)
Requirement already satisfied: cryptography>=2.5.0 in /opt/certbot/lib/python3.7/site-packages (from certbot==2.5.0) (2.8)
Requirement already satisfied: distro>=1.0.1 in /opt/certbot/lib/python3.7/site-packages (from certbot==2.5.0) (1.8.0)
Requirement already satisfied: josepy>=1.13.0 in /opt/certbot/lib/python3.7/site-packages (from certbot==2.5.0) (1.13.0)
Requirement already satisfied: parsedatetime>=2.4 in /opt/certbot/lib/python3.7/site-packages (from certbot==2.5.0) (2.6)
Requirement already satisfied: pyrfc3339 in /opt/certbot/lib/python3.7/site-packages (from certbot==2.5.0) (1.1)
Requirement already satisfied: pytz>=2019.3 in /opt/certbot/lib/python3.7/site-packages (from certbot==2.5.0) (2023.3)
Requirement already satisfied: setuptools>=41.6.0 in /opt/certbot/lib/python3.7/site-packages (from certbot==2.5.0) (67.8.0)
Requirement already satisfied: PyOpenSSL!=23.1.0,>=17.5.0 in /opt/certbot/lib/python3.7/site-packages (from acme>=2.5.0->certbot==2.5.0) (19.1.0)
Requirement already satisfied: requests>=2.20.0 in /opt/certbot/lib/python3.7/site-packages (from acme>=2.5.0->certbot==2.5.0) (2.31.0)
Requirement already satisfied: six in /opt/certbot/lib/python3.7/site-packages (from configobj>=5.0.6->certbot==2.5.0) (1.16.0)
Requirement already satisfied: cffi!=1.11.3,>=1.8 in /opt/certbot/lib/python3.7/site-packages (from cryptography>=2.5.0->certbot==2.5.0) (1.15.1)
Requirement already satisfied: pycparser in /opt/certbot/lib/python3.7/site-packages (from cffi!=1.11.3,>=1.8->cryptography>=2.5.0->certbot==2.5.0) (2.21)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/certbot/lib/python3.7/site-packages (from requests>=2.20.0->acme>=2.5.0->certbot==2.5.0) (3.1.0)
Requirement already satisfied: idna<4,>=2.5 in /opt/certbot/lib/python3.7/site-packages (from requests>=2.20.0->acme>=2.5.0->certbot==2.5.0) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/certbot/lib/python3.7/site-packages (from requests>=2.20.0->acme>=2.5.0->certbot==2.5.0) (2.0.2)
Requirement already satisfied: certifi>=2017.4.17 in /opt/certbot/lib/python3.7/site-packages (from requests>=2.20.0->acme>=2.5.0->certbot==2.5.0) (2023.5.7)
Installing collected packages: certbot
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
certbot-dns-cloudns 0.4.0 requires certbot<2.0.0,>=1.12.0, but you have certbot 2.5.0 which is incompatible.
Successfully installed certbot-2.5.0
[root@docker-225315d7801d:/app]#

Next, I used the info from here (https://github.com/inventage/certbot-dns-cloudns/issues/2):

apt install nano
nano /opt/certbot/lib/python3.7/site-packages/certbot_dns_cloudns-0.4.0.dist-info/METADATA

Changed Requires-Dist: certbot (>=1.12.0 ,<2.0.0 ) to Requires-Dist: certbot (>=1.12.0) and changed the zope reference to Requires-Dist: zope.interface (>=5.1.0)

Still fails. I got the following error:

Error: Command failed: certbot certonly --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --cert-name "npm-25" --agree-tos --email "example@example.com" --domains "*.example.com,example.com" --authenticator dns-cloudns --dns-cloudns-credentials "/etc/letsencrypt/credentials/credentials-25"
Saving debug log to /tmp/letsencrypt-log/letsencrypt.log
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /tmp/letsencrypt-log/letsencrypt.log or re-run Certbot with -v for more details.

    at ChildProcess.exithandler (node:child_process:402:12)
    at ChildProcess.emit (node:events:513:28)
    at maybeClose (node:internal/child_process:1100:16)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:304:5)

I'm sure the user id and password are correct.

UPDATE: It seems you cannot use special characters in the ClouDNS password, in the NPM DNS challange input box. Is this true? Can you use quotes?

For example:

dns_cloudns_auth_password="%passw#ord1"

I'm afraid I will reach my limit on cert requests so I haven't tried. I removed the special characters from the password and the cert was created without any errors being displayed.

UPDATE 2: You can wrap your password in quotes and use special characters in your password.

jeffshead commented 1 year ago

Bottom line -- I encountered the same exact error that @phillipmcmahon posted in his first post. Below are the steps that I had to perform to get things working:

  1. Stop the NPM container
  2. Run docker system prune -a to remove the NPM image.
  3. Pull a specific PR image of NPM: docker pull jc21/nginx-proxy-manager:github-pr-2971

Attach and use a console on the NPM container for the remaining steps:

  1. Check if the certbot and acme versions are mismatched: /opt/certbot/bin/pip show acme certbot
    • Output I got showed acme 2.5.0 and certbot 1.32.0 so I had to reinstall certbot.
  2. Uninstall certbot: /opt/certbot/bin/pip uninstall certbot
  3. Install certbot 2.5.0: /opt/certbot/bin/pip install certbot==2.5.0
  4. Edit requirements:
    • Install nano: apt install nano
    • Edit file: nano /opt/certbot/lib/python3.7/site-packages/certbot_dns_cloudns-0.4.0.dist-info/METADATA
      • Change Requires-Dist: certbot (>=1.12.0 ,<2.0.0 ) to Requires-Dist: certbot (>=1.12.0) and change the zope reference to Requires-Dist: zope.interface (>=5.1.0)

References: #2971 #2554 #2

jeffshead commented 1 year ago

I still have no idea how you have that old version of certbot in this container.

@wolviex - This has happened two me on two separate servers with totally different setups. I just did a fresh install a few hours ago and the first thing that I did was check for conflicting versions using /opt/certbot/bin/pip show acme certbot. It showed both were 2.5.0. However, I still encountered the error I listed above. Since #2971, alone, did not resolve this issue, I had to perform the operation documented in #2. In order to perform that operation, I had to install nano but apt install nano fails until you run apt update and then run apt install nano. I did NOT run apt upgrade. I tried to issue a cert a few times but it failed so I rebooted the server. I got the same symptoms of having mismatched certbot and acme versions so I confirmed that I did, in fact, have acme 2.5.0 and certbot 1.32.0. Not sure when that happened because I never ran apt upgrade. Running /opt/certbot/bin/pip install certbot==2.5.0 does not work because it returns "Requirement already satisfied". You have to uninstall certbot first and then run /opt/certbot/bin/pip install certbot==2.5.0.

wolviex commented 1 year ago

@jeffshead I think you might be facing two issues - the first that is resolved with PR #2971 and lines up with the error posted by OP. Your second error is a module version mismatch, likely because the plugin NPM uses for your DNS provider specifies a downgraded certbot, but that version of certbot doesn't indicate that acme needs to be the same version. In retrospect the correct resolution was likely downgrading acme, which would avoid the need to do any of the other things you've had to do. The less we can touch the better. I would suggest a clean install of jc21/nginx-proxy-manager:github-pr-2971 and downgrade acme to match certbot:

/opt/certbot/bin/pip install acme==1.32.0
jeffshead commented 1 year ago

@wolviex - Great advice and it worked. Thanks again!

wolviex commented 1 year ago

Appears to be resolved by PR #2921

0xD4 commented 1 year ago

I had to do both PR #2921 and the manual acme downgrade to 1.32.0. Pull request alone doesn't seem to solve the problem for me. Looks to me as ClouDNS needed the downgrade as well.

jdelgadocr commented 11 months ago

I was getting the "import ClientBase" error as well with namecheap DNS. Strangely it worked 2 hours earlier with namecheap and another provider, several domains. Fresh, latest NPM install. I got it fixed by running (in a container-attached console): pip install certbot pip install acme pip install certbot-dns-namecheap

ukewea commented 10 months ago

I was getting the "import ClientBase" error as well with namecheap DNS. Strangely it worked 2 hours earlier with namecheap and another provider, several domains. Fresh, latest NPM install. I got it fixed by running (in a container-attached console): pip install certbot pip install acme pip install certbot-dns-namecheap

Thanks for the workaround, I'm using GoDaddy for my DNS, so I installed certbot-dns-godaddy instead of certbot-dns-namecheap.

jerimiah797 commented 10 months ago

I'm running on ARM (rpi4) and trying to install acme or certbot always fails like this: 

Building wheel for cffi (setup.py): finished with status 'error'

c/_cffi_backend.c:2:10: fatal error: Python.h: No such file or directory
             #include <Python.h>
                      ^~~~~~~~~~
            compilation terminated.
            error: command 'arm-linux-gnueabihf-gcc' failed with exit status 1

Perhaps this is another problem with the python venv? how do I activate the venv while working in the container?

wolviex commented 10 months ago

That's a platform/os error, you need to install the headers for python first. Nothing to do with NPM afaik.

sudo apt-get install python-dev

CFFI is an external package providing a C Foreign Function Interface for Python

jerimiah797 commented 9 months ago

I ended up having to install several dependencies to be able to compile/install certbot. Here's what I did:

Had to update certbot in the docker container:

apt update apt install python3-dev libffi-dev pkg-config libssl-dev apt version of rustc is too old (1.41.1, >1.56.0 is required) install rustc with curl https://sh.rustup.rs -sSf | sh source "$HOME/.cargo/env" to activate rustc PATH

Activate venv source /opt/certbot/bin/activate

pip install certbot pip install acme pip install certbot-dns-godaddy

dennisdenk commented 9 months ago

Is there any chance, that this issue will be fixed? I have the exact same problem using cloudns dns challenge with NPM.

phillipmcmahon commented 9 months ago

Is there any chance, that this issue will be fixed? I have the exact same problem using cloudns dns challenge with NPM.

Sadly not, it requires manual updates within the image.

There sadly seems to be a growing backlog of issues with NPM, if I could code I would offer to help but I am but a mere end user.

Looking for alternatives that are more currently maintained is on my todo list.

jeffshead commented 9 months ago

There sadly seems to be a growing backlog of issues with NPM...

Sad indeed. There's alternatives like HAProxy, Traefik and Swag but nothing, that I'm aware of, with a simple GUI like NPM. I LOVE NPM for it's ease of use.

wolviex commented 9 months ago

Is there any chance, that this issue will be fixed? I have the exact same problem using cloudns dns challenge with NPM.

@dennisdenk Which issue are you referring to?

This thread has been hijacked a couple times, the actual OP issue was fixed in v2.10.4

@phillipmcmahon Many of the "issues" plaguing this repo aren't NPM issues, but belong to all the various DNS plugin packages(over 50) that NPM uses and I wouldn't blame jc for being overwhelmed by them (1.1k issues at time of writing).

I personally use a version with an automatic downgrade of acme to 1.32, but that only works for the specific plugin I use, certbot-dns-godaddy, and absolutely breaks other plugins. (I push this downgrade in my docker compose script)

It might be time to cut loose any plugins that aren't being maintained. I'll see if I can carve out some time to look at this, but it'll mean going through them and looking for dependency errors. There also needs to be a better way to identify when issues are plugin issues, not NPM issues. This probably means a team of mods for the repo to weed through the noise.

3165

jeffshead commented 9 months ago

I just checked two servers that were working back in June. No mods were made to either server. Now, I cannot add new certs. Same results with v2.10.3-pr-2971 and v2.10.4. I get the following error when I try to add a cert:

Error: Command failed: certbot certonly --config "/etc/letsencrypt.ini" --work-dir "/tmp/letsencrypt-lib" --logs-dir "/tmp/letsencrypt-log" --cert-name "npm-15" --agree-tos --email "jeff@example.com" --domains "*.example.com,*.example2.com," --authenticator dns-cloudns --dns-cloudns-credentials "/etc/letsencrypt/credentials/credentials-15"
Saving debug log to /tmp/letsencrypt-log/letsencrypt.log
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /tmp/letsencrypt-log/letsencrypt.log or re-run Certbot with -v for more details.

    at ChildProcess.exithandler (node:child_process:402:12)
    at ChildProcess.emit (node:events:513:28)
    at maybeClose (node:internal/child_process:1100:16)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:304:5)

Below is the log:

[root@docker-6fga4ahg21:/app]# cat /tmp/letsencrypt-log/letsencrypt.log
2023-11-29 21:37:07,968:DEBUG:certbot._internal.main:certbot version: 1.32.0
2023-11-29 21:37:07,968:DEBUG:certbot._internal.main:Location of certbot entry point: /usr/bin/certbot
2023-11-29 21:37:07,968:DEBUG:certbot._internal.main:Arguments: ['--config', '/etc/letsencrypt.ini', '--work-dir', '/tmp/letsencrypt-lib', '--logs-dir', '/tmp/letsencrypt-log', '--cert-name', 'npm-16', '--agree-tos', '--email', 'jeff@example.com', '--domains', 'example.com', '--authenticator', 'dns-cloudns', '--dns-cloudns-credentials', '/etc/letsencrypt/credentials/credentials-16']
2023-11-29 21:37:07,969:DEBUG:certbot._internal.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#certbot-dns-cloudns:dns-cloudns,PluginEntryPoint#dns-cloudns,PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2023-11-29 21:37:08,026:DEBUG:certbot._internal.log:Root logging level set at 30
2023-11-29 21:37:08,027:DEBUG:certbot._internal.plugins.selection:Requested authenticator dns-cloudns and installer None
2023-11-29 21:37:08,033:DEBUG:certbot._internal.plugins.selection:Single candidate plugin: * dns-cloudns
Description: Obtain certificates using a DNS TXT record (if you are using ClouDNS for DNS).
Interfaces: Authenticator, Plugin
Entry point: dns-cloudns = certbot_dns_cloudns._internal.authenticator:Authenticator
Initialized: <certbot_dns_cloudns._internal.authenticator.Authenticator object at 0xhh72da05908>
Prep: True
2023-11-29 21:37:08,033:DEBUG:certbot._internal.plugins.selection:Selected authenticator <certbot_dns_cloudns._internal.authenticator.Authenticator object at 0xhfh572da05908> and installer None
2023-11-29 21:37:08,033:INFO:certbot._internal.plugins.selection:Plugins selected: Authenticator dns-cloudns, Installer None
2023-11-29 21:37:08,057:DEBUG:certbot._internal.main:Picked account: <Account(RegistrationResource(body=Registration(key=None, contact=(), agreement=None, status=None, terms_of_service_agreed=None, only_return_existing=None, external_account_binding=None), uri='https://acme-v02.api.letsencrypt.org/acme/acct/641771426', new_authzr_uri=None, terms_of_service=None), dhdhdhdhdh, Meta(creation_dt=datetime.datetime(2022, 7, 22, 1, 22, 46, tzinfo=<UTC>), creation_host='4afgh3ec9d79', register_to_eff=None))>
2023-11-29 21:37:08,058:DEBUG:acme.client:Sending GET request to https://acme-v02.api.letsencrypt.org/directory.
2023-11-29 21:37:08,060:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): acme-v02.api.letsencrypt.org:443
2023-11-29 21:37:08,264:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "GET /directory HTTP/1.1" 200 752
2023-11-29 21:37:08,265:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Thu, 30 Nov 2023 02:37:08 GMT
Content-Type: application/json
Content-Length: 752
Connection: keep-alive
Cache-Control: public, max-age=0, no-cache
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "UP4wg00pP7E": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417",
  "keyChange": "https://acme-v02.api.letsencrypt.org/acme/key-change",
  "meta": {
    "caaIdentities": [
      "letsencrypt.org"
    ],
    "termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.3-September-21-2022.pdf",
    "website": "https://letsencrypt.org"
  },
  "newAccount": "https://acme-v02.api.letsencrypt.org/acme/new-acct",
  "newNonce": "https://acme-v02.api.letsencrypt.org/acme/new-nonce",
  "newOrder": "https://acme-v02.api.letsencrypt.org/acme/new-order",
  "renewalInfo": "https://acme-v02.api.letsencrypt.org/draft-ietf-acme-ari-01/renewalInfo/",
  "revokeCert": "https://acme-v02.api.letsencrypt.org/acme/revoke-cert"
}
2023-11-29 21:37:08,265:DEBUG:certbot._internal.display.obj:Notifying user: Requesting a certificate for example.com
2023-11-29 21:37:08,268:DEBUG:certbot.crypto_util:Generating ECDSA key (2048 bits): /etc/letsencrypt/keys/0020_key-certbot.pem
2023-11-29 21:37:08,270:DEBUG:certbot.crypto_util:Creating CSR: /etc/letsencrypt/csr/0020_csr-certbot.pem
2023-11-29 21:37:08,271:DEBUG:acme.client:Requesting fresh nonce
2023-11-29 21:37:08,271:DEBUG:acme.client:Sending HEAD request to https://acme-v02.api.letsencrypt.org/acme/new-nonce.
2023-11-29 21:37:08,322:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "HEAD /acme/new-nonce HTTP/1.1" 200 0
2023-11-29 21:37:08,322:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Thu, 30 Nov 2023 02:37:08 GMT
Connection: keep-alive
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: dhWZ6XrY2n8FYdhd-sKPhpMdhdhzQ_m7qkI
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

2023-11-29 21:37:08,322:DEBUG:acme.client:Storing nonce: dfhdhrY2n8FYK46EffCNdfhzKNGQdfh7qkI
2023-11-29 21:37:08,322:DEBUG:acme.client:JWS payload:
b'{\n  "identifiers": [\n    {\n      "type": "dns",\n      "value": "example.com"\n    }\n  ]\n}'
2023-11-29 21:37:08,324:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/new-order:
{
  "protected": "dfgdgdfgHNlgdg9hY21lL2FjY3QvNjQxNzcxNDI2IiwgIm5vbmNlIjzSEPLfZsY6w",
  "payload": "ewogICJpZGdfgHVlIjogInNlcnZ1dGssggXQp9"
}
2023-11-29 21:37:08,679:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/new-order HTTP/1.1" 201 338
2023-11-29 21:37:08,680:DEBUG:acme.client:Received response:
HTTP 201
Server: nginx
Date: Thu, 30 Nov 2023 02:37:08 GMT
Content-Type: application/json
Content-Length: 338
Connection: keep-alive
Boulder-Requester: 641771426
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Location: https://acme-v02.api.letsencrypt.org/acme/order/641771426/225786386786
Replay-Nonce: nbdfhNxjLyaadfhedw
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "status": "pending",
  "expires": "2023-12-07T02:37:08Z",
  "identifiers": [
    {
      "type": "dns",
      "value": "example.com"
    }
  ],
  "authorizations": [
    "https://acme-v02.api.letsencrypt.org/acme/authz-v3/288875058846"
  ],
  "finalize": "https://acme-v02.api.letsencrypt.org/acme/finalize/641771426/225786386786"
}
2023-11-29 21:37:08,680:DEBUG:acme.client:Storing nonce: nbWdfhzjKydgsgw
2023-11-29 21:37:08,681:DEBUG:acme.client:JWS payload:
b''
2023-11-29 21:37:08,686:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/authz-v3/288875058846:
{
  "protected": "edfgImh0dHBzOi8vYWNtZS12MDIuYXBpLmxldHNlbmNXV0aHotdjMvMjg4ODc1MDU4ODQ2In0",
  "signature": "dfhdfhdfhNI6KGRA6AKr31wqIHzsVvOhARcSqqH5XmO75-ogkJGKIHxFWLhA",
  "payload": ""
}
2023-11-29 21:37:08,741:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/authz-v3/288875058846 HTTP/1.1" 200 797
2023-11-29 21:37:08,742:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Thu, 30 Nov 2023 02:37:08 GMT
Content-Type: application/json
Content-Length: 797
Connection: keep-alive
Boulder-Requester: 641771426
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: v0bffdhnJoAidfhR8g2Kdfh28
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "identifier": {
    "type": "dns",
    "value": "example.com"
  },
  "status": "pending",
  "expires": "2023-12-07T02:37:08Z",
  "challenges": [
    {
      "type": "http-01",
      "status": "pending",
      "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/288875058846/FdsoXQ",
      "token": "2dfhT0eFRDnuduidfhnaN4DeU"
    },
    {
      "type": "dns-01",
      "status": "pending",
      "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/288875058846/Ygc2LA",
      "token": "dhRDnuduii0dfh4DeU"
    },
    {
      "type": "tls-alpn-01",
      "status": "pending",
      "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/288875058846/FMStIw",
      "token": "dhfduiidh4DeU"
    }
  ]
}
2023-11-29 21:37:08,743:DEBUG:acme.client:Storing nonce: dfhwldyDdfhr28
2023-11-29 21:37:08,744:INFO:certbot._internal.auth_handler:Performing the following challenges:
2023-11-29 21:37:08,745:INFO:certbot._internal.auth_handler:dns-01 challenge for example.com
2023-11-29 21:37:08,749:DEBUG:certbot_dns_cloudns._internal.resolve:Using nameserver 127.0.0.11
2023-11-29 21:37:08,756:DEBUG:certbot_dns_cloudns._internal.resolve:No CNAME record found for _acme-challenge.example.com.
2023-11-29 21:37:08,757:DEBUG:certbot_dns_cloudns._internal.client:Looking up zone example.com.
2023-11-29 21:37:08,761:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.cloudns.net:443
2023-11-29 21:37:09,128:DEBUG:urllib3.connectionpool:https://api.cloudns.net:443 "GET /dns/get-zone-info.json?auth-id=1fgh86&auth-password=###&domain-name=example.com HTTP/1.1" 200 None
2023-11-29 21:37:09,135:DEBUG:certbot_dns_cloudns._internal.client:ClouDNS API response: {'status_code': 200, 'success': True, 'payload': {'name': 'example.com', 'type': 'master', 'zone': 'domain', 'status': '1'}}
2023-11-29 21:37:09,135:DEBUG:certbot_dns_cloudns._internal.client:Found zone example.com for _acme-challenge.example.com.
2023-11-29 21:37:09,135:DEBUG:certbot_dns_cloudns._internal.client:Attempting to add record _acme-challenge to zone example.com (to validate example.com).
2023-11-29 21:37:09,139:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.cloudns.net:443
2023-11-29 21:37:09,481:DEBUG:urllib3.connectionpool:https://api.cloudns.net:443 "POST /dns/add-record.json?auth-id=1fgh86&auth-password=###&domain-name=example.com&host=_acme-challenge&ttl=60&record=xdfbovcZLik5lhWdfgh0&record-type=TXT HTTP/1.1" 200 None
2023-11-29 21:37:09,487:DEBUG:certbot_dns_cloudns._internal.client:ClouDNS API response: {'status_code': 200, 'success': True, 'payload': {'status': 'Success', 'status_description': 'The record was added successfully.', 'data': {'id': 386522144}}}
2023-11-29 21:37:09,490:DEBUG:certbot._internal.display.obj:Notifying user: Waiting 60 seconds for DNS changes to propagate
2023-11-29 21:38:09,552:DEBUG:acme.client:JWS payload:
b'{}'
2023-11-29 21:38:09,557:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/chall-v3/288875058846/Ygc2LA:
{
  "protected": "dfhgmNlIjogInYwYk1vd2Zpdfg1lLXYwMi5hcGku9",
  "signature": "dfhdfhBDO5lbLVgotyugqdhfdfhdh9QYuSdfhdfhw",
  "payload": "e30"
}
2023-11-29 21:38:09,618:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/chall-v3/2df75058846/Ysg2LA HTTP/1.1" 200 186
2023-11-29 21:38:09,619:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Thu, 30 Nov 2023 02:38:09 GMT
Content-Type: application/json
Content-Length: 186
Connection: keep-alive
Boulder-Requester: 641771426
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index", <https://acme-v02.api.letsencrypt.org/acme/authz-v3/288875058846>;rel="up"
Location: https://acme-v02.api.letsencrypt.org/acme/chall-v3/288875058846/Ygc2LA
Replay-Nonce: dfhyj4dfh
Strict-Transport-Security: max-age=604800

{
  "type": "dns-01",
  "status": "pending",
  "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/288875058846/Ygc2LA",
  "token": "dfh3naN4DeU"
}
2023-11-29 21:38:09,619:DEBUG:acme.client:Storing nonce: dfh-t5FHoD_xsSN-4k
2023-11-29 21:38:09,620:INFO:certbot._internal.auth_handler:Waiting for verification...
2023-11-29 21:38:10,622:DEBUG:acme.client:JWS payload:
b''
2023-11-29 21:38:10,627:DEBUG:acme.client:Sending POST request to https://acme-v02.api.letsencrypt.org/acme/authz-v3/288875058846:
{
  "protected": "df1lLdfhNlIjogIm5iV0",
  "signature": "dsfsgsgsiXCiLUYdPGAhpnFDvkLKAfgsdfgFQLyuzH7ug",
  "payload": ""
}
2023-11-29 21:38:10,694:DEBUG:urllib3.connectionpool:https://acme-v02.api.letsencrypt.org:443 "POST /acme/authz-v3/288875058846 HTTP/1.1" 200 649
2023-11-29 21:38:10,695:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Date: Thu, 30 Nov 2023 02:38:10 GMT
Content-Type: application/json
Content-Length: 649
Connection: keep-alive
Boulder-Requester: 641771426
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: fgjCuI5Sf2fgw8fgjaVg
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "identifier": {
    "type": "dns",
    "value": "example.com"
  },
  "status": "invalid",
  "expires": "2023-12-07T02:37:08Z",
  "challenges": [
    {
      "type": "dns-01",
      "status": "invalid",
      "error": {
        "type": "urn:ietf:params:acme:error:dns",
        "detail": "DNS problem: NXDOMAIN looking up TXT for _acme-challenge.example.com - check that a DNS record exists for this domain",
        "status": 400
      },
      "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/288875058846/Ygc2LA",
      "token": "fgduii0HYfgh4DeU",
      "validated": "2023-11-30T02:38:09Z"
    }
  ]
}
2023-11-29 21:38:10,696:DEBUG:acme.client:Storing nonce: fgSf2EMfgjh
2023-11-29 21:38:10,697:INFO:certbot._internal.auth_handler:Challenge failed for domain example.com
2023-11-29 21:38:10,697:INFO:certbot._internal.auth_handler:dns-01 challenge for example.com
2023-11-29 21:38:10,698:DEBUG:certbot._internal.display.obj:Notifying user: 
Certbot failed to authenticate some domains (authenticator: dns-cloudns). The Certificate Authority reported these problems:
  Domain: example.com
  Type:   dns
  Detail: DNS problem: NXDOMAIN looking up TXT for _acme-challenge.example.com - check that a DNS record exists for this domain

Hint: The Certificate Authority failed to verify the DNS TXT records created by --dns-cloudns. Ensure the above domains are hosted by this DNS provider, or try increasing --dns-cloudns-propagation-seconds (currently 60 seconds).

2023-11-29 21:38:10,699:DEBUG:certbot._internal.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/auth_handler.py", line 106, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, best_effort)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/auth_handler.py", line 206, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.

2023-11-29 21:38:10,700:DEBUG:certbot._internal.error_handler:Calling registered functions
2023-11-29 21:38:10,700:INFO:certbot._internal.auth_handler:Cleaning up challenges
2023-11-29 21:38:10,707:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.cloudns.net:443
2023-11-29 21:38:11,083:DEBUG:urllib3.connectionpool:https://api.cloudns.net:443 "GET /dns/records.json?auth-id=1gh86&auth-password=###&domain-name=example.com&host=_acme-challenge&type=TXT HTTP/1.1" 200 None
2023-11-29 21:38:11,088:DEBUG:certbot_dns_cloudns._internal.client:ClouDNS API response: {'status_code': 200, 'success': True, 'payload': {'3fhghfgh22h44': {'id': '38gffg2fg144', 'type': 'TXT', 'host': '_acme-challenge', 'record': 'fgjf7XhqbovcZLifgjRvUh0', 'failover': '0', 'ttl': '60', 'status': 1}}}
2023-11-29 21:38:11,092:DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.cloudns.net:443
2023-11-29 21:38:11,446:DEBUG:urllib3.connectionpool:https://api.cloudns.net:443 "POST /dns/delete-record.json?auth-id=1hf86&auth-password=###&domain-name=example.com&record-id=386522144 HTTP/1.1" 200 None
2023-11-29 21:38:11,451:DEBUG:certbot_dns_cloudns._internal.client:ClouDNS API response: {'status_code': 200, 'success': True, 'payload': {'status': 'Success', 'status_description': 'The record was deleted successfully.'}}
2023-11-29 21:38:11,451:DEBUG:certbot_dns_cloudns._internal.client:Successfully deleted TXT record.
2023-11-29 21:38:11,452:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/bin/certbot", line 8, in <module>
    sys.exit(main())
  File "/opt/certbot/lib/python3.7/site-packages/certbot/main.py", line 19, in main
    return internal_main.main(cli_args)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/main.py", line 1744, in main
    return config.func(config, plugins)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/main.py", line 1591, in certonly
    lineage = _get_and_save_cert(le_client, config, domains, certname, lineage)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/main.py", line 141, in _get_and_save_cert
    lineage = le_client.obtain_and_enroll_certificate(domains, certname)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/client.py", line 530, in obtain_and_enroll_certificate
    cert, chain, key, _ = self.obtain_certificate(domains)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/client.py", line 442, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/client.py", line 510, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, self.config, best_effort)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/auth_handler.py", line 106, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, best_effort)
  File "/opt/certbot/lib/python3.7/site-packages/certbot/_internal/auth_handler.py", line 206, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.
2023-11-29 21:38:11,455:ERROR:certbot._internal.log:Some challenges have failed.
phillipmcmahon commented 9 months ago

I just checked two servers that were working back in June. No mods were made to either server. Now, I cannot add new certs. Same results with v2.10.3-pr-2971 and v2.10.4. I get the following error when I try to add a cert:

I find you need to leave a longer delay for Cloud DNS otherwise it often fails. Test it out with acme.sh directly to see what timeout works consistently for you. I ended up having to go with 180 seconds.

jeffshead commented 9 months ago

I find you need to leave a longer delay for Cloud DNS otherwise it often fails.

I've tried 120, 200, 300. I get gateway timeout errors with them all.

image

phillipmcmahon commented 9 months ago

Try it directly with the acme.sh client (or some suitable alternative) to check the basic config and auth info is correct and you're getting back a fully signed valid cert.

If that works, then you should create an new NPM issue to try and get help for this specific problem.

dennisdenk commented 9 months ago

Is there any chance, that this issue will be fixed? I have the exact same problem using cloudns dns challenge with NPM.

@dennisdenk Which issue are you referring to?

This thread has been hijacked a couple times, the actual OP issue was fixed in v2.10.4

@phillipmcmahon Many of the "issues" plaguing this repo aren't NPM issues, but belong to all the various DNS plugin packages(over 50) that NPM uses and I wouldn't blame jc for being overwhelmed by them (1.1k issues at time of writing).

I personally use a version with an automatic downgrade of acme to 1.32, but that only works for the specific plugin I use, certbot-dns-godaddy, and absolutely breaks other plugins. (I push this downgrade in my docker compose script)

It might be time to cut loose any plugins that aren't being maintained. I'll see if I can carve out some time to look at this, but it'll mean going through them and looking for dependency errors. There also needs to be a better way to identify when issues are plugin issues, not NPM issues. This probably means a team of mods for the repo to weed through the noise. #3165

I have the feeling, that dns plugins in general are not that well maintained for certbot. Did you ever consider switching to acme.sh for NPM?

iresprite commented 8 months ago

I've tried the solutions suggested in this issue as well as the one outlined in inventage/certbot-dns-cloudns#2; I updated certbot-dns-godaddy, certbot, and acme to 2.6.0. I confirmed that /opt/certbot/lib/python3.7/site-packages/certbot_dns_godaddy-2.6.0.dist-info/METADATA had these:

Requires-Dist: acme (==2.6.0) Requires-Dist: certbot (==2.6.0)

However, I went back to add an SSL cert and got the same error. Back in the console, I ran /opt/certbot/bin/pip show acme certbot certbot-dns-godaddy, and somehow it keeps reinstalling certbot at 1.3.0 and certbot-dns-godaddy at 0.2.4. Has anyone run into this and found a solution?

I'll add that I did also try updating global/certbot-dns-plugins.js:270 to 2.6.0 to see if that made a difference.

wolviex commented 8 months ago

However, I went back to add an SSL cert and got the same error. What's the error your getting?

NPM uses a number of third party plugins and each has it's own opinion on which acme/certbot versions are appropriate. NPM hasn't upgraded the certbot-dns-godaddy specifically because pushing to 2.6.0 is not validated on many of them. I've found downgrading to 1.32.0 has been the most effective.

iresprite commented 8 months ago

Yep, good point. I downgraded instead and am no longer getting that error. Instead I'm getting:

certbot.errors.PluginError: Error determining zone identifier for ********: 401 Client Error: Unauthorized for url: https://api.godaddy.com/v1/domains/********.

So, progress, I guess!

wolviex commented 8 months ago

So, progress, I guess!

Progress indeed! With 401 being an authorization error, I'd be looking at the godaddy credentials next

iresprite commented 8 months ago

Yep. I tested with curl and it works fine. The issue appears to be that python always puts a space between the key and value of a header, whereas godaddy expects the key and secret to be sent without one (i.e., "key:secret"). Not sure what I can do about that.

wolviex commented 8 months ago

python always puts a space

I'm not sure what to make of that. This is working perfectly fine in my installation with godaddy DNS, and I think in most cases whitespace is ignored. In that case it would be a certbot and certbot-dns-godaddy issue, not an NPM one. NPM uses a credential file, which as is typical has multiple lines and lots of whitespace. This credentials file is passed to certbot in a command line argument certbot --certbot-dns-godaddy-credentials credentialsLocation <+more args>

I'd still be looking at the credentials to make sure they're right. You can confirm what NPM recorded in the credentials file at /etc/letsencrypt/credentials inside the container

iresprite commented 8 months ago

In that case it would be a certbot and certbot-dns-godaddy issue, not an NPM one.

Yep. I tried to reproduce the issue using Digital Ocean and that worked fine. I expect the issue is in certbot-dns-godaddy.

github-actions[bot] commented 1 month ago

Issue is now considered stale. If you want to keep it open, please comment :+1: