search

tteck / Proxmox

Proxmox VE Helper-Scripts
https://Helper-Scripts.com
MIT License
14.38k stars 2.36k forks source link

NGINX Proxy Manager - old Certbot version and problem with DirectAdmin wildcards domains. #3841

Closed MaverickPOL closed 1 month ago

MaverickPOL commented 1 month ago

Please verify that you have read and understood the guidelines.

yes

A clear and concise description of the issue.

Hi, i try to install NPM from install/nginxproxymanager-install.sh and in this image after instalation certbot have version 2.1.0 If i try to make certificate for some.domain.xxx everything is ok. But when i try to make cert for *.domain.xxx there are show some errors (i use DirectAdmin DNS Challenge)

Since now i use NPM on docker, and there everything works fine. I check versions, and in my docker container (jc21/nginx-proxy-manager) there is certbot in 2.11.0 version. I see that on github also newest version is 2.11.0

I know that this probably if debian fault, that they have 2.1.0 version, but maby there is some way to update certbot always to newest version use github page? So this script will update certbot to most recent version no matter what is in sources of debian? I check Debian 11 and Debian 12 both have 2.1.0 version instaled after runing this istalation script.

What settings are you currently utilizing?

Default Settings

Which Linux distribution are you employing?

Debian 12

If relevant, including screenshots or a code block can be helpful in clarifying the issue.

No response

Please provide detailed steps to reproduce the issue.

  1. Use script to install NPM (Debian 11 or Debian 12 version)
  2. Try to apt update apt upgrade
  3. certbot is in 2.1.0 version
  4. generating certificate with wildcard (*.somedomain.xxx) dont work (with DirectAdmin DNS Challange)
  5. Manually copy new version of bin certbot (2.11.0)
  6. Everything works fine.
MaverickPOL commented 1 month ago

Unfortunatelly after use this new version of the script certbot is still in 2.1.0 version and dont work.

tteck commented 1 month ago

Should be 2.11.0 https://pypi.org/project/certbot/

tteck commented 1 month ago

Version 2.1.0 is from December 2022. You must mean version 2.10.0

MaverickPOL commented 1 month ago

No... 2.1.0 two, point one, point zero I use script from this page: https://tteck.github.io/Proxmox/#proxmox-ve-tools

tteck commented 1 month ago

How can this be?

MaverickPOL commented 1 month ago

I dont know...

And now i saw that file that i copy was not binary... this is file after NPM instalation:

root@nginxproxymanager:~# certbot --version
certbot 2.1.0
root@nginxproxymanager:~# cat /usr/bin/certbot 
#!/usr/bin/python3
# EASY-INSTALL-ENTRY-SCRIPT: 'certbot==2.1.0','console_scripts','certbot'
import re
import sys

# for compatibility with easy_install; see #2198
__requires__ = 'certbot==2.1.0'

try:
    from importlib.metadata import distribution
except ImportError:
    try:
        from importlib_metadata import distribution
    except ImportError:
        from pkg_resources import load_entry_point

def importlib_load_entry_point(spec, group, name):
    dist_name, _, _ = spec.partition('==')
    matches = (
        entry_point
        for entry_point in distribution(dist_name).entry_points
        if entry_point.group == group and entry_point.name == name
    )
    return next(matches).load()

globals().setdefault('load_entry_point', importlib_load_entry_point)

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(load_entry_point('certbot==2.1.0', 'console_scripts', 'certbot')())

And this is file copy from other machine, which works fine:
    [root@docker-7561659817a8:/app]# cat /usr/bin/certbot
#!/opt/certbot/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from certbot.main import main
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(main())
tteck commented 1 month ago

Try

apt remove python3-certbot
pip3 install certbot
hash -r
certbot --version
tteck commented 1 month ago

Found the issue, it was python3-certbot-dns-cloudflare, it triggers the installation of python3-certbot as a dependency.

apt-cache depends python3-certbot-dns-cloudflare

output:

python3-certbot-dns-cloudflare
  Depends: certbot
  Depends: <python3-certbot-abi-2>
    python3-certbot
  Depends: python3-acme
  Depends: python3-certbot
  Depends: python3-cloudflare
  Depends: python3-pkg-resources
  Depends: <python3:any>
    python3
  Enhances: certbot

I removed python3-certbot-dns-cloudflare since pip3 install certbot certbot-dns-multi also contains Cloudflare.

root@nginxproxymanager:~# certbot --version
certbot 2.11.0
MaverickPOL commented 1 month ago

Now certbot --version show 2.11.0 but when i try to generate certificate i still get:

CommandError: usage: 
  certbot [SUBCOMMAND] [options] [-d DOMAIN] [-d DOMAIN] ...

Certbot can obtain and install HTTPS/TLS/SSL certificates.  By default,
it will attempt to use a webserver both for obtaining and installing the
certificate. 
certbot: error: unrecognized arguments: --directadmin-credentials /etc/letsencrypt/credentials/credentials-2

    at /app/lib/utils.js:16:13
    at ChildProcess.exithandler (node:child_process:410:5)
    at ChildProcess.emit (node:events:513:28)
    at maybeClose (node:internal/child_process:1100:16)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:304:5)
Maby frontend use something else?  So right now is even worse, becouse i have right version but it didn't work at all with wildcards.
tteck commented 1 month ago

certbot: error: unrecognized arguments: --directadmin-credentials /etc/letsencrypt/credentials/credentials-2

If you want to use the DirectAdmin DNS plugin, confirm that you have it installed https://pypi.org/project/certbot-dns-directadmin/

It is NOT included in certbot-dns-multi

MaverickPOL commented 1 month ago

But it works before (when i swap this /usr/bin/certbot file) and without any aditional installations... And yes, i want to generate certificate for *.domain.xyz so i need to check DNS Challange (and i use Direct Admin)

I will try again on clean install, give me 5 minutes.

MaverickPOL commented 1 month ago

No. Still not working. I make clean install NPM, then i type pip install certbot-dns-directadmin on console... and still i can't generate certificate for wildcard domain :(

No i can't also fix it by copy files from my docker NPM. It look like there was something in those packages that you removed and that was used by NPM

tteck commented 1 month ago

cerbot path changed

which certbot
MaverickPOL commented 1 month ago

clean install, without any changes

root@nginxproxymanager:~# certbot --version
certbot 2.11.0
root@nginxproxymanager:~# which certbot
/usr/local/bin/certbot

root@nginxproxymanager:~# ls -l /opt/certbot/bin/certbot 
lrwxrwxrwx 1 root root 16 Oct  5 02:42 /opt/certbot/bin/certbot -> /usr/bin/certbot
tteck commented 1 month ago

Looks correct

You can create a symbolic link

ln -s /usr/local/bin/certbot /usr/bin/certbot

I may need to add the symlink to the script EDIT: added symlink to the scipt

MaverickPOL commented 1 month ago

I run those commands on clean installation:

root@nginxproxymanager:~# ln -sf /usr/local/bin/certbot /opt/certbot/bin/certbot 
root@nginxproxymanager:~# ln -sf /usr/local/bin/certbot /usr/bin/certbot
root@nginxproxymanager:~# ls -l /opt/certbot/bin/certbot 
lrwxrwxrwx 1 root root 22 Oct  5 02:47 /opt/certbot/bin/certbot -> /usr/local/bin/certbot
root@nginxproxymanager:~# ls -l /usr/bin/certbot 
lrwxrwxrwx 1 root root 22 Oct  5 02:48 /usr/bin/certbot -> /usr/local/bin/certbot

And still not work. So i run pip install certbot-dns-directadmin

And now certbot --version show this (after certbot-dns-directadmin install)

root@nginxproxymanager:~# certbot --version
certbot 1.32.0
tteck commented 1 month ago

why ln -sf /usr/local/bin/certbot /opt/certbot/bin/certbot?

MaverickPOL commented 1 month ago

on my working docker this is used path. I try without it once again.

tteck commented 1 month ago

All that you need to do is:

Create a Nginx Proxy Manager LXC execute pip3 install certbot-dns-directadmin systemctl restart npm

Everything should work

MaverickPOL commented 1 month ago

No. This dont work.

Once again, clean instal from this script bash -c "$(wget -qLO - https://github.com/tteck/Proxmox/raw/main/ct/nginxproxymanager.sh)"

And here is my console:

root@npm-0304:~# certbot --version
certbot 2.11.0
root@npm-0304:~# pip3 install certbot-dns-directadmin
Collecting certbot-dns-directadmin
  Downloading certbot_dns_directadmin-1.0.6-py3-none-any.whl (10 kB)
Collecting acme<2.0.0,>=1.32.0
  Downloading acme-1.32.0-py3-none-any.whl (50 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 50.0/50.0 kB 1.4 MB/s eta 0:00:00
Collecting certbot<2.0.0,>=1.8.0
  Downloading certbot-1.32.0-py3-none-any.whl (273 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 273.0/273.0 kB 4.6 MB/s eta 0:00:00
Requirement already satisfied: cryptography>=2.5.0 in /usr/local/lib/python3.11/dist-packages (from acme<2.0.0,>=1.32.0->certbot-dns-directadmin) (43.0.1)
Requirement already satisfied: josepy>=1.13.0 in /usr/local/lib/python3.11/dist-packages (from acme<2.0.0,>=1.32.0->certbot-dns-directadmin) (1.14.0)
Requirement already satisfied: PyOpenSSL>=17.5.0 in /usr/local/lib/python3.11/dist-packages (from acme<2.0.0,>=1.32.0->certbot-dns-directadmin) (24.2.1)
Requirement already satisfied: pyrfc3339 in /usr/local/lib/python3.11/dist-packages (from acme<2.0.0,>=1.32.0->certbot-dns-directadmin) (1.1)
Requirement already satisfied: pytz>=2019.3 in /usr/local/lib/python3.11/dist-packages (from acme<2.0.0,>=1.32.0->certbot-dns-directadmin) (2024.2)
Requirement already satisfied: requests>=2.20.0 in /usr/lib/python3/dist-packages (from acme<2.0.0,>=1.32.0->certbot-dns-directadmin) (2.28.1)
Collecting requests-toolbelt>=0.3.0
  Downloading requests_toolbelt-1.0.0-py2.py3-none-any.whl (54 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.5/54.5 kB 11.5 MB/s eta 0:00:00
Requirement already satisfied: setuptools>=41.6.0 in /usr/lib/python3/dist-packages (from acme<2.0.0,>=1.32.0->certbot-dns-directadmin) (66.1.1)
Requirement already satisfied: ConfigArgParse>=0.9.3 in /usr/local/lib/python3.11/dist-packages (from certbot<2.0.0,>=1.8.0->certbot-dns-directadmin) (1.7)
Requirement already satisfied: configobj>=5.0.6 in /usr/local/lib/python3.11/dist-packages (from certbot<2.0.0,>=1.8.0->certbot-dns-directadmin) (5.0.9)
Requirement already satisfied: distro>=1.0.1 in /usr/local/lib/python3.11/dist-packages (from certbot<2.0.0,>=1.8.0->certbot-dns-directadmin) (1.9.0)
Requirement already satisfied: parsedatetime>=2.4 in /usr/local/lib/python3.11/dist-packages (from certbot<2.0.0,>=1.8.0->certbot-dns-directadmin) (2.6)
Collecting zope.component
  Downloading zope.component-6.0-py3-none-any.whl (68 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 68.8/68.8 kB 15.6 MB/s eta 0:00:00
Collecting zope.interface
  Downloading zope.interface-7.0.3-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (259 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 260.0/260.0 kB 22.5 MB/s eta 0:00:00
Requirement already satisfied: cffi>=1.12 in /usr/lib/python3/dist-packages (from cryptography>=2.5.0->acme<2.0.0,>=1.32.0->certbot-dns-directadmin) (1.15.1)
Collecting zope.event
  Downloading zope.event-5.0-py3-none-any.whl (6.8 kB)
Collecting zope.hookable>=4.2.0
  Downloading zope.hookable-7.0-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (24 kB)
Installing collected packages: zope.interface, zope.hookable, zope.event, requests-toolbelt, zope.component, acme, certbot, certbot-dns-directadmin
  Attempting uninstall: acme
    Found existing installation: acme 2.11.0
    Uninstalling acme-2.11.0:
      Successfully uninstalled acme-2.11.0
  Attempting uninstall: certbot
    Found existing installation: certbot 2.11.0
    Uninstalling certbot-2.11.0:
      Successfully uninstalled certbot-2.11.0
Successfully installed acme-1.32.0 certbot-1.32.0 certbot-dns-directadmin-1.0.6 requests-toolbelt-1.0.0 zope.component-6.0 zope.event-5.0 zope.hookable-7.0 zope.interface-7.0.3
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
root@npm-0304:~# certbot --version
certbot 1.32.0
root@npm-0304:~# systemctl restart npm.service 
root@npm-0304:~#

When i try to generate certificate with wildcard i get:

CommandError: usage: 
  certbot [SUBCOMMAND] [options] [-d DOMAIN] [-d DOMAIN] ...

Certbot can obtain and install HTTPS/TLS/SSL certificates.  By default,
it will attempt to use a webserver both for obtaining and installing the
certificate. 
certbot: error: unrecognized arguments: --directadmin-credentials /etc/letsencrypt/credentials/credentials-2

    at /app/lib/utils.js:16:13
    at ChildProcess.exithandler (node:child_process:410:5)
    at ChildProcess.emit (node:events:513:28)
    at maybeClose (node:internal/child_process:1100:16)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:304:5)
tteck commented 1 month ago

certbot-1.32.0 is from November 2022

Sorry, I'm out of ideas 🤷🏻‍♂️

MaverickPOL commented 1 month ago

certbot-1.32.0 appears when i run pip3 install certbot-dns-directadmin

I belive your changes fix problem with certbot 2.1.0, now it's 2.11.0 after clean install as intended... but becouse you remove this "cloudfare" package, something is missing, or maby nginx try to use diffrent link?
I try to install once again this "old script" from before all today changes, and swap this certbot file like on the begining and see if this will fix problem again.

tteck commented 1 month ago

By removing python3-certbot-dns-cloudflare it would not cause any issues with NPM, nothing is "missing" (yes I'm sure).

tteck commented 1 month ago

Why not try placing your certbot file in /use/local/bin/cerbot with the current script? BTW, the symlink is in the script.

tteck commented 1 month ago

Anyway, good luck to you.

MaverickPOL commented 1 month ago

OK. I'm not sure how to install all of this with this old version of the file...

Last working version for me was this one: https://github.com/tteck/Proxmox/blob/8322aede45d7c230430dcc923e09b92f47f7a848/install/nginxproxymanager-install.sh

After clean install from this script: bash -c "$(wget -qLO - https://github.com/tteck/Proxmox/raw/main/ct/nginxproxymanager.sh)"

I need to change file /usr/bin/certbot to this one:

#!/opt/certbot/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from certbot.main import main
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

When i do that. NPM start work OK. I can generate certificate to domain *.domain.xyz

This works. But i really dont know why. Strange was that certbot --version in this moment still show "2.1.0 version" but NPM works. (after this /usr/bin/certbot change)

Sorry, i don't know how can i help you to diagnose this issue, i only know that now i cannot fix it or workaround in any way :(

MaverickPOL commented 1 month ago

Can you "restore" this version of the file https://github.com/tteck/Proxmox/blob/8322aede45d7c230430dcc923e09b92f47f7a848/install/nginxproxymanager-install.sh" At least i can make workaround on this version so i can use NPM.

tteck commented 1 month ago

https://github.com/tteck/Proxmox/issues/3841#issuecomment-2394851289

MaverickPOL commented 1 month ago

This dont work on this new version...

OK. Thanks for trying. I will turn back to the docker version :(

DNS Challange for Direct Admin dont work right now on this proxmox version :(

MaverickPOL commented 1 month ago

I think this ticket should be open, and wait for some kind of update or maby fix. Right now Direct Admin DNS Challange dont work when deployed by this script.

tteck commented 1 month ago

Direct Admin DNS is not installed by this script. Don't use wildcards

MaverickPOL commented 1 month ago

As i mention before. It was OK, and it's work when i use "old script" and change this one certbot file in /usr/bin. So there is some kind of bug in this script. If you prefer not to fix this, i can't do anything with that. I just try to help so other ppl can use it and it will work.

Right now i have diffrent workaround to use NPM on proxmox. Here is my steps:

  1. I deploy Debian 12 LXC from template (debian-12-standard-12.7-1_amd64.tar.zst)
  2. apt update && apt dist-upgrade
  3. Instal NPM from this script (https://github.com/ej52/proxmox-scripts/tree/main/apps/nginx-proxy-manager) sh -c "$(wget --no-cache -qO- https://raw.githubusercontent.com/ej52/proxmox/main/install.sh)" -s --app nginx-proxy-manager

Everything work OK, DNS on DirectAdmin work OK. Looks like this is nice working version.

Maby can you consider look and try to fix this, or maby build Helper-script to quick deploy from this guide. I leave this to your consideration, unfortunatelly i dont have skills in programing or building scripts, so i can't be more helpfull even if i want to :(

Thank you for your time and help. Best regards.