Certbot Renew of Cloudflare Funtions Not Works

[ChongZhiJie0216]

Checklist

• Have you pulled and found the error with jc21/nginx-proxy-manager:latest docker image?
  • Yes
• Are you sure you're not using someone else's docker image?
  • Yes
• Have you searched for similar issues (both open and closed)?
  • Yes

Describe the bug

Nginx Proxy Manager Version

2.11.1

To Reproduce Steps to reproduce the behavior:

1. Go to 'SSL Certificates'
2. Click on '...'
3. Scroll down to 'Renew Now'
4. See error

Expected behavior

Traceback (most recent call last):
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/plugins/disco.py", line 191, in find_all
    cls._load_entry_point(entry_point, plugins)
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/plugins/disco.py", line 203, in _load_entry_point
    plugin_ep = PluginEntryPoint(entry_point)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/plugins/disco.py", line 42, in __init__
    self.plugin_cls: Type[interfaces.Plugin] = entry_point.load()
                                               ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/importlib/metadata/__init__.py", line 202, in load
    module = import_module(match.group('module'))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/opt/certbot/lib/python3.11/site-packages/certbot_dns_cloudflare/_internal/dns_cloudflare.py", line 9, in <module>
    import CloudFlare
ModuleNotFoundError: No module named 'CloudFlare'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/certbot/bin/certbot", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/certbot/lib/python3.11/site-packages/certbot/main.py", line 19, in main
    return internal_main.main(cli_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/main.py", line 1866, in main
    plugins = plugins_disco.PluginsRegistry.find_all()
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/plugins/disco.py", line 193, in find_all
    raise errors.PluginError(
certbot.errors.PluginError: The 'certbot_dns_cloudflare._internal.dns_cloudflare' plugin errored while loading: No module named 'CloudFlare'. You may need to remove or update this plugin. The Certbot log will contain the full error details and this should be reported to the plugin developer.
2024-06-25 05:33:39,715:ERROR:certbot._internal.log:The 'certbot_dns_cloudflare._internal.dns_cloudflare' plugin errored while loading: No module named 'CloudFlare'. You may need to remove or update this plugin. The Certbot log will contain the full error details and this should be reported to the plugin developer.

Screenshots

Operating System

Additional context

Client: Docker Engine - Community
 Version:           26.1.1
 API version:       1.45
 Go version:        go1.21.9
 Git commit:        4cf5afa
 Built:             Tue Apr 30 11:48:02 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          26.1.1
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.9
  Git commit:       ac2de55
  Built:            Tue Apr 30 11:48:02 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.31
  GitCommit:        e377cd56a71523140ca6ae87e30244719194a521
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

[zaourzag]

Crosslinking my issue #3824

[ChongZhiJie0216]

Crosslinking my issue #3824

I encountered the following error after following your method

I tried using certbot renew and the result was the same

Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/npm-17.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Renewal configuration file /etc/letsencrypt/renewal/npm-17.conf is broken.
The error was: expected /etc/letsencrypt/live/npm-17/cert.pem to be a symlink
Skipping.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/npm-3.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Renewal configuration file /etc/letsencrypt/renewal/npm-3.conf is broken.
The error was: expected /etc/letsencrypt/live/npm-3/cert.pem to be a symlink
Skipping.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
No renewals were attempted.

Additionally, the following renewal configurations were invalid:
  /etc/letsencrypt/renewal/npm-17.conf (parsefail)
  /etc/letsencrypt/renewal/npm-3.conf (parsefail)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
0 renew failure(s), 2 parse failure(s)
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

This is /var/log/letsencrypt/letsencrypt.log

2024-06-25 08:38:12,732:DEBUG:certbot._internal.main:certbot version: 2.11.0
2024-06-25 08:38:12,732:DEBUG:certbot._internal.main:Location of certbot entry point: /opt/certbot/bin/certbot
2024-06-25 08:38:12,732:DEBUG:certbot._internal.main:Arguments: []
2024-06-25 08:38:12,732:DEBUG:certbot._internal.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#dns-cloudflare,PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2024-06-25 08:38:12,745:DEBUG:certbot._internal.log:Root logging level set at 30
2024-06-25 08:38:12,746:DEBUG:certbot._internal.display.obj:Notifying user: Processing /etc/letsencrypt/renewal/npm-17.conf
2024-06-25 08:38:12,747:ERROR:certbot._internal.renewal:Renewal configuration file /etc/letsencrypt/renewal/npm-17.conf is broken.
2024-06-25 08:38:12,747:ERROR:certbot._internal.renewal:The error was: expected /etc/letsencrypt/live/npm-17/cert.pem to be a symlink
Skipping.
2024-06-25 08:38:12,748:DEBUG:certbot._internal.renewal:Traceback was:
Traceback (most recent call last):
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/renewal.py", line 76, in reconstitute
    renewal_candidate = storage.RenewableCert(full_path, config)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/storage.py", line 510, in __init__
    self._check_symlinks()
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/storage.py", line 589, in _check_symlinks
    raise errors.CertStorageError(
certbot.errors.CertStorageError: expected /etc/letsencrypt/live/npm-17/cert.pem to be a symlink

2024-06-25 08:38:12,748:DEBUG:certbot._internal.display.obj:Notifying user: Processing /etc/letsencrypt/renewal/npm-3.conf
2024-06-25 08:38:12,749:ERROR:certbot._internal.renewal:Renewal configuration file /etc/letsencrypt/renewal/npm-3.conf is broken.
2024-06-25 08:38:12,749:ERROR:certbot._internal.renewal:The error was: expected /etc/letsencrypt/live/npm-3/cert.pem to be a symlink
Skipping.
2024-06-25 08:38:12,749:DEBUG:certbot._internal.renewal:Traceback was:
Traceback (most recent call last):
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/renewal.py", line 76, in reconstitute
    renewal_candidate = storage.RenewableCert(full_path, config)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/storage.py", line 510, in __init__
    self._check_symlinks()
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/storage.py", line 589, in _check_symlinks
    raise errors.CertStorageError(
certbot.errors.CertStorageError: expected /etc/letsencrypt/live/npm-3/cert.pem to be a symlink

2024-06-25 08:38:12,749:DEBUG:certbot._internal.display.obj:Notifying user:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
2024-06-25 08:38:12,750:DEBUG:certbot._internal.display.obj:Notifying user: No renewals were attempted.
2024-06-25 08:38:12,750:DEBUG:certbot._internal.display.obj:Notifying user:
Additionally, the following renewal configurations were invalid:
2024-06-25 08:38:12,750:DEBUG:certbot._internal.display.obj:Notifying user:   /etc/letsencrypt/renewal/npm-17.conf (parsefail)
  /etc/letsencrypt/renewal/npm-3.conf (parsefail)
2024-06-25 08:38:12,750:DEBUG:certbot._internal.display.obj:Notifying user: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
2024-06-25 08:38:12,750:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "/opt/certbot/bin/certbot", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/certbot/lib/python3.11/site-packages/certbot/main.py", line 19, in main
    return internal_main.main(cli_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/main.py", line 1894, in main
    return config.func(config, plugins)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/main.py", line 1642, in renew
    renewed_domains, failed_domains = renewal.handle_renewal_request(config)
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/renewal.py", line 568, in handle_renewal_request
    raise errors.Error(
certbot.errors.Error: 0 renew failure(s), 2 parse failure(s)
2024-06-25 08:38:12,751:ERROR:certbot._internal.log:0 renew failure(s), 2 parse failure(s)

[ransbachm]

Have you tried the newest dev build? Seems like it's available on hub

[ChongZhiJie0216]

Have you tried the newest dev build? Seems like it's available on hub

Sorry,When I click on the HUB link you sent, a 404 screen appears.

[ransbachm]

https://hub.docker.com/layers/jc21/nginx-proxy-manager/github-develop/images/sha256-c69e1466d79ea641bfa3f675b8babeaaf48f3bfc850908056f1c8f97ea8b1ee9

whoops

[ChongZhiJie0216]

I change it to to jc21/nginx-proxy-manager:github-develop but still have error it

2024-06-26 08:22:24,312:DEBUG:certbot._internal.main:certbot version: 2.11.0
2024-06-26 08:22:24,312:DEBUG:certbot._internal.main:Location of certbot entry point: /opt/certbot/bin/certbot
2024-06-26 08:22:24,312:DEBUG:certbot._internal.main:Arguments: ['--force-renewal', '--config', '/etc/letsencrypt.ini', '--work-dir', '/tmp/letsencrypt-lib', '--logs-dir', '/tmp/letsencrypt-log', '--cert-name', 'npm-17', '--disable-hook-validation', '--no-random-sleep-on-renew']
2024-06-26 08:22:24,312:DEBUG:certbot._internal.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#dns-cloudflare,PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2024-06-26 08:22:24,329:DEBUG:certbot._internal.log:Root logging level set at 30
2024-06-26 08:22:24,330:DEBUG:certbot._internal.display.obj:Notifying user: Processing /etc/letsencrypt/renewal/npm-17.conf
2024-06-26 08:22:24,331:ERROR:certbot._internal.renewal:Renewal configuration file /etc/letsencrypt/renewal/npm-17.conf is broken.
2024-06-26 08:22:24,331:ERROR:certbot._internal.renewal:The error was: expected /etc/letsencrypt/live/npm-17/cert.pem to be a symlink
Skipping.
2024-06-26 08:22:24,332:DEBUG:certbot._internal.renewal:Traceback was:
Traceback (most recent call last):
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/renewal.py", line 76, in reconstitute
    renewal_candidate = storage.RenewableCert(full_path, config)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/storage.py", line 510, in __init__
    self._check_symlinks()
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/storage.py", line 589, in _check_symlinks
    raise errors.CertStorageError(
certbot.errors.CertStorageError: expected /etc/letsencrypt/live/npm-17/cert.pem to be a symlink

2024-06-26 08:22:24,332:DEBUG:certbot._internal.display.obj:Notifying user:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
2024-06-26 08:22:24,332:DEBUG:certbot._internal.display.obj:Notifying user: No renewals were attempted.
2024-06-26 08:22:24,332:DEBUG:certbot._internal.display.obj:Notifying user:
Additionally, the following renewal configurations were invalid:
2024-06-26 08:22:24,333:DEBUG:certbot._internal.display.obj:Notifying user:   /etc/letsencrypt/renewal/npm-17.conf (parsefail)
2024-06-26 08:22:24,333:DEBUG:certbot._internal.display.obj:Notifying user: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
2024-06-26 08:22:24,333:DEBUG:certbot._internal.log:Exiting abnormally:
Traceback (most recent call last):
  File "/opt/certbot/bin/certbot", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/certbot/lib/python3.11/site-packages/certbot/main.py", line 19, in main
    return internal_main.main(cli_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/main.py", line 1894, in main
    return config.func(config, plugins)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/main.py", line 1642, in renew
    renewed_domains, failed_domains = renewal.handle_renewal_request(config)
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/certbot/lib/python3.11/site-packages/certbot/_internal/renewal.py", line 568, in handle_renewal_request
    raise errors.Error(
certbot.errors.Error: 0 renew failure(s), 1 parse failure(s)
2024-06-26 08:22:24,334:ERROR:certbot._internal.log:0 renew failure(s), 1 parse failure(s)

[ransbachm]

I have fixed this once by removing all of the container including any data (certs and hosts) which was a bit of work. There’s probably another, better, fix but I can’t help you there. Also not sure if it will work for you

[paradox1612]

the Cloudflare module is not installed in your Certbot environment. All you need to do is reinstall the cloudflare modeule using pip. docker exec -it /bin/bash pip install --upgrade cloudflare==2.19.* restart the coointainer. I have written a detailed guide about it : https://blog.thekush.dev/how-to-fix-nginx-manager-certbot_dns_cloudflare-_internal-dns_cloudflare-plugin-error/

[RomanTrifanov]

the Cloudflare module is not installed in your Certbot environment. All you need to do is reinstall the cloudflare modeule using pip. docker exec -it /bin/bash pip install --upgrade cloudflare==2.19.* restart the coointainer. I have written a detailed guide about it : https://blog.thekush.dev/how-to-fix-nginx-manager-certbot_dns_cloudflare-_internal-dns_cloudflare-plugin-error/

It works for me, thanks