Dietpi-Letsencrypt importlib.metadata.PackageNotFoundError: certbot

[billouetaudrey]

Creating a bug report/issue

Required Information

• DietPi version | cat /boot/dietpi/.version
• Distro version | bulleyes 1
• Kernel version | uname -a

Steps to reproduce

1. dietpi-letsencrypt -- Renew

Actual behaviour

raceback (most recent call last):

  File "/usr/bin/certbot", line 33, in <module>
    sys.exit(load_entry_point('certbot==1.12.0', 'console_scripts', 'certbot')())
  File "/usr/bin/certbot", line 22, in importlib_load_entry_point
    for entry_point in distribution(dist_name).entry_points
  File "/usr/local/lib/python3.9/importlib/metadata.py", line 524, in distribution
    return Distribution.from_name(distribution_name)
  File "/usr/local/lib/python3.9/importlib/metadata.py", line 187, in from_name
    raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: certbot

Extra details

I think it's since I update python using source, to 3.9.9

Do you have any idea how to fix it without doing clean install ?

Thanks

[billouetaudrey]

So I fix it but now I have this

2022-01-24 10:55:32,949:DEBUG:certbot._internal.main:Arguments: []
2022-01-24 10:55:32,950:DEBUG:certbot._internal.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2022-01-24 10:55:32,979:DEBUG:certbot._internal.log:Root logging level set at 20
2022-01-24 10:55:32,980:INFO:certbot._internal.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2022-01-24 10:55:32,982:DEBUG:certbot._internal.plugins.selection:Requested authenticator None and installer None
2022-01-24 10:55:32,983:DEBUG:certbot._internal.plugins.selection:No candidate plugin
2022-01-24 10:55:32,983:DEBUG:certbot._internal.plugins.selection:Selected authenticator None and installer None
2022-01-24 10:55:45,367:DEBUG:certbot._internal.main:certbot version: 1.12.0
2022-01-24 10:55:45,368:DEBUG:certbot._internal.main:Location of certbot entry point: /usr/local/bin/certbot
2022-01-24 10:55:45,368:DEBUG:certbot._internal.main:Arguments: ['--apache', '--redirect', '--agree-tos', '--no-eff-email', '--rsa-key-size', '4096', '-m', 'audreydarllen@gmail.com', '-d', 'audreydarllen.fr,audreydarllen.com,billouetaudrey.site']
2022-01-24 10:55:45,368:DEBUG:certbot._internal.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2022-01-24 10:55:45,398:DEBUG:certbot._internal.log:Root logging level set at 20
2022-01-24 10:55:45,399:INFO:certbot._internal.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2022-01-24 10:55:45,401:DEBUG:certbot._internal.plugins.selection:Requested authenticator apache and installer apache
2022-01-24 10:55:45,401:DEBUG:certbot._internal.plugins.selection:No candidate plugin
2022-01-24 10:55:45,401:DEBUG:certbot._internal.plugins.selection:Selected authenticator None and installer None

[billouetaudrey]

I think it's because it want Certbot python 1.12.0 , I have this error with 1.22.0

root@raspberrypi:~# certbot
/usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py:116: PkgResourcesDeprecationWarning: 1.0.8-pyro-mod is an invalid version and will not be supported in a future release
  warnings.warn(
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py", line 573, in _build_master
    ws.require(__requires__)
  File "/usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py", line 891, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py", line 782, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (certbot 1.22.0 (/usr/local/lib/python3.9/site-packages), Requirement.parse('certbot==1.12.0'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/certbot", line 33, in <module>
    sys.exit(load_entry_point('certbot==1.12.0', 'console_scripts', 'certbot')())
  File "/usr/bin/certbot", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/usr/local/lib/python3.9/importlib/metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/usr/local/lib/python3.9/site-packages/certbot/main.py", line 6, in <module>
    from certbot._internal import main as internal_main
  File "/usr/local/lib/python3.9/site-packages/certbot/_internal/main.py", line 25, in <module>
    from certbot import configuration
  File "/usr/local/lib/python3.9/site-packages/certbot/configuration.py", line 10, in <module>
    from certbot import util
  File "/usr/local/lib/python3.9/site-packages/certbot/util.py", line 27, in <module>
    from certbot._internal import constants
  File "/usr/local/lib/python3.9/site-packages/certbot/_internal/constants.py", line 6, in <module>
    import pkg_resources
  File "/usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3267, in <module>
    def _initialize_master_working_set():
  File "/usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3241, in _call_aside
    f(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py", line 3279, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py", line 575, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py", line 588, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/local/lib/python3.9/site-packages/pkg_resources/__init__.py", line 777, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'certbot==1.12.0' distribution was not found and is required by the application'

[Joulinar]

You could have filled the bug report. At least this would have give some indication on your system. Maybe you could do that still

• DietPi version | cat /boot/dietpi/.version
• Distro version | echo $G_DISTRO_NAME $G_RASPBIAN
• Kernel version | uname -a
• SBC model | echo $G_HW_MODEL_NAME or (EG: RPi3)
• Web server used

[billouetaudrey]

cat /boot/dietpi/.version G_DIETPI_VERSION_CORE=8 G_DIETPI_VERSION_SUB=0 G_DIETPI_VERSION_RC=2 G_GITBRANCH='beta' G_GITOWNER='MichaIng' echo $G_DISTRO_NAME $G_RASPBIA bullseye uname -a Linux raspberrypi 5.10.92-v7l+ #1514 SMP Mon Jan 17 17:38:03 GMT 2022 armv7l GNU/Linux echo $G_HW_MODEL_NAME RPi 4 Model B (armv7l) Webserver : Apache2

[MichaIng]

pkg_resources.VersionConflict: (certbot 1.22.0 (/usr/local/lib/python3.9/site-packages), Requirement.parse('certbot==1.12.0'))

I hope it is clear that you cannot run Debian's Certbot package on Bullseye with Python 3.9.9. You need to pull a Python 3.9.9 compatible version from elsewhere then and assure that all required Python modules are installed.

Based on the error message above, probably pip3 install 'certbot==1.12.0' may solve it, or apt purge certbot and either calling Certbot via python3 -m certbot or installing/creating an own certbot command which uses/expects the newest Certbot versions.

[Joulinar]

but if it is Apache, does it require python module certbot-apache ?

[MichaIng]

Ah yes, that is true as well, at least dietpi-letsencrypt uses the --apache flag then to apply the webserver config:

pip3 install certbot-apache

I'm not sure how it is handled with the manual Python wheel/module installs, whether python3 -m certbot or python3 -m certbot-apache then needs to be called. It however must be done via certbot command, in case a shell wrapper script, since this is what dietpi-letsencrypt calls. Of course it can be called manually as well, where Certbot should guide you through everything.

[billouetaudrey]

I think I kill everything lool

I will abandoned, sorry

[MichaIng]

Generally, mixing Debian Python packages with custom Python versions and modules is not a good idea, unless you know how to resolve, respectively how to replace all Debian Python packages with the respective upstream modules then e.g. via pip/pip3. The packages then need to be purged at best to not leave any non-functional executables in place (like the certbot command in your case) and the tools may need to be called differently.

[billouetaudrey]

Ok will know for next clean install. Thanks

[MichaIng]

Btw, not sure why you aim to install Python 3.9.9, but the version provided by stable/maintained Debian repositories should be pretty fine, they get security patches and bug fixes cherry picked from upstream, when anything comes up, especially an important library/runtime like Python, used and monitored by a lot of Debian users and developers 🙂.

[billouetaudrey]

No real reason lol I use Telegram bot and I like to see I have the latest version lmao

[MichaIng]

pyenv allows to install/compile a different Python version in an isolated environment. We use this e.g. for Home Assistant, which has strict Python version and modules version requirements: https://github.com/MichaIng/DietPi/blob/936e364/dietpi/dietpi-software#L12076

That way you can run specific applications with other Python versions without messing with the Debian packages 😉.

[billouetaudrey]

Will check how pyenv can help me, or I will just not update python ;)

Why python debian is not often update ?

[MichaIng]

A major concept of Debian for stability and cross-compatibility of its ~60,000 packages is that packages do not receive (feature) version upgrades, once a release hit a freeze stage (~half a year before official release). At best patch versions are applied and security + bug fix patches backported. This is in opposition to rolling release distro models, where new major software versions are available quickly after official/upstream release, but when you upgrade, they may break everything else which depends on them, invalidate your configurations etc, just like you faced with Python and Certbot now 😉. This means, on Debian you can always run apt update && apt upgrade and never need to fear that anything is not working anymore due to conflicts, dependency issues, breaking changes, config invalidation and all such. This is why some may say that Arch Linux is a "difficult" Linux distribution and Arch users are proud to be Arch users: You are much more responsible yourself to keep up everything running when applying upgrades, while in every other aspect its just as simple as any other non-GUI distro. This is why servers use Debian, Ubuntu, RHEL (or derivatives) or other non-rolling-release distros, where security patches are automatically served and safe to apply, while a rolling-release distro would easily break your server, completely inapplicable for anything serious, as long as the admin has not much fun with reviewing and tinkering regularly on a testing clone.

[billouetaudrey]

So I really need to stop always wanting to update everything, like I do on Android. If it works, I don't touch

Thanks very much !

[MichaIng]

like I do on Android

On Android you just do not recognise how outdated everything is under the hood, like old Linux kernel and libraries etc 😄. Also what you get offered from the Play Store may not be the actual newest upstream release, especially when using older Android versions, just like it is with Debian and apt update && apt upgrade.

But yes, basically keeping everything on bleeding edge is not an applicable aim for a server and extremely maintenance-heavy when trying to achieve, including the fact that you cannot use Debian packages then in many cases but need to compile yourself + watch out for security patches and bug fixes yourself, then recompiling etc.

Check out for available APT upgrades, which is shown as well in the DietPi login banner. These are reasonable and save to apply 🙂.

[billouetaudrey]

I talk about update new build as soon as build is out ( I use custom rom / custom kernel )

It's more easy to restore an Android device as Linux lol Because with Android, you can backup only app+data ;)

"Check out for available APT upgrades, which is shown as well in the DietPi login banner. These are reasonable and save to apply slightly_smiling_face." Yes I will now !

Best regards

[MichaIng]

I talk about update new build as soon as build is out ( I use custom rom / custom kernel )

Ah okay. Did so in the past, but difficult with current phone, closed source, many regressions with a custom kernel 😢.

you can backup only app+data ;)

That is indeed difficult on Linux distros with FHS since configs and data are spread across so many different directories, many tied to distro/package versions which you do not want to backup/migrate, other you do etc...

[billouetaudrey]

Ohok :(

So need to backup more than one backup :)

[MichaIng]

Ah btw, also with Debian there is an option to be on near to bleeding edge: You can use Debian Bookworm, which is the "testing" version, being released ~summer 2023. Using Debian testing is something in the middle between stable and rolling-release. The repository itself is guaranteed to be consistent, i.e. packages are all compatible with each other at any point (compared to "unstable" and "experimental" suites). And new software versions/packages are usually well reviewed and tested before being merged from "unstable" into "testing". But you need to take care about breaking changes, like with Arch Linux and other rolling-release distros.

E.g. I run my home server on Bookworm, and some testing VMs and images for SBCs. Just today the upgrade to Python 3.9.10 was offered and the final migration step to PHP8.1. The letter caused me exactly the maintenance trouble I talked about above: PHP7.4 was purged automatically with the upgrade, but my Nextcloud does not support PHP8.1 yet. So I needed to manually hack the Nextcloud version check to accept PHP8.1, check whether it is basically working (throws a bunch of deprecation warnings but otherwise fine), migrate custom systemd and PHP configs/modules and adjust the webserver configuration to use the new PHP-FPM server. For a home server, with some experience, not a big issues, but if anything depends on it, well difficult at least.

We offer some Bookworm images already for testing: https://dietpi.com/downloads/images/ Surely not all software installs will work without issues yet, Nextcloud one known example due to outstanding PHP8.1 support, we however start to test it whenever doing changes to a software implementation: https://github.com/MichaIng/DietPi/wiki/Debian-Bookworm-testing