Cannot add allowed content checksums to existing Pulp instance

[maximiliankolb]

Version

collections:
  - name: pulp.pulp_installer
    version: 3.20.3

Describe the bug If I add a list of allowed_content_checksums, Ansible does not automatically run pulpcore-manager handle-artifact-checksums to reflect the change in the database.

To Reproduce Add the following variable to your Ansible playbook to deploy Pulp:

  vars:
    ...
    pulp_settings:
      allowed_content_checksums:
        - sha1
        - sha224
        - sha256
        - sha384
        - sha512

Expected behavior Probably a handler to run this command if this variable is changed.

Additional context I am trying to synchronize https://yum.puppet.com/puppet7/el/7/x86_64/. Pulp throws the following error:

"error": {
    "traceback": "  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/tasking/pulpcore_worker.py\", line 445, in _perform_task\n    result = func(*args, **kwargs)\n  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulp_rpm/app/tasks/synchronizing.py\", line 556, in synchronize\n    repo_version = dv.create() or repo.latest_version()\n  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/plugin/stages/declarative_version.py\", line 161, in create\n    loop.run_until_complete(pipeline)\n  File \"/usr/lib64/python3.9/asyncio/base_events.py\", line 642, in run_until_complete\n    return future.result()\n  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/plugin/stages/api.py\", line 225, in create_pipeline\n    await asyncio.gather(*futures)\n  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/plugin/stages/api.py\", line 43, in __call__\n    await self.run()\n  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulp_rpm/app/tasks/synchronizing.py\", line 767, in run\n    downloader = self.remote.get_downloader(\n  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulp_rpm/app/models/repository.py\", line 104, in get_downloader\n    return super().get_downloader(remote_artifact=remote_artifact, url=url, **kwargs)\n  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/app/models/repository.py\", line 456, in get_downloader\n    return download_factory.build(url, **kwargs)\n  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/download/factory.py\", line 176, in build\n    return builder(download_class, url, **kwargs)\n  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/download/factory.py\", line 208, in _http_or_https\n    return download_class(url, **options, **kwargs)\n  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulp_rpm/app/downloaders.py\", line 65, in __init__\n    super().__init__(*args, **kwargs)\n  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/download/http.py\", line 177, in __init__\n    super().__init__(url, **kwargs)\n  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/download/base.py\", line 103, in __init__\n    raise UnsupportedDigestValidationError(\n",
    "description": "Content at the URL 'https://yum.puppet.com/puppet7/el/7/x86_64/repodata/cacb2abfc50eaaf1acf955f20b4dd725df4e2926-filelists.xml.gz' does not contain at least one trusted hasher which is specified in the 'ALLOWED_CONTENT_CHECKSUMS' setting (['sha512', 'sha384', 'sha256', 'sha224']). The downloader expected one of the following hashers: {'sha1'}"
  },

[maximiliankolb]

cc @quba42 @goarsna

[maximiliankolb]

Output when running the pulp_installer:

TASK [pulp.pulp_installer.pulp_database_config : Check if admin account has been created] ***
fatal: [pulp_server]: FAILED! => {"changed": false, "cmd": ["/usr/local/lib/pulp/bin/pulpcore-manager", "shell", "-c", "from django.contrib.auth import get_user_model; print(get_user_model().objects.filter(username=\"admin\").exists())"], "delta": "0:00:02.842596", "end": "2022-12-09 09:15:01.934929", "msg": "non-zero return code", "rc": 1, "start": "2022-12-09 09:14:59.092333", "stderr": "Traceback (most recent call last):\n  File \"/usr/local/lib/pulp/bin/pulpcore-manager\", line 8, in <module>\n    sys.exit(manage())\n  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/app/manage.py\", line 11, in manage\n    execute_from_command_line(sys.argv)\n  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/django/core/management/__init__.py\", line 419, in execute_from_command_line\n    utility.execute()\n  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/django/core/management/__init__.py\", line 395, in execute\n    django.setup()\n  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/django/__init__.py\", line 24, in setup\n    apps.populate(settings.INSTALLED_APPS)\n  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/django/apps/registry.py\", line 114, in populate\n    app_config.import_models()\n  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/django/apps/config.py\", line 301, in import_models\n    self.models_module = import_module(models_module_name)\n  File \"/usr/lib64/python3.9/importlib/__init__.py\", line 127, in import_module\n    return _bootstrap._gcd_import(name[level:], package, level)\n  File \"<frozen importlib._bootstrap>\", line 1030, in _gcd_import\n  File \"<frozen importlib._bootstrap>\", line 1007, in _find_and_load\n  File \"<frozen importlib._bootstrap>\", line 986, in _find_and_load_unlocked\n  File \"<frozen importlib._bootstrap>\", line 680, in _load_unlocked\n  File \"<frozen importlib._bootstrap_external>\", line 850, in exec_module\n  File \"<frozen importlib._bootstrap>\", line 228, in _call_with_frames_removed\n  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/app/models/__init__.py\", line 32, in <module>\n    from .exporter import (  # noqa\n  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/app/models/exporter.py\", line 11, in <module>\n    from pulpcore.app.models.repository import Repository\n  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/app/models/repository.py\", line 25, in <module>\n    from pulpcore.cache import Cache\n  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/cache/__init__.py\", line 1, in <module>\n    from .cache import (  # noqa\n  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/cache/cache.py\", line 16, in <module>\n    from pulpcore.app.settings import settings\n  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/app/settings.py\", line 453, in <module>\n    raise e\n  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/app/settings.py\", line 412, in <module>\n    raise ImproperlyConfigured(\ndjango.core.exceptions.ImproperlyConfigured: There have been identified artifacts missing checksum 'sha1'. Run 'pulpcore-manager handle-artifact-checksums' first to populate missing artifact checksums.", "stderr_lines": ["Traceback (most recent call last):", "  File \"/usr/local/lib/pulp/bin/pulpcore-manager\", line 8, in <module>", "    sys.exit(manage())", "  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/app/manage.py\", line 11, in manage", "    execute_from_command_line(sys.argv)", "  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/django/core/management/__init__.py\", line 419, in execute_from_command_line", "    utility.execute()", "  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/django/core/management/__init__.py\", line 395, in execute", "    django.setup()", "  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/django/__init__.py\", line 24, in setup", "    apps.populate(settings.INSTALLED_APPS)", "  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/django/apps/registry.py\", line 114, in populate", "    app_config.import_models()", "  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/django/apps/config.py\", line 301, in import_models", "    self.models_module = import_module(models_module_name)", "  File \"/usr/lib64/python3.9/importlib/__init__.py\", line 127, in import_module", "    return _bootstrap._gcd_import(name[level:], package, level)", "  File \"<frozen importlib._bootstrap>\", line 1030, in _gcd_import", "  File \"<frozen importlib._bootstrap>\", line 1007, in _find_and_load", "  File \"<frozen importlib._bootstrap>\", line 986, in _find_and_load_unlocked", "  File \"<frozen importlib._bootstrap>\", line 680, in _load_unlocked", "  File \"<frozen importlib._bootstrap_external>\", line 850, in exec_module", "  File \"<frozen importlib._bootstrap>\", line 228, in _call_with_frames_removed", "  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/app/models/__init__.py\", line 32, in <module>", "    from .exporter import (  # noqa", "  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/app/models/exporter.py\", line 11, in <module>", "    from pulpcore.app.models.repository import Repository", "  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/app/models/repository.py\", line 25, in <module>", "    from pulpcore.cache import Cache", "  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/cache/__init__.py\", line 1, in <module>", "    from .cache import (  # noqa", "  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/cache/cache.py\", line 16, in <module>", "    from pulpcore.app.settings import settings", "  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/app/settings.py\", line 453, in <module>", "    raise e", "  File \"/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/app/settings.py\", line 412, in <module>", "    raise ImproperlyConfigured(", "django.core.exceptions.ImproperlyConfigured: There have been identified artifacts missing checksum 'sha1'. Run 'pulpcore-manager handle-artifact-checksums' first to populate missing artifact checksums."], "stdout": "", "stdout_lines": []}

[quba42]

Essentially pulpcore-manager handle-artifact-checksums should be run from a handler, whenever a change to allowed_content_checksums is applied to /etc/pulp/settings.py.

I am not sure in what role allowed_content_checksums is actually applied, but this should not be so hard to find out/implement.

[stale[bot]]

This issue has been marked 'stale' due to lack of recent activity. If there is no further activity, the issue will be closed in another 30 days. Thank you for your contribution!

[stale[bot]]

This issue has been closed due to inactivity. If you feel this is in error, please reopen the issue or file a new issue with the relevant details.