Impossible to sign my personnal deb repository

[JeanFrancois38]

Version pulp : 3.28.19 katello : 4.10.0

Describe the bug CONTENT_ORIGIN is set in my /etc/pulp/settings.py Then, as pulp I run : export PULP_SETTINGS=“/etc/pulp/settings.py”; pulpcore-manager add-signing-service --class deb:AptReleaseSigningService ubuntu _my_key0B93…0900

Expected behavior Be able to add the sign ability to my pulp deb personnal repo.

Additional context Traceback (most recent call last): File “/usr/bin/pulpcore-manager”, line 33, in sys.exit(load_entry_point(‘pulpcore==3.28.19’, ‘console_scripts’, ‘pulpcore-manager’)()) File “/usr/lib/python3.9/site-packages/pulpcore/app/manage.py”, line 11, in manage execute_from_command_line(sys.argv) File “/usr/lib/python3.9/site-packages/django/core/management/init.py”, line 442, in execute_from_command_line utility.execute() File “/usr/lib/python3.9/site-packages/django/core/management/init.py”, line 382, in execute settings.INSTALLED_APPS File “/usr/lib/python3.9/site-packages/django/conf/init.py”, line 102, in getattr self._setup(name) File “/usr/lib/python3.9/site-packages/django/conf/init.py”, line 89, in _setup self._wrapped = Settings(settings_module) File “/usr/lib/python3.9/site-packages/django/conf/init.py”, line 217, in init mod = importlib.import_module(self.SETTINGS_MODULE) File “/usr/lib64/python3.9/importlib/init.py”, line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File “”, line 1030, in _gcd_import File “”, line 1007, in _find_and_load File “”, line 986, in _find_and_load_unlocked File “”, line 680, in _load_unlocked File “”, line 850, in exec_module File “”, line 228, in _call_with_frames_removed File “/usr/lib/python3.9/site-packages/pulpcore/app/settings.py”, line 381, in settings = DjangoDynaconf( File “/usr/lib/python3.9/site-packages/dynaconf/contrib/django_dynaconf_v2.py”, line 84, in load lazy_settings.populate_obj(django_settings_module) File “/usr/lib/python3.9/site-packages/dynaconf/base.py”, line 115, in getattr self._setup() File “/usr/lib/python3.9/site-packages/dynaconf/base.py”, line 174, in _setup self._wrapped = Settings( File “/usr/lib/python3.9/site-packages/dynaconf/base.py”, line 253, in init self.execute_loaders() File “/usr/lib/python3.9/site-packages/dynaconf/base.py”, line 1025, in execute_loaders self.pre_load(env, silent=silent, key=key) File “/usr/lib/python3.9/site-packages/dynaconf/base.py”, line 1044, in pre_load self.load_file(path=preloads, env=env, silent=silent, key=key) File “/usr/lib/python3.9/site-packages/dynaconf/base.py”, line 1071, in load_file if py_loader.try_to_load_from_py_module_name( File “/usr/lib/python3.9/site-packages/dynaconf/loaders/py_loader.py”, line 68, in try_to_load_from_py_module_name mod = importlib.import_module(str(name)) File “/usr/lib64/python3.9/importlib/init.py”, line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File “”, line 1030, in _gcd_import File “”, line 1007, in _find_and_load File “”, line 986, in _find_and_load_unlocked File “”, line 680, in _load_unlocked File “”, line 850, in exec_module File “”, line 228, in _call_with_frames_removed File “/usr/lib/python3.9/site-packages/pulp_ansible/app/settings.py”, line 24, in ANSIBLE_CONTENT_HOSTNAME = settings.CONTENT_ORIGIN + “/pulp/content” File “/usr/lib/python3.9/site-packages/dynaconf/base.py”, line 115, in getattr self._setup() File “/usr/lib/python3.9/site-packages/dynaconf/base.py”, line 174, in _setup self._wrapped = Settings( File “/usr/lib/python3.9/site-packages/dynaconf/base.py”, line 256, in init self.validators.validate( File “/usr/lib/python3.9/site-packages/dynaconf/validator.py”, line 467, in validate validator.validate( File “/usr/lib/python3.9/site-packages/dynaconf/validator.py”, line 213, in validate self._validate_items( File “/usr/lib/python3.9/site-packages/dynaconf/validator.py”, line 274, in _validate_items raise ValidationError(_message, details=[(self, _message)]) dynaconf.validator.ValidationError: CONTENT_ORIGIN is a required setting but it was not configured. This may be caused by invalid read permissions of the settings file. Note that CONTENT_ORIGIN is set by the installer automatically.

Signing a file works : ./sign_deb_release.sh testfile { "signatures": { "inline": "/tmp/tmp.Nw0QtLbD0h/InRelease", "detached": "/tmp/tmp.Nw0QtLbD0h/Release.gpg" } }

Thanks for your help.

JF

[quba42]

To me this looks like a settings import order bug in pulpcore-manager:

It looks like /usr/lib/python3.9/site-packages/pulp_ansible/app/settings.py wants to use settings.CONTENT_ORIGIN, which is defined in /etc/pulp/settings.py, but for some reason this is not read in.

[quba42]

Note: This issue was prompted by this thread: https://community.theforeman.org/t/try-to-sign-my-personnal-deb-repository/36453/17

[gerrod3]

Can you report what the value of DJANGO_SETTINGS_MODULE is? It should be set to pulpcore.app.settings or be unset (https://github.com/pulp/pulpcore/blob/main/pulpcore/app/manage.py#L7) in order for the management commands to work.

[JeanFrancois38]

It seems to be unset. echo $DJANGO_SETTINGS_MODULE is empty.

[gerrod3]

Re-reading the error message can you list the permissions on the /etc/pulp/settings.py file?

[quba42]

@gerrod3 From the foreman community thread, the permissions were:

$ ls -al /etc/pulp/settings.py
-rw-r----- 1 root pulp 2938 Nov 24 10:58 /etc/pulp/settings.py

This is readable by the Pulp user, and is the same for every other Katello installation I have looked at. I don't think it is a file permissions error. I think the suggestion about file permissions in the error message is a bit misleading, since there are many other possible reasons why a setting might not be set.

[gerrod3]

I am not really sure what is causing the issue, maybe they just need to update to the latest pulp z stream. Has anyone else reproduced this? If you have a link to the katello install instructions I can try to reproduce it.

[quba42]

I will have a go at reproducing this.

[quba42]

I was able to reproduce this on a Foreman 3.9/Katello 4.11 installation that somehow already has a working signing service.

I already have the following working signing service:

# pulp signing-service list --field=name
[
  {
    "name": "katello_deb_sign"
  }
]

I then tried to recreate it under a new name and got:

# sudo -u pulp pulpcore-manager add-signing-service --class deb:AptReleaseSigningService test_signing_service /root/test_sigining_service_sctipr.sh A342C740A62527814A0F491AE345CD7D3ECF7C13
Traceback (most recent call last):
  File "/bin/pulpcore-manager", line 33, in <module>
    sys.exit(load_entry_point('pulpcore==3.39.9', 'console_scripts', 'pulpcore-manager')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pulpcore/app/manage.py", line 11, in manage
    execute_from_command_line(sys.argv)
  File "/usr/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python3.11/site-packages/django/core/management/__init__.py", line 382, in execute
    settings.INSTALLED_APPS
  File "/usr/lib/python3.11/site-packages/django/conf/__init__.py", line 102, in __getattr__
    self._setup(name)
  File "/usr/lib/python3.11/site-packages/django/conf/__init__.py", line 89, in _setup
    self._wrapped = Settings(settings_module)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/django/conf/__init__.py", line 217, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/lib/python3.11/site-packages/pulpcore/app/settings.py", line 383, in <module>
    settings = DjangoDynaconf(
               ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/dynaconf/contrib/django_dynaconf_v2.py", line 84, in load
    lazy_settings.populate_obj(django_settings_module)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/dynaconf/base.py", line 115, in __getattr__
    self._setup()
  File "/usr/lib/python3.11/site-packages/dynaconf/base.py", line 174, in _setup
    self._wrapped = Settings(
                    ^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/dynaconf/base.py", line 253, in __init__
    self.execute_loaders()
  File "/usr/lib/python3.11/site-packages/dynaconf/base.py", line 1025, in execute_loaders
    self.pre_load(env, silent=silent, key=key)
  File "/usr/lib/python3.11/site-packages/dynaconf/base.py", line 1044, in pre_load
    self.load_file(path=preloads, env=env, silent=silent, key=key)
  File "/usr/lib/python3.11/site-packages/dynaconf/base.py", line 1071, in load_file
    if py_loader.try_to_load_from_py_module_name(
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/dynaconf/loaders/py_loader.py", line 68, in try_to_load_from_py_module_name
    mod = importlib.import_module(str(name))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/lib/python3.11/site-packages/pulp_ansible/app/settings.py", line 24, in <module>
    ANSIBLE_CONTENT_HOSTNAME = settings.CONTENT_ORIGIN + "/pulp/content"
                               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/dynaconf/base.py", line 115, in __getattr__
    self._setup()
  File "/usr/lib/python3.11/site-packages/dynaconf/base.py", line 174, in _setup
    self._wrapped = Settings(
                    ^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/dynaconf/base.py", line 256, in __init__
    self.validators.validate(
  File "/usr/lib/python3.11/site-packages/dynaconf/validator.py", line 467, in validate
    validator.validate(
  File "/usr/lib/python3.11/site-packages/dynaconf/validator.py", line 213, in validate
    self._validate_items(
  File "/usr/lib/python3.11/site-packages/dynaconf/validator.py", line 274, in _validate_items
    raise ValidationError(_message, details=[(self, _message)])
dynaconf.validator.ValidationError: CONTENT_ORIGIN is a required setting but it was not configured. This may be caused by invalid read permissions of the settings file. Note that CONTENT_ORIGIN is set by the installation automatically.

Relevant env vars:

# echo $DJANGO_SETTINGS_MODULE

# echo $PULP_SETTINGS
/etc/pulp/settings.py

The full /etc/pulp/settings.py:

# cat /etc/pulp/settings.py 
################################################################################
# File managed by Puppet module: pulpcore
################################################################################
# Not only will edits be overwritten later, there is also a strong
# possibility of breaking the system if changes are made here without making
# required corresponding changes elsewhere. Refer to the documentation used to
# install Pulpcore to determine the safe and persistent way to modify the
# configuration.
################################################################################

# Send anonymous usage data to https://analytics.pulpproject.org/ unless this is
# explicitly defined with value False. This data is used by the Pulp project
# to make informed, data-driven decisions about future feature development.
TELEMETRY = False

CONTENT_ORIGIN = "https://or.master.dev.atix"
SECRET_KEY = "???"
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'pulpcore',
        'USER': 'pulp',
        'PASSWORD': '???',
        'HOST': 'localhost',
        'PORT': '5432',
    },
}
REDIS_URL = "redis://localhost:6379/8"

MEDIA_ROOT = "/var/lib/pulp/media"
STATIC_ROOT = "/var/lib/pulp/assets"
STATIC_URL = "/pulp/assets/"
FILE_UPLOAD_TEMP_DIR = "/var/lib/pulp/tmp"
WORKING_DIRECTORY = "/var/lib/pulp/tmp"

REMOTE_USER_ENVIRON_NAME = 'HTTP_REMOTE_USER'
AUTHENTICATION_BACKENDS = ['pulpcore.app.authentication.PulpNoCreateRemoteUserBackend']

REST_FRAMEWORK__DEFAULT_AUTHENTICATION_CLASSES = (
    'rest_framework.authentication.SessionAuthentication',
    'pulpcore.app.authentication.PulpRemoteUserAuthentication'
)

ALLOWED_IMPORT_PATHS = ["/var/lib/pulp/sync_imports", "/var/lib/pulp/imports"]
ALLOWED_EXPORT_PATHS = ["/var/lib/pulp/exports"]
ALLOWED_CONTENT_CHECKSUMS = ["md5", "sha1", "sha224", "sha256", "sha384", "sha512"]

# Derive HTTP/HTTPS via the X-Forwarded-Proto header set by Apache
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')

CACHE_ENABLED = True

HIDE_GUARDED_DISTRIBUTIONS = True

# IMPORT_WORKERS_PERCENT = 100
LOGGING = {
    "dynaconf_merge": True,
    "loggers": {
        '': {
            'handlers': ['console'],
            'level': 'INFO',
        },
        'pulpcore.deprecation': {
            'handlers': ['console'],
            'level': 'ERROR',
            'propagate': False,
        },
        'django_guid': {
            'handlers': ['console'],
            'level': 'WARNING',
            'propagate': False,
        },
    },
}

# ansible plugin settings
ANSIBLE_API_HOSTNAME = "https://or.master.dev.atix"
ANSIBLE_CONTENT_HOSTNAME = "https://or.master.dev.atix/pulp/content"
ANSIBLE_PERMISSION_CLASSES = []

# container plugin settings
TOKEN_AUTH_DISABLED=True

# deb plugin settings
FORCE_IGNORE_MISSING_PACKAGE_INDICES = True

[quba42]

Update: Looks like I switched to the pulp user, but my PULP_SETTINGS env var was not taken along for the ride. The following worked:

# sudo -u pulp PULP_SETTINGS=/etc/pulp/settings.py pulpcore-manager add-signing-service --class deb:AptReleaseSigningService test_signing_service /opt/orcharhino/apt_signing_service.sh A342C740A62527814A0F491AE345CD7D3ECF7C13
Successfully added signing service test_signing_service for key A342C740A62527814A0F491AE345CD7D3ECF7C13.

Conclusion: The error appears to be a consequence of Pulp not reading the /etc/pulp/settings.py file. @JeanFrancois38 Can you try running the following variation of your command?:

PULP_SETTINGS=/etc/pulp/settings.py pulpcore-manager add-signing-service --class deb:AptReleaseSigningService ubuntu my_key_0B93…0900

Don't use export ...; just run the above as a single command.

[JeanFrancois38]

It works !!!

Thanks