Closed JeanFrancois38 closed 7 months ago
To me this looks like a settings import order bug in pulpcore-manager:
It looks like /usr/lib/python3.9/site-packages/pulp_ansible/app/settings.py
wants to use settings.CONTENT_ORIGIN
, which is defined in /etc/pulp/settings.py
, but for some reason this is not read in.
Note: This issue was prompted by this thread: https://community.theforeman.org/t/try-to-sign-my-personnal-deb-repository/36453/17
Can you report what the value of DJANGO_SETTINGS_MODULE
is? It should be set to pulpcore.app.settings
or be unset (https://github.com/pulp/pulpcore/blob/main/pulpcore/app/manage.py#L7) in order for the management commands to work.
It seems to be unset. echo $DJANGO_SETTINGS_MODULE is empty.
Re-reading the error message can you list the permissions on the /etc/pulp/settings.py file?
@gerrod3 From the foreman community thread, the permissions were:
$ ls -al /etc/pulp/settings.py
-rw-r----- 1 root pulp 2938 Nov 24 10:58 /etc/pulp/settings.py
This is readable by the Pulp user, and is the same for every other Katello installation I have looked at. I don't think it is a file permissions error. I think the suggestion about file permissions in the error message is a bit misleading, since there are many other possible reasons why a setting might not be set.
I am not really sure what is causing the issue, maybe they just need to update to the latest pulp z stream. Has anyone else reproduced this? If you have a link to the katello install instructions I can try to reproduce it.
I will have a go at reproducing this.
I was able to reproduce this on a Foreman 3.9/Katello 4.11 installation that somehow already has a working signing service.
I already have the following working signing service:
# pulp signing-service list --field=name
[
{
"name": "katello_deb_sign"
}
]
I then tried to recreate it under a new name and got:
# sudo -u pulp pulpcore-manager add-signing-service --class deb:AptReleaseSigningService test_signing_service /root/test_sigining_service_sctipr.sh A342C740A62527814A0F491AE345CD7D3ECF7C13
Traceback (most recent call last):
File "/bin/pulpcore-manager", line 33, in <module>
sys.exit(load_entry_point('pulpcore==3.39.9', 'console_scripts', 'pulpcore-manager')())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/pulpcore/app/manage.py", line 11, in manage
execute_from_command_line(sys.argv)
File "/usr/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
utility.execute()
File "/usr/lib/python3.11/site-packages/django/core/management/__init__.py", line 382, in execute
settings.INSTALLED_APPS
File "/usr/lib/python3.11/site-packages/django/conf/__init__.py", line 102, in __getattr__
self._setup(name)
File "/usr/lib/python3.11/site-packages/django/conf/__init__.py", line 89, in _setup
self._wrapped = Settings(settings_module)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/django/conf/__init__.py", line 217, in __init__
mod = importlib.import_module(self.SETTINGS_MODULE)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/usr/lib/python3.11/site-packages/pulpcore/app/settings.py", line 383, in <module>
settings = DjangoDynaconf(
^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/dynaconf/contrib/django_dynaconf_v2.py", line 84, in load
lazy_settings.populate_obj(django_settings_module)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/dynaconf/base.py", line 115, in __getattr__
self._setup()
File "/usr/lib/python3.11/site-packages/dynaconf/base.py", line 174, in _setup
self._wrapped = Settings(
^^^^^^^^^
File "/usr/lib/python3.11/site-packages/dynaconf/base.py", line 253, in __init__
self.execute_loaders()
File "/usr/lib/python3.11/site-packages/dynaconf/base.py", line 1025, in execute_loaders
self.pre_load(env, silent=silent, key=key)
File "/usr/lib/python3.11/site-packages/dynaconf/base.py", line 1044, in pre_load
self.load_file(path=preloads, env=env, silent=silent, key=key)
File "/usr/lib/python3.11/site-packages/dynaconf/base.py", line 1071, in load_file
if py_loader.try_to_load_from_py_module_name(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/dynaconf/loaders/py_loader.py", line 68, in try_to_load_from_py_module_name
mod = importlib.import_module(str(name))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/usr/lib/python3.11/site-packages/pulp_ansible/app/settings.py", line 24, in <module>
ANSIBLE_CONTENT_HOSTNAME = settings.CONTENT_ORIGIN + "/pulp/content"
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/dynaconf/base.py", line 115, in __getattr__
self._setup()
File "/usr/lib/python3.11/site-packages/dynaconf/base.py", line 174, in _setup
self._wrapped = Settings(
^^^^^^^^^
File "/usr/lib/python3.11/site-packages/dynaconf/base.py", line 256, in __init__
self.validators.validate(
File "/usr/lib/python3.11/site-packages/dynaconf/validator.py", line 467, in validate
validator.validate(
File "/usr/lib/python3.11/site-packages/dynaconf/validator.py", line 213, in validate
self._validate_items(
File "/usr/lib/python3.11/site-packages/dynaconf/validator.py", line 274, in _validate_items
raise ValidationError(_message, details=[(self, _message)])
dynaconf.validator.ValidationError: CONTENT_ORIGIN is a required setting but it was not configured. This may be caused by invalid read permissions of the settings file. Note that CONTENT_ORIGIN is set by the installation automatically.
Relevant env vars:
# echo $DJANGO_SETTINGS_MODULE
# echo $PULP_SETTINGS
/etc/pulp/settings.py
The full /etc/pulp/settings.py
:
# cat /etc/pulp/settings.py
################################################################################
# File managed by Puppet module: pulpcore
################################################################################
# Not only will edits be overwritten later, there is also a strong
# possibility of breaking the system if changes are made here without making
# required corresponding changes elsewhere. Refer to the documentation used to
# install Pulpcore to determine the safe and persistent way to modify the
# configuration.
################################################################################
# Send anonymous usage data to https://analytics.pulpproject.org/ unless this is
# explicitly defined with value False. This data is used by the Pulp project
# to make informed, data-driven decisions about future feature development.
TELEMETRY = False
CONTENT_ORIGIN = "https://or.master.dev.atix"
SECRET_KEY = "???"
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'pulpcore',
'USER': 'pulp',
'PASSWORD': '???',
'HOST': 'localhost',
'PORT': '5432',
},
}
REDIS_URL = "redis://localhost:6379/8"
MEDIA_ROOT = "/var/lib/pulp/media"
STATIC_ROOT = "/var/lib/pulp/assets"
STATIC_URL = "/pulp/assets/"
FILE_UPLOAD_TEMP_DIR = "/var/lib/pulp/tmp"
WORKING_DIRECTORY = "/var/lib/pulp/tmp"
REMOTE_USER_ENVIRON_NAME = 'HTTP_REMOTE_USER'
AUTHENTICATION_BACKENDS = ['pulpcore.app.authentication.PulpNoCreateRemoteUserBackend']
REST_FRAMEWORK__DEFAULT_AUTHENTICATION_CLASSES = (
'rest_framework.authentication.SessionAuthentication',
'pulpcore.app.authentication.PulpRemoteUserAuthentication'
)
ALLOWED_IMPORT_PATHS = ["/var/lib/pulp/sync_imports", "/var/lib/pulp/imports"]
ALLOWED_EXPORT_PATHS = ["/var/lib/pulp/exports"]
ALLOWED_CONTENT_CHECKSUMS = ["md5", "sha1", "sha224", "sha256", "sha384", "sha512"]
# Derive HTTP/HTTPS via the X-Forwarded-Proto header set by Apache
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
CACHE_ENABLED = True
HIDE_GUARDED_DISTRIBUTIONS = True
# IMPORT_WORKERS_PERCENT = 100
LOGGING = {
"dynaconf_merge": True,
"loggers": {
'': {
'handlers': ['console'],
'level': 'INFO',
},
'pulpcore.deprecation': {
'handlers': ['console'],
'level': 'ERROR',
'propagate': False,
},
'django_guid': {
'handlers': ['console'],
'level': 'WARNING',
'propagate': False,
},
},
}
# ansible plugin settings
ANSIBLE_API_HOSTNAME = "https://or.master.dev.atix"
ANSIBLE_CONTENT_HOSTNAME = "https://or.master.dev.atix/pulp/content"
ANSIBLE_PERMISSION_CLASSES = []
# container plugin settings
TOKEN_AUTH_DISABLED=True
# deb plugin settings
FORCE_IGNORE_MISSING_PACKAGE_INDICES = True
Update: Looks like I switched to the pulp
user, but my PULP_SETTINGS env var was not taken along for the ride. The following worked:
# sudo -u pulp PULP_SETTINGS=/etc/pulp/settings.py pulpcore-manager add-signing-service --class deb:AptReleaseSigningService test_signing_service /opt/orcharhino/apt_signing_service.sh A342C740A62527814A0F491AE345CD7D3ECF7C13
Successfully added signing service test_signing_service for key A342C740A62527814A0F491AE345CD7D3ECF7C13.
Conclusion: The error appears to be a consequence of Pulp not reading the /etc/pulp/settings.py
file.
@JeanFrancois38 Can you try running the following variation of your command?:
PULP_SETTINGS=/etc/pulp/settings.py pulpcore-manager add-signing-service --class deb:AptReleaseSigningService ubuntu my_key_0B93…0900
Don't use export ...;
just run the above as a single command.
It works !!!
Thanks
Version pulp : 3.28.19 katello : 4.10.0
Describe the bug CONTENT_ORIGIN is set in my /etc/pulp/settings.py Then, as pulp I run : export PULP_SETTINGS=“/etc/pulp/settings.py”; pulpcore-manager add-signing-service --class deb:AptReleaseSigningService ubuntu _my_key0B93…0900
Expected behavior Be able to add the sign ability to my pulp deb personnal repo.
Additional context Traceback (most recent call last): File “/usr/bin/pulpcore-manager”, line 33, in sys.exit(load_entry_point(‘pulpcore==3.28.19’, ‘console_scripts’, ‘pulpcore-manager’)()) File “/usr/lib/python3.9/site-packages/pulpcore/app/manage.py”, line 11, in manage execute_from_command_line(sys.argv) File “/usr/lib/python3.9/site-packages/django/core/management/init.py”, line 442, in execute_from_command_line utility.execute() File “/usr/lib/python3.9/site-packages/django/core/management/init.py”, line 382, in execute settings.INSTALLED_APPS File “/usr/lib/python3.9/site-packages/django/conf/init.py”, line 102, in getattr self._setup(name) File “/usr/lib/python3.9/site-packages/django/conf/init.py”, line 89, in _setup self._wrapped = Settings(settings_module) File “/usr/lib/python3.9/site-packages/django/conf/init.py”, line 217, in init mod = importlib.import_module(self.SETTINGS_MODULE) File “/usr/lib64/python3.9/importlib/init.py”, line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File “”, line 1030, in _gcd_import File “”, line 1007, in _find_and_load File “”, line 986, in _find_and_load_unlocked File “”, line 680, in _load_unlocked File “”, line 850, in exec_module File “”, line 228, in _call_with_frames_removed File “/usr/lib/python3.9/site-packages/pulpcore/app/settings.py”, line 381, in settings = DjangoDynaconf( File “/usr/lib/python3.9/site-packages/dynaconf/contrib/django_dynaconf_v2.py”, line 84, in load lazy_settings.populate_obj(django_settings_module) File “/usr/lib/python3.9/site-packages/dynaconf/base.py”, line 115, in getattr self._setup() File “/usr/lib/python3.9/site-packages/dynaconf/base.py”, line 174, in _setup self._wrapped = Settings( File “/usr/lib/python3.9/site-packages/dynaconf/base.py”, line 253, in init self.execute_loaders() File “/usr/lib/python3.9/site-packages/dynaconf/base.py”, line 1025, in execute_loaders self.pre_load(env, silent=silent, key=key) File “/usr/lib/python3.9/site-packages/dynaconf/base.py”, line 1044, in pre_load self.load_file(path=preloads, env=env, silent=silent, key=key) File “/usr/lib/python3.9/site-packages/dynaconf/base.py”, line 1071, in load_file if py_loader.try_to_load_from_py_module_name( File “/usr/lib/python3.9/site-packages/dynaconf/loaders/py_loader.py”, line 68, in try_to_load_from_py_module_name mod = importlib.import_module(str(name)) File “/usr/lib64/python3.9/importlib/init.py”, line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File “”, line 1030, in _gcd_import File “”, line 1007, in _find_and_load File “”, line 986, in _find_and_load_unlocked File “”, line 680, in _load_unlocked File “”, line 850, in exec_module File “”, line 228, in _call_with_frames_removed File “/usr/lib/python3.9/site-packages/pulp_ansible/app/settings.py”, line 24, in ANSIBLE_CONTENT_HOSTNAME = settings.CONTENT_ORIGIN + “/pulp/content” File “/usr/lib/python3.9/site-packages/dynaconf/base.py”, line 115, in getattr self._setup() File “/usr/lib/python3.9/site-packages/dynaconf/base.py”, line 174, in _setup self._wrapped = Settings( File “/usr/lib/python3.9/site-packages/dynaconf/base.py”, line 256, in init self.validators.validate( File “/usr/lib/python3.9/site-packages/dynaconf/validator.py”, line 467, in validate validator.validate( File “/usr/lib/python3.9/site-packages/dynaconf/validator.py”, line 213, in validate self._validate_items( File “/usr/lib/python3.9/site-packages/dynaconf/validator.py”, line 274, in _validate_items raise ValidationError(_message, details=[(self, _message)]) dynaconf.validator.ValidationError: CONTENT_ORIGIN is a required setting but it was not configured. This may be caused by invalid read permissions of the settings file. Note that CONTENT_ORIGIN is set by the installer automatically.
Signing a file works : ./sign_deb_release.sh testfile { "signatures": { "inline": "/tmp/tmp.Nw0QtLbD0h/InRelease", "detached": "/tmp/tmp.Nw0QtLbD0h/Release.gpg" } }
Thanks for your help.
JF