search

kernelci / kernelci-backend

Upstream Linux Kernel Validation Project API
http://api.kernelci.org
GNU Lesser General Public License v2.1
40 stars 27 forks source link

T27888 config rework #287

Closed mgalka closed 2 years ago

mgalka commented 3 years ago

1) Move mongodb database name to configuration

Introduced mongodb_dbname parameter which allows to specify mongodb database name in application configuraion istead of it being harcoded in models.init.py. This commit also refactors database connection handling functions accordingly.

2) Enable config file and unix soccket paths as config options

Configuration file path can be now passed as a CLI option. If the option is not present kernelci-backend will look for the config at /etc/kernelci/kernelci-backend.cfg A Unix socket path can be now defined as a "socket" option. It can be provided as a CLI or config file option. Default value is /tmp/kernelci-backend.socket

3) Fail if master_key is not defined

Prevent application from starting when master_key is not defined either as a CLI option or in the config file. The server doesn't set a random uuid when master_key is not provided any more.

4) Add user configuration file path as celery CLI option

Added -u/--user-config parameter to celery app which can be used to pass alternative configuration file path. Default value remains defined as /etc/kernelci/kernelci-celery.cfg

gctucker commented 3 years ago

It's causing some exceptions on staging:

May 24 09:01:11 kernelci-staging kernelci-celery[49887]:   File "/srv/.venv/api.staging.kernelci.org/local/lib/python2.7/site-packages/pymongo/database.py", line 47, in _check_name
May 24 09:01:11 kernelci-staging kernelci-celery[49887]:     raise InvalidName("database name cannot be the empty string")
May 24 09:01:11 kernelci-staging kernelci-celery[49887]: InvalidName: database name cannot be the empty string
mgalka commented 3 years ago

It's causing some exceptions on staging:

May 24 09:01:11 kernelci-staging kernelci-celery[49887]:   File "/srv/.venv/api.staging.kernelci.org/local/lib/python2.7/site-packages/pymongo/database.py", line 47, in _check_name
May 24 09:01:11 kernelci-staging kernelci-celery[49887]:     raise InvalidName("database name cannot be the empty string")
May 24 09:01:11 kernelci-staging kernelci-celery[49887]: InvalidName: database name cannot be the empty string

It looks that there is an issue that cannot or rather shouldn't be overcome without a modification of a config file or refactoring the way how default values are handled in the celery app. Celery app and the main app (server.py) use different configuration files. The default paths are /etc/kernelci/kernelci-celery.cfg and /etc/kernelci/kernelci-backend.cfg respectively. The issue observed above is caused because mongodb_dbname is not present in the former config file. While the main/server application has default values for all the options for which it makes sense, the Celery application doesn't have any default values defined for its configuration options. I can see two possible solutions here. Either we introduce default values for all configuration options for Celery app or we add mongodb_dbname to the /etc/kernelci/kernelci-celery.cfg config file like it is done at the moment.

gctucker commented 3 years ago

Right, I think we should add mongodb_dbname to the Celery config file. The main issue is that the config is not automatically updated because the Ansible playbook has some technical debt to be cleared, so it has to be edited by hand. It's easy enough to do that, we can merge this once it's been tested with manual config edits and then we can look into cleaning up Ansible or using Docker so the config can be automatically recreated from scratch without losing anything.

mgalka commented 2 years ago

As we're replacing the old backend, I think there is no use for this PR anymore