Configure django rest applications using environment variables

[annashamray]

Description

Registration APIs (but not limited to these projects) should be configurable without manual actions in the Admin (following the configuration-as-code infrastructure principle). So the configuration using only environment variables is needed for the following APIs:

• Open Zaak (1st prio)
• Open Notificaties (1st prio)
• Objects API (2nd prio)
• Objecttypes API (2nd prio)
• Open Klant (2nd prio)

Tasklist

1st iteration

• create a management command setup_configuration, which would run through all the configuration steps, execute them and (optionally) self-test them. This management command should be run in non-interactive mode. It can be executed in k8s job or init container.
• The command should have an option overwrite, which will replace the existing configuration - making the command viable for first-time setup or to enforce a specific configuration
• the steps and their order should be set up as a django setting in the same fashion as, for example, MIDDLEWARE variable
• each step should have the same shape/signature and be a subclass of the same ConfigurationStep base class
• each API will create its own configurations steps with provided env variables.

2nd iteration

• some env variables can point to yaml files to store complicated configuration for the whole tables (for example, zgw_consumers.Service). The could be used to load data similar to django fixtures, but with a framework-agnostic format. Where relevant, the data in the YAML files should follow the shape/format as the API endpoints that disclose this information. The files can be mounted as files or configmaps (kubernetes) to provide them to the application container(s).
• each relevant library loading the data is responsible for dictating the format, parsing, validating and importing the data
• loading the data needs to happen in an atomic transaction by dropping existing records and re-inserting them, or possibly updating existing records (there is a complexity risk here with breaking FK references). The truncate + load approach is preferred for simplicity sake

Result / acceptance criteria

• a docker compose for Open Zaak and Open Notificaties, which has with init containers with setup_configurationcommand. The APIs should be able to interaction with each other and pass a smoke test

Related issues in other projects

• https://github.com/maykinmedia/mozilla-django-oidc-db/issues/69

[joeribekker]

Example:

export OPEN_ZAAK_DEFAULT_SELECTIELIJST_YEAR=2020
export OPEN_ZAAK_AUTH_APPLICATIONS=/volumes/auth.yaml

$ python src/manage.py setup_configuration --overwrite 

This will set the default selectielijst year to 2020 and will store/overwrite all applications with the permission in the yaml file.