Better way of passing deploy time config values to Scala Web Service

[t83714]

Better way of passing deploy time config values to Scala Web Service

We currently pass deploy time config values to Scale Web Service via JVM system properties (set via command line parameters).

e.g. a final deployed k8s pod manifest (generated by helm chart) might look like the following:

spec:
  containers:
  - command:
    - bin/magda-registry-api
    - -Dhttp.port=6101
    - -Dhttp.externalUrl.v0=https://minikube.data.gov.au/api/v0/registry
    - -Ddb.default.url=jdbc:postgresql://registry-db/postgres
    - -Dakka.loglevel=INFO
    - -DauthApi.baseUrl=http://authorization-api
    - -Dscalikejdbc.global.loggingSQLAndTime.logLevel=info
    - -Dauthorization.skip=false
    - -Dauthorization.skipOpaQuery=false
    - -Drole=full
    - -DvalidateJsonSchema=false
    - -Dakka.http.server.request-timeout=60s
    - -Dakka.http.server.idle-timeout=60s
    - -Dscalikejdbc.global.loggingSQLAndTime.enabled=false

Problems with the current approach

• limitation of the command line argument size
• hard to encode to pass a complex data structure
  • The internal HOCON config file format we use is actually a superset of JSON that can support all JSON data structure
• needs to modify helm chart to add the support of new configuration values

Proposed Solution

• 1> Support a new config field appConfig in helm chart values file.
  • Users are allowed to set JSON value (in yaml format) under it to supply equivalent config values used in application HOCON config file.

e.g. The appConfig (that passes the same config data as the k8s pod manifest example above) might look like this:

appConfig:
  http: 
    port: 6101
    externalUrl:
      v0: https://minikube.data.gov.au/api/v0/registry
  db:
    default:
      url: jdbc:postgresql://registry-db/postgres
  authApi:
    baseUrl: http://authorization-api
  scalikejdbc:
    global:
      loggingSQLAndTime:
        logLevel: info
        enabled: false
  authorization:
    skip: false
    skipOpaQuery: false
  role: full
  validateJsonSchema: false
  akka:
    loglevel: INFO
    http:
      server:
        "request-timeout": "60s"
        "idle-timeout": "60s"

• 2> We then can use mustRegexReplaceAllLiteral & mustToPrettyJson to convert it to HOCON format and store as configMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: "xxxx-srv-application-conf"
data:
  deploy-application.conf: {{ (mustRegexReplaceAllLiteral "^\\s*{" (.Values. appConfig | mustToPrettyJson) "{\n  include \"application\"") | quote }}

The string stored in deploy-application.conf in the configMap above will look like the following:

{
  # load the default application.conf in jar
  include "application"
  "http": {
    "port": 6101,
    "externalUrl": {
      "v0": "https://minikube.data.gov.au/api/v0/registry"
    }
  },
  "db": {
    "default": {
      "url": "jdbc:postgresql://registry-db/postgres"
    }
  },
  "authApi": {
    "baseUrl": "http://authorization-api"
  },
  "scalikejdbc": {
    "global": {
      "loggingSQLAndTime": {
        "logLevel": "info",
        "enabled": false
      }
    }
  },
  "authorization": {
    "skip": false,
    "skipOpaQuery": false
  },
  "role": "full",
  "validateJsonSchema": false,
  "akka": {
    "loglevel": "INFO",
    "http": {
      "server": {
        "request-timeout": "60s",
        "idle-timeout": "60s"
      }
    }
  }
}

• 3> The web service could:
  • map the configMap as /etc/config/deploy-application.conf
  • override the config via system properties: -Dconfig.file=/etc/config/deploy-application.conf

spec:
  containers:
  - command:
    - bin/magda-registry-api
    - -Dconfig.file=/etc/config/deploy-application.conf
    volumeMounts:
    - mountPath: /etc/config
      name: deploy-application-config
  volumes:
  - name: config
    configMap:
      name: xxxx-srv-application-conf

[t83714]

Further technical notes:

• how to ensure backward compatibility? We can construct the config data in the following way:
  • retrieve base config data from .Values and strip non-app-config common fields e.g. images, autoscaler etc
  • map base config data to empty appConfig dict
  • overwrite appConfig dict with .Values.appConfig

[t83714]

closed via PR: https://github.com/magda-io/magda/pull/3468