Use configuration providers instead of `envsubst` in Kafka configuration files

[scholzj]

Type of change

• Enhancement / new feature

Description

This PR changes how we handle various variables in the Kafka broker (or controller) configuration files. Instead of using environment variables and replacing them with envsubst, it moves to use configuration providers to have them loaded inside Kafka.

This helps us to:

• Minimize our dependency on bash scripts
• • Improves the test coverage as the bash scripts in the container images have no unit tests (they are of course run as part of system tests)
• Reduces our footprint in the container images (right now, this probably has little value, but the smaller footprint we have in the container image, the easier to support more different images)

The way this is implemented is that we initialize two different providers:

• The file provider (used only in broker nodes)
• The environment variable provider
• (We also respect and keep any providers configured by the user - although that might not be usual in Kafka brokers/controllers)

And we use these providers to read data from files or environment variables. The PKCS12 passwords and the OAuth secrets are passed through environment variables and rendered suing the environment variable provider. the rack ID and worker node address (used for node port listeners) are passed as a file and loaded through the file config provider. For this, the layout of the files generated by the init container was slightly modified,

Checklist

• [x] Write tests
• [x] Make sure all tests pass
• [x] Try your changes from Pod inside your Kubernetes and OpenShift cluster, not just locally

[scholzj]

/azp run regression

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).

[scholzj]

/azp run upgrade

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).

[scholzj]

/azp run regression

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).

[scholzj]

/azp run regression

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).

[scholzj]

/azp run upgrade

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).

[scholzj]

/azp run regression

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).

[scholzj]

/azp run zookeeper-regression

[scholzj]

/azp run migration

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).

[scholzj]

/azp run upgrade

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).