ClickHouse production configuration and testing

[bmtcril]

There are a number of production use cases and configurations that we need to test and fix where broken. This issue is to track all of those, and they can be broken out into separate issues if the size of the work warrants it. Right now this is just collecting things in one place.

• [ ] Test with SSL
• [x] Test in K8s, including ClickHouse container running in a different namespace than other services
• [x] Pin image and package versions
• [x] Run a security pass on packages versions, image, and configuration

[Ian2012]

We should update this line in the init. The "superset" is hardcoded, that's intended for use with docker-compose but it's incompatible if the service is outside the running instance

https://github.com/openedx/tutor-contrib-oars/blob/b46e88c3c9111dae76949ecd96edb2113884d052/tutoroars/templates/oars/apps/pythonpath/superset-dashboard.py#L13

[Ian2012]

The following tasks were performed:

• Test in k8s
• Pin image version and packages.

[mariajgrimaldi]

These are the checks we ran on this repo (security-wise):

Packages up-to-date: each package must be in its latest security release.

• Pypi packages: we're covered by the requirements bot, but just for the packages found in the requirements directory which are the only ones defined.
• Images: why aren't we using the latest tag instead of 23.3? https://github.com/openedx/tutor-contrib-clickhouse/blob/main/tutorclickhouse/plugin.py#L25
• Github actions: we planned on adding dependabot to update Github actions automatically like these PRs shows.

Configurations: no hard-coded secrets/passwords. Done.

Scan for vulnerabilities in the implementation: security measures are implemented. So we checked each file manually:

• Should running clickhouse with a secure connection be false by default? https://github.com/openedx/tutor-contrib-clickhouse/blob/main/tutorclickhouse/plugin.py#L52

[Ian2012]

@mariajgrimaldi as clickhouse is a database we would prefer to use the latest LTS version (as with MySQL) and as updates are needed, provide and easy migration path from one to another LTS version.

[bmtcril]

I think ideally we would want to run secure by default, but it's a non-trivial configuration to set up: https://clickhouse.com/docs/en/guides/sre/configuring-ssl

It's definitely a worthy task, though, so: https://github.com/openedx/tutor-contrib-clickhouse/issues/20

[Ian2012]

@bmtcril can we close this one and take the conversation in the other issue (as this is not blocking v1)?