Add support for integrated MariaDB and PostgreSQL

[djjudas21]

This PR adds basic support for deploying MariaDB or PostgreSQL as part of an Authelia deployment.

Typically the dependency subcharts would be integrated under the top level mariadb: or postgresql: keys, where the config can be inherited by the subcharts, and the main app (i.e. Authelia) would also draw its config from those keys. However the database config in the chart is under storage.mysql and storage.postgres and changing this would be a breaking change.

So I have added two booleans storage.mysql.deploy and storage.postgres.deploy which enable the user to deploy the Bitnami subcharts. This does not affect any existing Authelia deployments. At the moment the user has to copy a few settings (e.g. credentials) from the mariadb section to the storage.mysql section.

In future we could consider a tighter integration, but it would be a breaking change, so let's go with this for now.

Happy to discuss or tweak this, including adding a Redis subchart.

Relevant to #172 and #89 - but won't actually close them until Redis support is added.

Summary by CodeRabbit

• New Features
  • Introduced deployment options for MariaDB and PostgreSQL instances within the application.

[coderabbitai[bot]]

[!IMPORTANT]

Auto Review Skipped

Auto reviews are disabled on base/target branches other than the default branch. Please add the base/target branch pattern to the list of additional branches to be reviewed in the settings.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository.

To trigger a single review, invoke the @coderabbitai review command.

Walkthrough

This update introduces the functionality to deploy MariaDB and PostgreSQL instances directly via configuration options in Authelia's Helm chart. This enhancement simplifies the process of setting up databases for Authelia, making it more convenient for users to configure their instances according to their preferences and requirements.

Changes

Files	Change Summary
charts/authelia/README.md	Added configuration options for deploying MariaDB and PostgreSQL instances.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

- [X](https://twitter.com/intent/tweet?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A&url=https%3A//coderabbit.ai) - [Mastodon](https://mastodon.social/share?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A%20https%3A%2F%2Fcoderabbit.ai) - [Reddit](https://www.reddit.com/submit?title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&text=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code.%20Check%20it%20out%3A%20https%3A//coderabbit.ai) - [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcoderabbit.ai&mini=true&title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&summary=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code)

---

Tips

### Chat There are 3 ways to chat with CodeRabbit: - Review comments: Directly reply to a review comment made by CodeRabbit. Example: - `I pushed a fix in commit .` - `Generate unit testing code for this file.` - `Open a follow-up GitHub issue for this discussion.` - Files and specific lines of code (under the "Files changed" tab): Tag `@coderabbitai` in a new review comment at the desired location with your query. Examples: - `@coderabbitai generate unit testing code for this file.` - `@coderabbitai modularize this function.` - PR comments: Tag `@coderabbitai` in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples: - `@coderabbitai generate interesting stats about this repository and render them as a table.` - `@coderabbitai show all the console.log statements in this repository.` - `@coderabbitai read src/utils.ts and generate unit testing code.` - `@coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.` Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. ### CodeRabbit Commands (invoked as PR comments) - `@coderabbitai pause` to pause the reviews on a PR. - `@coderabbitai resume` to resume the paused reviews. - `@coderabbitai review` to trigger a review. This is useful when automatic reviews are disabled for the repository. - `@coderabbitai resolve` resolve all the CodeRabbit review comments. - `@coderabbitai help` to get help. Additionally, you can add `@coderabbitai ignore` anywhere in the PR description to prevent this PR from being reviewed. ### CodeRabbit Configration File (`.coderabbit.yaml`) - You can programmatically configure CodeRabbit by adding a `.coderabbit.yaml` file to the root of your repository. - The JSON schema for the configuration file is available [here](https://coderabbit.ai/integrations/coderabbit-overrides.v2.json). - If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: `# yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json` ### CodeRabbit Discord Community Join our [Discord Community](https://discord.com/invite/GsXnASn26c) to get help, request features, and share feedback.

[djjudas21]

Great, thanks. As Redis is also a dependency I plan to follow this up with another near-identical PR to pull in the Redis helm chart, which means users will then be able to deploy a full working Authelia stack from one helm chart. I'll get on that a bit later today.

[james-d-elliott]

Nice! Thanks a lot, I've been stretched thin.

[james-d-elliott]

Looks like this included a bug making the chart unable to be packaged. I've reverted it for now in bbbd05d75c0cd1b2cec0e4d3bd669488d805f7e2. There's also this comment which I've failed to decode: https://github.com/authelia/chartrepo/discussions/220#discussioncomment-9083712

We'd welcome these changes in another PR. You should be able to just git revert bbbd05d75c0cd1b2cec0e4d3bd669488d805f7e2 && git checkout origin/master -- .buildkite/pipeline.yaml && git commit -C HEAD --amend to get back to the previous state with PostgreSQL, MariaDB, and Redis.

[djjudas21]

OK, I'll have a look at this and see if I can track down what went wrong.

I saw https://github.com/authelia/chartrepo/discussions/220#discussioncomment-9083712 too and also didn't understand it. I guess it might be to do with what I mentioned a little while ago, about having to set some of the values in two places to make the new integrations work. If you're not concerned about breaking changes, I'll try and integrate those more smoothly.

[james-d-elliott]

OK, I'll have a look at this and see if I can track down what went wrong.

I saw #220 (comment) too and also didn't understand it. I guess it might be to do with what I mentioned a little while ago, about having to set some of the values in two places to make the new integrations work. If you're not concerned about breaking changes, I'll try and integrate those more smoothly.

We're in v0 (every minor bump should be expected as breaking), and once we make v1 I'll be fairly happy to bump major versions whenever necessary. The maintenance cycle of the chart will likely never perfectly marry-up to Authelia itself and we have much more freedom.

Only real concern is deploying these dependencies by default should not be the case, it should be explicit. Otherwise as long as the breaking change is documented and makes sense (I've done my best to document the other ones, if you notice any missed ones I'd appreciate a heads up or for those to be fixed too).

[djjudas21]

@james-d-elliott I've just got the bandwidth to look at this again.

Looks like this included a bug making the chart unable to be packaged.

You got a link to the actual problem? I couldn't see any failed runs in Github Actions - thanks.

[james-d-elliott]

https://buildkite.com/authelia/charts/builds/529#018ef5e1-c9e9-4073-89f1-4f4cf9bb8ca2

[djjudas21]

OK, I've figured it out. When deps are packaged with a Helm chart, there is a helm dep lock or helm dep update step that resolves the dependencies, updates a lockfile and packages the subcharts too. However in your .gitignore, all of these artifacts are ignored (which is quite common)

# .gitignore

# Helm
charts/*/charts
charts/*/Chart.lock

So the usual way of handling this is for your CI/CD pipeline to fetch these deps when the chart is packaged. You will need to execute helm repo add bitnami https://charts.bitnami.com/bitnami in your pipeline immediately before any of the cr steps, i.e. just before here: https://github.com/authelia/chartrepo/blob/master/.buildkite/pipeline.yaml#L19

Then the subcharts will be included in the package you build and publish, but not your git repo.

[james-d-elliott]

Same issue running that command before the cr pakage. See here:

• https://buildkite.com/authelia/charts/builds/546#018f271b-2e6e-4427-ba1b-da9d23985ef0
• https://github.com/authelia/chartrepo/pull/232

[djjudas21]

OK. Can you try explicitly running helm dep lock after adding the Bitnami repo?

[james-d-elliott]

That command doesn't exist, looks like it should be helm dependency build but that has the same issues.. also I can reliably replicate this locally with helm 3.14.3 (without using chart-releaser).

[james-d-elliott]

helm repo add bitnami https://charts.bitnami.com/bitnami
helm dependency build charts/authelia
helm package charts/authelia

Error: found in Chart.yaml, but missing in charts/ directory: postgresql, mariadb, redis

[djjudas21]

Interesting, I'm also on Helm 3.14.3 and it works for me:

[jonathan@poseidon authelia-chartrepo]$ helm dependency build charts/authelia
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "immich" chart repository
...Successfully got an update from the "beluga-cloud" chart repository
...Unable to get an update from the "openebs" chart repository (https://openebs.github.io/charts):
    failed to fetch https://openebs.github.io/charts/index.yaml : 404 Not Found
...Successfully got an update from the "rook-release" chart repository
...Successfully got an update from the "node-feature-discovery" chart repository
...Successfully got an update from the "portainer" chart repository
...Successfully got an update from the "eugen" chart repository
...Successfully got an update from the "authelia" chart repository
...Successfully got an update from the "vikunja" chart repository
...Successfully got an update from the "jetstack" chart repository
...Successfully got an update from the "democratic-csi" chart repository
...Successfully got an update from the "harbor" chart repository
...Successfully got an update from the "kongz" chart repository
...Successfully got an update from the "vmware-tanzu" chart repository
...Successfully got an update from the "djjudas21" chart repository
...Successfully got an update from the "sosivio" chart repository
...Successfully got an update from the "kokuwa" chart repository
...Successfully got an update from the "camerahub" chart repository
...Successfully got an update from the "actions-runner-controller" chart repository
...Successfully got an update from the "k8s-home-lab" chart repository
...Successfully got an update from the "cloudhippie" chart repository
...Successfully got an update from the "dex" chart repository
...Successfully got an update from the "fairwinds-stable" chart repository
...Successfully got an update from the "runix" chart repository
...Successfully got an update from the "percona" chart repository
...Successfully got an update from the "deliveryhero" chart repository
...Successfully got an update from the "intel" chart repository
...Successfully got an update from the "oauth2-proxy" chart repository
...Successfully got an update from the "nextcloud" chart repository
...Successfully got an update from the "influxdata" chart repository
...Successfully got an update from the "node-exporter-textfiles" chart repository
...Successfully got an update from the "adfinis" chart repository
...Successfully got an update from the "mattermost" chart repository
...Successfully got an update from the "kasten" chart repository
...Successfully got an update from the "nicholaswilde" chart repository
...Successfully got an update from the "ananace-charts" chart repository
...Successfully got an update from the "secureCodeBox" chart repository
...Successfully got an update from the "ntppool" chart repository
...Successfully got an update from the "gabe565" chart repository
...Successfully got an update from the "grafana" chart repository
...Successfully got an update from the "prometheus-community" chart repository
...Successfully got an update from the "bitnami" chart repository
...Successfully got an update from the "truecharts" chart repository
Update Complete. ⎈Happy Helming!⎈
Saving 3 charts
Downloading postgresql from repo https://charts.bitnami.com/bitnami
Downloading mariadb from repo https://charts.bitnami.com/bitnami
Downloading redis from repo https://charts.bitnami.com/bitnami
Deleting outdated charts

It is supposed to update all its chart repos when you run helm dependency build, but can you also try running helm repo update bitnami and helm search repo mariadb to make sure it can find the bitnami/mariadb chart?

[james-d-elliott]

Yep, it finds all the charts, and it downloads them fine. What does the packaged chart look like when you extract it?

[djjudas21]

So weird, I'm running into the same error too:

[jonathan@latitude authelia]$ pwd
/home/jonathan/git/authelia-chartrepo/charts/authelia

[jonathan@latitude authelia]$ helm dependency update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "awx-operator" chart repository
...Successfully got an update from the "tx-smtp-relay-helm" chart repository
...Successfully got an update from the "node-feature-discovery" chart repository
...Unable to get an update from the "openebs" chart repository (https://openebs.github.io/charts):
    failed to fetch https://openebs.github.io/charts/index.yaml : 404 Not Found
...Successfully got an update from the "immich" chart repository
...Successfully got an update from the "sosivio" chart repository
...Successfully got an update from the "authelia" chart repository
...Successfully got an update from the "democratic-csi" chart repository
...Successfully got an update from the "camerahub" chart repository
...Successfully got an update from the "actions-runner-controller" chart repository
...Successfully got an update from the "portainer" chart repository
...Successfully got an update from the "jetstack" chart repository
...Successfully got an update from the "beluga-cloud" chart repository
...Successfully got an update from the "runix" chart repository
...Successfully got an update from the "intel" chart repository
...Successfully got an update from the "vikunja" chart repository
...Successfully got an update from the "oauth2-proxy" chart repository
...Successfully got an update from the "k8s-home-lab" chart repository
...Successfully got an update from the "vmware-tanzu" chart repository
...Successfully got an update from the "t3n" chart repository
...Successfully got an update from the "deliveryhero" chart repository
...Successfully got an update from the "rook-release" chart repository
...Successfully got an update from the "harbor" chart repository
...Successfully got an update from the "nextcloud" chart repository
...Successfully got an update from the "influxdata" chart repository
...Successfully got an update from the "djjudas21" chart repository
...Successfully got an update from the "mattermost" chart repository
...Successfully got an update from the "kasten" chart repository
...Successfully got an update from the "ananace-charts" chart repository
...Successfully got an update from the "adfinis" chart repository
...Successfully got an update from the "gitlab" chart repository
...Successfully got an update from the "grafana" chart repository
...Successfully got an update from the "secureCodeBox" chart repository
...Successfully got an update from the "k8s-at-home" chart repository
...Successfully got an update from the "gabe565" chart repository
...Successfully got an update from the "fairwinds-stable" chart repository
...Successfully got an update from the "bitnami" chart repository
...Successfully got an update from the "prometheus-community" chart repository
...Successfully got an update from the "truecharts" chart repository
Update Complete. ⎈Happy Helming!⎈
Saving 3 charts
Downloading postgresql from repo https://charts.bitnami.com/bitnami
Downloading redis from repo https://charts.bitnami.com/bitnami
Downloading mariadb from repo https://charts.bitnami.com/bitnami
Deleting outdated charts

[jonathan@latitude authelia]$ ls charts/
mariadb-18.0.2.tgz  postgresql-15.2.5.tgz  redis-19.1.3.tgz

[jonathan@latitude authelia]$ helm package .
Error: found in Chart.yaml, but missing in charts/ directory: postgresql, redis, mariadb

So we can deduce that the problem is not with dependency resolving, but with packaging behaviour. I've had a look in Helm's issue tracker and I can't see anything relevant there. This is definitely broken behaviour so I'm going to report it as a bug in Helm.

[djjudas21]

Reported https://github.com/helm/helm/issues/12992

[james-d-elliott]

it's the *.tgz line in ./charts/authelia/.helmignore

[djjudas21]

it's the *.tgz line in ./charts/authelia/.helmignore

Good catch!

Are you able to re-merge this PR now, or does it need rebasing etc? Thanks

[djjudas21]

@james-d-elliott what do you need me to do to progress this? Which branch is the right one to develop against now, v0.9.0-beta3? I'm happy to rework this PR and #232 against the latest branch. Would be great to get #172 and #225 closed :grinning:

[james-d-elliott]

master is the correct branch, I don't really have time to work on it but would welcome the PR