search

timescale / helm-charts

Configuration and Documentation to run TimescaleDB in your Kubernetes cluster
Apache License 2.0
263 stars 223 forks source link

Database initialization #102

Open depeele opened 4 years ago

depeele commented 4 years ago

I'm relatively new to kubernetes and helm but I don't see a way to inject initialization scripts or more generally, initialize a database/tables/indices on install.

In the postgres helm chart, there is an initdbScriptsConfigMap value that may be provided or, you can simply include content directly via files/docker-entrypoint-initdb.d/ (which will be used to create a config map). In either case, the result is that the content of the files are mounted into the postgres container (in /docker-entrypoint-initdb.d/) and used during database initialization.

Is there a similar capability with the timescaledb-kubernetes helm chart?

feikesteenbergen commented 4 years ago

No currently there is nothing user-specified that will run during initialization. I like the suggestion and this is probably something that can be added.

files/docker-entrypoint-initdb.d/

These Helm Charts do not use any of these scripts, as Patroni is bootstrapping (initdb etc) the whole cluster.

You could use callbacks, by creating a ConfigMap that contains a script, which will be called during all Patroni related events. If you create one that works idem-potently, you can have it run on each event - Patroni events should be rare on a stable deployment.

Thinking about it a bit more, it might be best to have Patroni support the on_init callback for those scripts that should only be run once.

invernizzie commented 4 years ago

@feikesteenbergen

Thinking about it a bit more, it might be best to have Patroni support the on_init callback for those scripts that should only be run once.

Definitely helpful. Some DDL statements like role creation are hard to write idempotently.

As for my particular use case, making use of Kubernetes' secrets to set user passwords during initialization provides a level of security that is impossible to accomplish using some schema migration tools—which run after initdb. But to set passwords (which can be done idempotently) first I need to create roles (which isn't idempotent, although it can be worked around: see here).

feikesteenbergen commented 4 years ago

@depeele @invernizzie I would love some initial feedback on this PR:

https://github.com/timescale/timescaledb-kubernetes/pull/191/files

It should be self explaining (if not, the PR needs more documentation).

It cannot be merged until the Patroni issue has been released, but until then, we can at least discuss the interface.

Nana-EC commented 3 years ago

@feikesteenbergen great work on the repo.

Any word on the resolution of this issue? My team is looking to pick up TimescaleDB using this chart but the lack of initialization scripts would make it hard for our adoption. Also would this feature be present in the chart for TimescaleDB 2.0?

Thanks