search

oxidecomputer / omicron

Omicron: Oxide control plane
Mozilla Public License 2.0
252 stars 40 forks source link

Configure a backup destination for ClickHouse #6612

Open bnaecker opened 2 months ago

bnaecker commented 2 months ago

I've often wanted to collect the latest ClickHouse data from Dogfood. I've been doing that by manually taking a ZFS snapshot; creating a tarball; and copying it out to my own files. However, ClickHouse actually supports in-band backups, that take a consistent view of the database or any subset of it. It would be very nice to let ClickHouse do this work for us. That requires that we configure a backup destination, a set of paths that the system is configured to use. That can only be specified in the XML configuration file. I don't see a CLI flag for it. We don't ship that at all for single-node ClickHouse, but we do in the replica configuration XML files. We should set that parameter, to allow backups.

This may also be a good mechanism to collect ClickHouse data for support bundles.

jclulow commented 2 months ago

What format does it use for writing all the backup data out?

bnaecker commented 2 months ago

It looks like it creates a zip, but I’m not sure of the contents. One can choose particular pieces of the database to backup, such as tables or views, so I think it creates some subset of the directory tree it normally operates on. One can also choose a range of compression options, and do incremental backups.

This may not be the best approach. For example a ZFS snapshot may be preferable. But it is built in and convenient, and gives a consistent view of the database state.