argoproj / argo-workflows

Workflow Engine for Kubernetes
https://argo-workflows.readthedocs.io/
Apache License 2.0
15.11k stars 3.21k forks source link

build: add `make` targets to dump the database #13771

Closed MasonM closed 1 month ago

MasonM commented 1 month ago

Motivation

When working on the DB locally, it's sometimes necessary to make backups, particularly when testing schema migrations (e.g. https://github.com/argoproj/argo-workflows/issues/13601).

This is similar to https://github.com/argoproj/argo-workflows/pull/13715, and I thought about combining the two PRs, but I figured it's better to keep them separate, since they don't conflict with each other.

Modifications

This adds two Make targets, make mysql-dump and make postgres-dump, to run mysqldump and pg_dump, respectively. I also simplified the existing make mysql-cli and make postgres-cli commands to use kubectl exec svc/... instead of manually grabbing the pod, since passing a service to kubectl exec will cause it to automatically select the first (and only) pod in the service.

Verification

Tested with both make start PROFILE=postgres and make start PROFILE=mysql.

Results for MySQL ```console $ make mysql-dump GIT_COMMIT=c9f0376594aed5ae0ebc53d3bf2970300406860f GIT_BRANCH=feat-cli-db-generator2 GIT_TAG=untagged GIT_TREE_STATE=dirty RELEASE_TAG=false DEV_BRANCH=true VERSION=latest KUBECTX=k3d-k3s-default DOCKER_DESKTOP=false K3D=true DOCKER_PUSH=false TARGET_PLATFORM=linux/amd64 RUN_MODE=local PROFILE=minimal AUTH_MODE=hybrid SECURE=false STATIC_FILES=false ALWAYS_OFFLOAD_NODE_STATUS=false UPPERIO_DB_DEBUG=0 LOG_LEVEL=debug NAMESPACED=true kubectl exec svc/mysql -- mysqldump --no-tablespaces -u mysql -ppassword argo > "db-dumps/mysql-2024-10-16T17:23:43Z.sql" mysqldump: [Warning] Using a password on the command line interface can be insecure. $ make mysql-cli < db-dumps/mysql-2024-10-16T17\:23\:43Z.sql GIT_COMMIT=c9f0376594aed5ae0ebc53d3bf2970300406860f GIT_BRANCH=feat-cli-db-generator2 GIT_TAG=untagged GIT_TREE_STATE=dirty RELEASE_TAG=false DEV_BRANCH=true VERSION=latest KUBECTX=k3d-k3s-default DOCKER_DESKTOP=false K3D=true DOCKER_PUSH=false TARGET_PLATFORM=linux/amd64 RUN_MODE=local PROFILE=minimal AUTH_MODE=hybrid SECURE=false STATIC_FILES=false ALWAYS_OFFLOAD_NODE_STATUS=false UPPERIO_DB_DEBUG=0 LOG_LEVEL=debug NAMESPACED=true kubectl exec -ti svc/mysql -- mysql -u mysql -ppassword argo Unable to use a TTY - input is not a terminal or the right kind of file mysql: [Warning] Using a password on the command line interface can be insecure. ```
Results for PostgreSQL ``` $ make postgres-dump GIT_COMMIT=c9f0376594aed5ae0ebc53d3bf2970300406860f GIT_BRANCH=feat-cli-db-generator2 GIT_TAG=untagged GIT_TREE_STATE=dirty RELEASE_TAG=false DEV_BRANCH=true VERSION=latest KUBECTX=k3d-k3s-default DOCKER_DESKTOP=false K3D=true DOCKER_PUSH=false TARGET_PLATFORM=linux/amd64 RUN_MODE=local PROFILE=minimal AUTH_MODE=hybrid SECURE=false STATIC_FILES=false ALWAYS_OFFLOAD_NODE_STATUS=false UPPERIO_DB_DEBUG=0 LOG_LEVEL=debug NAMESPACED=true kubectl exec svc/postgres -- pg_dump -U postgres > "db-dumps/postgres-2024-10-16T17:26:56Z.sql" $ make postgres-cli < db-dumps/postgres-2024-10-16T17\:28\:02Z.sql GIT_COMMIT=d05cf6459cdd956cb621a6552f009e21db0a3789 GIT_BRANCH=main GIT_TAG=untagged GIT_TREE_STATE=dirty RELEASE_TAG=false DEV_BRANCH=false VERSION=latest KUBECTX=k3d-k3s-default DOCKER_DESKTOP=false K3D=true DOCKER_PUSH=false TARGET_PLATFORM=linux/amd64 RUN_MODE=local PROFILE=minimal AUTH_MODE=hybrid SECURE=false STATIC_FILES=true ALWAYS_OFFLOAD_NODE_STATUS=false UPPERIO_DB_DEBUG=0 LOG_LEVEL=debug NAMESPACED=true kubectl exec -ti svc/postgres -- psql -U postgres Unable to use a TTY - input is not a terminal or the right kind of file SET SET SET SET SET set_config ------------ (1 row) SET SET SET SET ALTER TABLE DROP INDEX DROP INDEX DROP INDEX DROP INDEX DROP INDEX DROP INDEX ALTER TABLE ALTER TABLE ALTER TABLE DROP TABLE DROP TABLE DROP TABLE DROP TABLE SET SET CREATE TABLE ALTER TABLE CREATE TABLE ALTER TABLE CREATE TABLE ALTER TABLE CREATE TABLE ALTER TABLE COPY 0 COPY 0 COPY 0 COPY 2 ALTER TABLE ALTER TABLE ALTER TABLE CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX CREATE INDEX ALTER TABLE ```