argoproj / argo-workflows

Workflow Engine for Kubernetes
https://argo-workflows.readthedocs.io/
Apache License 2.0
15.11k stars 3.21k forks source link

docs: additional details on jsonb migration for 3.6 #13816

Closed MasonM closed 3 weeks ago

MasonM commented 4 weeks ago

Motivation

This adds some additional details about the migration introduced in https://github.com/argoproj/argo-workflows/pull/13779, primarily to assist users with a very large number of archived workflows who may not be comfortable with extended downtime.

Modifications

The manual minimal-downtime migration strategy is inspired by Altering a Postgres Column with Minimal Downtime. I tested it locally after populating my local database with 100,000 rows using https://github.com/argoproj/argo-workflows/pull/13715:

$ time make postgres-cli < migrate_before.sql
GIT_COMMIT=f25960ac6f63bc3cf1a3b8a9813cf43c61921262 GIT_BRANCH=fix-migrate-jsonb GIT_TAG=untagged GIT_TREE_STATE=dirty RELEASE_TAG=false DEV_BRANCH=true VERSION=latest
KUBECTX=k3d-k3s-default DOCKER_DESKTOP=false K3D=true DOCKER_PUSH=false TARGET_PLATFORM=linux/amd64
RUN_MODE=local PROFILE=minimal AUTH_MODE=hybrid SECURE=false  STATIC_FILES=false ALWAYS_OFFLOAD_NODE_STATUS=false UPPERIO_DB_DEBUG=0 LOG_LEVEL=debug NAMESPACED=true
kubectl exec -ti svc/postgres -- psql -U postgres
Unable to use a TTY - input is not a terminal or the right kind of file
ALTER TABLE
CREATE FUNCTION
CREATE TRIGGER
UPDATE 100000

real    1m16.118s
user    0m2.639s
sys     0m0.548s

$ time make postgres-cli < migrate_after.sql
GIT_COMMIT=49ff7a44ba307416282a1f5cd3b844d19bce7f88 GIT_BRANCH=main GIT_TAG=untagged GIT_TREE_STATE=dirty RELEASE_TAG=false DEV_BRANCH=false VERSION=latest
KUBECTX=k3d-k3s-default DOCKER_DESKTOP=false K3D=true DOCKER_PUSH=false TARGET_PLATFORM=linux/amd64
RUN_MODE=local PROFILE=minimal AUTH_MODE=hybrid SECURE=false  STATIC_FILES=true ALWAYS_OFFLOAD_NODE_STATUS=false UPPERIO_DB_DEBUG=0 LOG_LEVEL=debug NAMESPACED=true
kubectl exec -ti svc/postgres -- psql -U postgres
Unable to use a TTY - input is not a terminal or the right kind of file
BEGIN
LOCK TABLE
DROP TRIGGER
ALTER TABLE
ALTER TABLE
ALTER TABLE
COMMIT

real    0m1.503s
user    0m2.786s
sys     0m0.540s

Verification

Ran make docs-serve:

image

MasonM commented 3 weeks ago

@terrytangyuan Are you referring to the minimal-downtime strategy? That's only something users that with a very large database that also care about minimizing downtime would consider, and I don't think it makes sense to script it.

I pushed a commit (4e29a3dda4fefa7d898299d88d4836cd4cd4d02e) to make it clear most users don't need to take any action.