search

CrunchyData / crunchy-containers

Containers for Managing PostgreSQL on Kubernetes by Crunchy Data
https://www.crunchydata.com/
Apache License 2.0
1.01k stars 328 forks source link

postgres-upgrade container doesn't include all versions of pgsql binaries #1461

Closed ArgonV closed 2 years ago

ArgonV commented 2 years ago

Howdy all, I am experiencing an issue with the crunchy-upgrade container... I am using this image tag: crunchydata/crunchy-upgrade:centos8-13.6-4.7.5

I am mounting the volume of the original container DB (named pg-folio) at /pgolddata, and the OLD_DATABASE_NAME env var set to pg-folio-0

Using a different PV/PVC, I am mounting it at /pgnewdata, and the NEW_DATABASE_NAME env var set to pg-folio-0

The error I am getting is that, when going from 11 to 12, the 12 binaries cannot be found. Here is the full error:

nss_wrapper: user exists nss_wrapper: group exists nss_wrapper: environment configured INFO: Setting PGBINNEW to 12. INFO: Setting PGBINOLD to 11. INFO: Using the --locale=en_US.UTF-8 flags for initdb.. /opt/crunchy/bin/start.sh: line 141: /usr/pgsql-12/bin/initdb: No such file or directory INFO: Cleaning up the old postmaster.pid file.. rm: cannot remove '/pgolddata/pg-folio-0/postmaster.pid': No such file or directory /opt/crunchy/bin/start.sh: line 157: /usr/pgsql-12/bin/pg_upgrade: No such file or directory ERROR: Problem with upgrade. rc=127

I am assuming these are supposed to be inside the container, when the upgrade is being performed?

Lastly, I'm doing this all in K8s as a Job.

ArgonV commented 2 years ago

I found the issue: None of the new (from 3-ish months ago) crunchy-upgrade tags have all of the versions of the Postgres binaries built-in as they should. They only have the pgsql-13 binary.

I managed to start the container up at the /bin/bash entrypoint without running the start.sh script - and looked under /usr/, only pgsql-13 existed.

I went back to an older tag, centos8-13.1-4.6.0 and it had all three versions I needed (11, 12, 13). However, when it started up it gave me an error along the lines of "NEW_VERSION 12 UNSUPPORTED".

So, I ended up copying out the pgsql-11/12 binary folders onto an NFS mount that I mounted inside the container. Then spun up a newer version of the crunchy-upgrade container whose code worked, but that was missing the binary - at the proper path of /usr/pgsql-11 and /usr/pgsql-12. Now the upgrade is progressing!

ArgonV commented 2 years ago

Upgrade succeeded. Please be sure when the container builds, all of the previous Postgres version binaries are in /usr

andrewlecuyer commented 2 years ago

@ArgonV thank your for the detailed information and feedback!

We will ensure all versions of PostgreSQL are included in any future upgrade container binaries.

Thanks!