search

eeshugerman / postgres-backup-s3

A handy Docker container to periodically backup PostgreSQL to S3
MIT License
485 stars 167 forks source link

Add support for PostgreSQL 16 #39

Closed ledermann closed 9 months ago

ledermann commented 1 year ago

PostgreSQL 16 was released on 2023-09-14: https://www.postgresql.org/about/news/postgresql-16-released-2715/

PostgreSQL 11 has reached end-of-life by 2023-11-09: https://www.postgresql.org/support/versioning/

This PR enhances the build matrix (adding v16, removing v11), simplifies installing aws-cli (by using apk) and updates the README.

IgorZvezdilin commented 1 year ago

Hi guys! I believe that you approve this commit!

mederic-p commented 1 year ago

Hey, can we merge this?

kije commented 1 year ago

Also eagerly awaiting this :)

@eeshugerman any chance this Pr could get merged and the image published? 🚀

eeshugerman commented 12 months ago

Thanks @ledermann for the PR! Has anyone tested this out? Best I can tell Postgres v16 is not available in Alpine v3.18.

ledermann commented 12 months ago

@eeshugerman Yes, it seems this PR is a bit premature - support for PostgreSQL 16 has not landed in Alpine 3.18:

$ docker run -it --rm alpine /bin/sh

/ # apk update
fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/main/aarch64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/community/aarch64/APKINDEX.tar.gz
v3.18.4-82-gbad06ca2153 [https://dl-cdn.alpinelinux.org/alpine/v3.18/main]
v3.18.4-80-g1df2bca78e9 [https://dl-cdn.alpinelinux.org/alpine/v3.18/community]
OK: 19946 distinct packages available

/ # apk add postgresql-client
(1/8) Installing postgresql-common (1.2-r0)
Executing postgresql-common-1.2-r0.pre-install
(2/8) Installing lz4-libs (1.9.4-r4)
(3/8) Installing libpq (15.4-r0)
(4/8) Installing ncurses-terminfo-base (6.4_p20230506-r0)
(5/8) Installing libncursesw (6.4_p20230506-r0)
(6/8) Installing readline (8.2.1-r1)
(7/8) Installing zstd-libs (1.5.5-r4)
(8/8) Installing postgresql15-client (15.4-r0)
Executing busybox-1.36.1-r2.trigger
Executing postgresql-common-1.2-r0.trigger
* Setting postgresql15 as the default version
OK: 13 MiB in 23 packages

/ # apk add postgresql16-client
ERROR: unable to select packages:
  postgresql16-client (no such package):
    required by: world[postgresql16-client]

Currently, postgresql16-client is available in Edge only: https://pkgs.alpinelinux.org/packages?name=postgresql16-client&branch=edge&repo=&arch=&maintainer=

I guess we'll have to wait a little longer.

ledermann commented 10 months ago

In the meantime, Alpine 3.19 has been released with support for PostgreSQL 16, so this PR should be ready to merge.

eeshugerman commented 10 months ago

Any volunteers to test this?

Ardakilic commented 10 months ago

@eeshugerman I can try if that would help. I can try if a test version is tagged. I make a daily backup on a database on my docker compose and since v16 it has been stopped.

eeshugerman commented 10 months ago

Thanks @Ardakilic; I found some free time so went ahead and tested myself.

There is a minor snag:

#7 5.888 + pip3 install awscli
#7 7.026 error: externally-managed-environment
#7 7.026 
#7 7.026 × This environment is externally managed
#7 7.026 ╰─> 
#7 7.026     The system-wide python installation should be maintained using the system
#7 7.026     package manager (apk) only.
#7 7.026     
#7 7.026     If the package in question is not packaged already (and hence installable via
#7 7.026     "apk add py3-somepackage"), please consider installing it inside a virtual
#7 7.026     environment, e.g.:
#7 7.026     
#7 7.026     python3 -m venv /path/to/venv
#7 7.026     . /path/to/venv/bin/activate
#7 7.026     pip install mypackage
#7 7.026     
#7 7.026     To exit the virtual environment, run:
#7 7.026     
#7 7.026     deactivate
#7 7.026     
#7 7.026     The virtual environment is not deleted, and can be re-entered by re-sourcing
#7 7.026     the activate file.
#7 7.026     
#7 7.026     To automatically manage virtual environments, consider using pipx (from the
#7 7.026     pipx package).
#7 7.026 
#7 7.026 note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
#7 7.026 hint: See PEP 668 for the detailed specification.
#7 ERROR: process "/bin/sh -c sh install.sh && rm install.sh" did not complete successfully: exit code: 1

We can address this by switching from pip3 install awscli to apk add aws-cli in install.sh. This doesn't work on Alpine v3.10, which we use for Postgres v11, but that's fine -- we can drop Postgres v11 support.

Ardakilic commented 10 months ago

Since Postgres 11 users can still use the old tag, and v11 is EOL already, I'd personally go for it.

muratoflu commented 10 months ago

Hey folks, can we merge this =) ?

Ardakilic commented 10 months ago

I believe this line should be removed:

https://github.com/eeshugerman/postgres-backup-s3/pull/39/files#diff-aa526adb0a3d690b8bce5b9ae75da3c3a254ebeffe4b108851b60d8746356276R14

And these two lines should be modified as @eeshugerman mentioned:

https://github.com/eeshugerman/postgres-backup-s3/blob/aca41c2af4a3633557af29a2a5d2a988f6211972/src/install.sh#L15-L16C19

@ledermann could you check please?

ledermann commented 10 months ago

Sorry, folks, for not testing my own PR. I have made the changes as requested. Now, the Docker image builds fine for Alpine from 3.12 to 3.19.

eeshugerman commented 9 months ago

Looks good, thanks @ledermann!