zonkyio / embedded-postgres-binaries

Lightweight bundles of PostgreSQL binaries with reduced size intended for testing purposes.
Apache License 2.0
130 stars 29 forks source link

upgrade postgres versions to the latest ones, partial fix for #43 #44

Closed t0r0X closed 2 years ago

t0r0X commented 2 years ago

New versions of PostgreSQL were released, with security and bug fixes: https://www.postgresql.org/about/news/postgresql-141-135-129-1114-1019-and-9624-released-2349/ This partially fixes #43.

t0r0X commented 2 years ago

The checks seem to fail because of a expired certificate from nixos.org:

Processing triggers for libc-bin (2.19-0ubuntu6.15) ...
+ wget -O patchelf.tar.gz https://nixos.org/releases/patchelf/patchelf-0.9/patchelf-0.9.tar.gz
--2021-11-25 10:12:53--  https://nixos.org/releases/patchelf/patchelf-0.9/patchelf-0.9.tar.gz
Resolving nixos.org (nixos.org)... 54.205.240.192, 67.207.80.24, 2600:1f18:2489:8202:1542:e5:19c0:d28b, ...
Connecting to nixos.org (nixos.org)|54.205.240.192|:443... connected.
ERROR: cannot verify nixos.org's certificate, issued by '/C=US/O=Let\'s Encrypt/CN=R3':
  Issued certificate has expired.
To connect to nixos.org insecurely, use `--no-check-certificate'.
+ cp '/usr/lib/*/libssl.so.1.0.0' '/usr/lib/*/libcrypto.so.1.0.0' ./lib
cp: cannot stat '/usr/lib/*/libssl.so.1.0.0': No such file or directory
cp: cannot stat '/usr/lib/*/libcrypto.so.1.0.0': No such file or directory

I looked at that site's certificate, and it seems they have refreshed it, please could someone restart the checks? Or, what's the recommended procedure for these cases?

tomix26 commented 2 years ago

I guess the docker image that is used to build the project does not contain the new certificate. That's because the build pipeline rely on older images with older operating systems for backwards compatibility of the resulting binaries. So it will be necessary to either add the --no-check-certificate option to each call of the wget and curl commands, or to incorporate the new certificate to the older images. The first option is a potential security threat, so the second option is preferred. But don't worry I'll look into this during the weekend and try to sort it out.

t0r0X commented 2 years ago

Oh, now I understand the issue. I'd also prefer the second option (add new certificate to older images), better safe than sorry. The concerned image is ubuntu:14.04.

I'm not worried :-) just trying to understand, learn and help, if possible.

t0r0X commented 2 years ago

Beside the ubuntu:14.04 CA issue, there is another issue for PostgreSQL 13.5 on MacOS, in scripts/test-postgres-darwin.sh line 40:

++ psql -qAtX -h localhost -p 65432 -U postgres -d postgres -c 'SET jit_above_cost = 10; SELECT SUM(relpages) FROM pg_class;'
ERROR:  could not load library "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmp.ewEBRd46/pg-test/lib/postgresql/llvmjit.so": dlopen(/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmp.ewEBRd46/pg-test/lib/postgresql/llvmjit.so, 10): Library not loaded: @loader_path/../..//opt/local/lib/libncurses.6.dylib
2021-11-25 10:50:23.236 UTC [5220] ERROR:  could not load library "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmp.ewEBRd46/pg-test/lib/postgresql/llvmjit.so": dlopen(/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmp.ewEBRd46/pg-test/lib/postgresql/llvmjit.so, 10): Library not loaded: @loader_path/../..//opt/local/lib/libncurses.6.dylib
  Referenced from: /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmp.ewEBRd46/pg-test/lib/postgresql/llvmjit.so
  Reason: no suitable image found.  Did find:
    file system relative paths not allowed in hardened programs

PS I was able to reproduce this in my fork (Actions: learned again something...). Looks like #21. What's frustrating is that the Test PostgreSQL bundle workflow step fails only after 6 hours (!) :-( I wonder if this can be accelerated.

t0r0X commented 2 years ago

Sorry for the commit spam, it took me a while to get used to the Github actions and to understand how to run them from a feature branch. I've extensively run/tested the checks in my fork, and the only open issue is with PostgreSQL 13.5 on MacOS: https://github.com/t0r0X/embedded-postgres-binaries/actions/runs/1505818551

tomix26 commented 2 years ago

@t0r0X Thank you very much for your contribution. I really appreciate it. You saved me a lot of time.

t0r0X commented 2 years ago

@tomix26 I'm glad I could help. I was searching for a way to replace H2 with PostgreSQL, for my local development and test when I stumbled upon the embedded-postgres/embedded-database projects. Great stuff, thanks! And I learned a lot!