search

pgcentralfoundation / pgrx

Build Postgres Extensions with Rust!
Other
3.55k stars 237 forks source link

Postgres 17 has subtly different dependencies, which can cause `cargo pgrx init` to fail #1828

Open bpsbits-org opened 2 weeks ago

bpsbits-org commented 2 weeks ago

The v17beta3 causes cargo pgrx init to fail.

If trying to run on any system cargo pgrx init, it fails. It happens in phase of installing v17beta3.

If to run cargo pgrx init --pg16 download then init completes.

#cargo-pgrx --version
cargo 0.12.1

Could there be for init an option to exlude beta versions? For example?

cargo pgrx init --no-beta
workingjubilee commented 2 weeks ago

Did it segfault or what? What does "fail" actually mean?

workingjubilee commented 2 weeks ago

Please write down the errors you encountered, otherwise nothing will come of this.

workingjubilee commented 2 weeks ago

I ran cargo pgrx init and it worked fine. Not reproducible (yet?). Deleting the pgrx dir doesn't seem to matter.

bpsbits-org commented 2 weeks ago

I ran cargo pgrx init and it worked fine. Not reproducible (yet?). Deleting the pgrx dir doesn't seem to matter.

Test

#cargo --version
cargo 1.80.1 (376290515 2024-07-16)
cargo pgrx init --pg17 download

macOS Sonoma 14.6.1:

   Discovered Postgres v12.20, v13.16, v14.13, v15.8, v16.4, v17beta3
  Downloading Postgres v17beta3 from https://ftp.postgresql.org/pub/source/v17beta3/postgresql-17beta3.tar.bz2
    Untarring Postgres v17beta3 to /Users/dev/.pgrx/17beta3_unpack
     Renaming /Users/dev/.pgrx/17beta3_unpack/postgresql-17beta3 -> /Users/dev/.pgrx/17beta3
  Configuring Postgres v17beta3
Error: 
   0: cd "/Users/dev/.pgrx/17beta3" && env -u DEBUG -u DYLD_FALLBACK_LIBRARY_PATH -u HOST -u LIBRARY_PATH -u MAKEFLAGS -u MAKELEVEL -u MFLAGS -u NUM_JOBS -u OPT_LEVEL -u OUT_DIR -u PROFILE -u TARGET CPPFLAGS=" -DMEMORY_CONTEXT_CHECKING=1 -DCLOBBER_FREED_MEMORY=1 -DRANDOMIZE_ALLOCATED_MEMORY=1 " PATH="/Users/dev/.pgrx/17beta3:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/opt/podman/bin:/Users/dev/.cargo/bin:/Users/dev/Library/Application Support/JetBrains/Toolbox/scripts" "/Users/dev/.pgrx/17beta3/configure" "--prefix=/Users/dev/.pgrx/17beta3/pgrx-install" "--with-pgport=28817" "--enable-debug" "--enable-cassert"
      checking build system type... aarch64-apple-darwin23.6.0
      checking host system type... aarch64-apple-darwin23.6.0
      checking which template to use... darwin
      checking whether NLS is wanted... no
      checking for default port number... 28817
      checking for block size... 8kB
      checking for segment size... 1GB
      checking for WAL block size... 8kB
      checking for gcc... gcc
      checking whether the C compiler works... yes
      checking for C compiler default output file name... a.out
      checking for suffix of executables... 
      checking whether we are cross compiling... no
      checking for suffix of object files... o
      checking whether we are using the GNU C compiler... yes
      checking whether gcc accepts -g... yes
      checking for gcc option to accept ISO C89... none needed
      checking for gcc option to accept ISO C99... none needed
      checking for g++... g++
      checking whether we are using the GNU C++ compiler... yes
      checking whether g++ accepts -g... yes
      checking for gawk... no
      checking for mawk... no
      checking for nawk... no
      checking for awk... awk
      checking whether gcc supports -Wdeclaration-after-statement, for CFLAGS... yes
      checking whether gcc supports -Werror=vla, for CFLAGS... yes
      checking whether gcc supports -Werror=unguarded-availability-new, for CFLAGS... yes
      checking whether g++ supports -Werror=unguarded-availability-new, for CXXFLAGS... yes
      checking whether gcc supports -Wendif-labels, for CFLAGS... yes
      checking whether g++ supports -Wendif-labels, for CXXFLAGS... yes
      checking whether gcc supports -Wmissing-format-attribute, for CFLAGS... yes
      checking whether g++ supports -Wmissing-format-attribute, for CXXFLAGS... yes
      checking whether gcc supports -Wimplicit-fallthrough=3, for CFLAGS... no
      checking whether g++ supports -Wimplicit-fallthrough=3, for CXXFLAGS... no
      checking whether gcc supports -Wcast-function-type, for CFLAGS... yes
      checking whether g++ supports -Wcast-function-type, for CXXFLAGS... yes
      checking whether gcc supports -Wshadow=compatible-local, for CFLAGS... no
      checking whether g++ supports -Wshadow=compatible-local, for CXXFLAGS... no
      checking whether gcc supports -Wformat-security, for CFLAGS... yes
      checking whether g++ supports -Wformat-security, for CXXFLAGS... yes
      checking whether gcc supports -fno-strict-aliasing, for CFLAGS... yes
      checking whether g++ supports -fno-strict-aliasing, for CXXFLAGS... yes
      checking whether gcc supports -fwrapv, for CFLAGS... yes
      checking whether g++ supports -fwrapv, for CXXFLAGS... yes
      checking whether gcc supports -fexcess-precision=standard, for CFLAGS... no
      checking whether g++ supports -fexcess-precision=standard, for CXXFLAGS... no
      checking whether gcc supports -funroll-loops, for CFLAGS_UNROLL_LOOPS... yes
      checking whether gcc supports -ftree-vectorize, for CFLAGS_VECTORIZE... yes
      checking whether gcc supports -Wunused-command-line-argument, for NOT_THE_CFLAGS... yes
      checking whether gcc supports -Wcompound-token-split-by-macro, for NOT_THE_CFLAGS... yes
      checking whether gcc supports -Wformat-truncation, for NOT_THE_CFLAGS... no
      checking whether gcc supports -Wstringop-truncation, for NOT_THE_CFLAGS... no
      checking whether gcc supports -Wcast-function-type-strict, for NOT_THE_CFLAGS... no
      checking whether gcc supports -fvisibility=hidden, for CFLAGS_SL_MODULE... yes
      checking whether g++ supports -fvisibility=hidden, for CXXFLAGS_SL_MODULE... yes
      checking whether g++ supports -fvisibility-inlines-hidden, for CXXFLAGS_SL_MODULE... yes
      checking whether the C compiler still works... yes
      checking how to run the C preprocessor... gcc -E
      checking for pkg-config... /opt/homebrew/bin/pkg-config
      checking pkg-config is at least version 0.9.0... yes
      checking whether to build with ICU support... yes
      checking for icu-uc icu-i18n... no
      configure: error: ICU library not found
      If you have ICU already installed, see config.log for details on the
      failure.  It is possible the compiler isn't looking in the proper directory.
      Use --without-icu to disable ICU support.

Location:
   /Users/dev/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cargo-pgrx-0.12.1/src/command/init.rs:458

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

   0: cargo_pgrx::command::init::download_postgres with pg_version=17beta3
      at /Users/dev/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cargo-pgrx-0.12.1/src/command/init.rs:237
   1: cargo_pgrx::command::init::init_pgrx
      at /Users/dev/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cargo-pgrx-0.12.1/src/command/init.rs:163
   2: cargo_pgrx::command::init::execute
      at /Users/dev/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cargo-pgrx-0.12.1/src/command/init.rs:88

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

Fedora 40 6.8.11-300.fc40.x86_64:

   Discovered Postgres v12.20, v13.16, v14.13, v15.8, v16.4, v17beta3
  Downloading Postgres v17beta3 from https://ftp.postgresql.org/pub/source/v17beta3/postgresql-17beta3.tar.bz2
     Removing /home/developer/.pgrx/17beta3_unpack
    Untarring Postgres v17beta3 to /home/developer/.pgrx/17beta3_unpack
     Removing /home/developer/.pgrx/17beta3
     Renaming /home/developer/.pgrx/17beta3_unpack/postgresql-17beta3 -> /home/developer/.pgrx/17beta3
  Configuring Postgres v17beta3
Error: 
   0: cd "/home/developer/.pgrx/17beta3" && env -u DEBUG -u DYLD_FALLBACK_LIBRARY_PATH -u HOST -u LIBRARY_PATH -u MAKEFLAGS -u MAKELEVEL -u MFLAGS -u NUM_JOBS -u OPT_LEVEL -u OUT_DIR -u PROFILE -u TARGET CPPFLAGS=" -DMEMORY_CONTEXT_CHECKING=1 -DCLOBBER_FREED_MEMORY=1 -DRANDOMIZE_ALLOCATED_MEMORY=1 " PATH="/home/developer/.pgrx/17beta3:/home/developer/.nvm/versions/node/v22.5.1/bin:/home/developer/.cargo/bin:/home/developer/.local/bin:/home/developer/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin:/home/developer/.local/share/JetBrains/Toolbox/scripts" "/home/developer/.pgrx/17beta3/configure" "--prefix=/home/developer/.pgrx/17beta3/pgrx-install" "--with-pgport=28817" "--enable-debug" "--enable-cassert"
      checking build system type... x86_64-pc-linux-gnu
      checking host system type... x86_64-pc-linux-gnu
      checking which template to use... linux
      checking whether NLS is wanted... no
      checking for default port number... 28817
      checking for block size... 8kB
      checking for segment size... 1GB
      checking for WAL block size... 8kB
      checking for gcc... gcc
      checking whether the C compiler works... yes
      checking for C compiler default output file name... a.out
      checking for suffix of executables... 
      checking whether we are cross compiling... no
      checking for suffix of object files... o
      checking whether we are using the GNU C compiler... yes
      checking whether gcc accepts -g... yes
      checking for gcc option to accept ISO C89... none needed
      checking for gcc option to accept ISO C99... none needed
      checking for g++... g++
      checking whether we are using the GNU C++ compiler... yes
      checking whether g++ accepts -g... yes
      checking for gawk... gawk
      checking whether gcc supports -Wdeclaration-after-statement, for CFLAGS... yes
      checking whether gcc supports -Werror=vla, for CFLAGS... yes
      checking whether gcc supports -Werror=unguarded-availability-new, for CFLAGS... no
      checking whether g++ supports -Werror=unguarded-availability-new, for CXXFLAGS... no
      checking whether gcc supports -Wendif-labels, for CFLAGS... yes
      checking whether g++ supports -Wendif-labels, for CXXFLAGS... yes
      checking whether gcc supports -Wmissing-format-attribute, for CFLAGS... yes
      checking whether g++ supports -Wmissing-format-attribute, for CXXFLAGS... yes
      checking whether gcc supports -Wimplicit-fallthrough=3, for CFLAGS... yes
      checking whether g++ supports -Wimplicit-fallthrough=3, for CXXFLAGS... yes
      checking whether gcc supports -Wcast-function-type, for CFLAGS... yes
      checking whether g++ supports -Wcast-function-type, for CXXFLAGS... yes
      checking whether gcc supports -Wshadow=compatible-local, for CFLAGS... yes
      checking whether g++ supports -Wshadow=compatible-local, for CXXFLAGS... yes
      checking whether gcc supports -Wformat-security, for CFLAGS... yes
      checking whether g++ supports -Wformat-security, for CXXFLAGS... yes
      checking whether gcc supports -fno-strict-aliasing, for CFLAGS... yes
      checking whether g++ supports -fno-strict-aliasing, for CXXFLAGS... yes
      checking whether gcc supports -fwrapv, for CFLAGS... yes
      checking whether g++ supports -fwrapv, for CXXFLAGS... yes
      checking whether gcc supports -fexcess-precision=standard, for CFLAGS... yes
      checking whether g++ supports -fexcess-precision=standard, for CXXFLAGS... yes
      checking whether gcc supports -funroll-loops, for CFLAGS_UNROLL_LOOPS... yes
      checking whether gcc supports -ftree-vectorize, for CFLAGS_VECTORIZE... yes
      checking whether gcc supports -Wunused-command-line-argument, for NOT_THE_CFLAGS... no
      checking whether gcc supports -Wcompound-token-split-by-macro, for NOT_THE_CFLAGS... no
      checking whether gcc supports -Wformat-truncation, for NOT_THE_CFLAGS... yes
      checking whether gcc supports -Wstringop-truncation, for NOT_THE_CFLAGS... yes
      checking whether gcc supports -Wcast-function-type-strict, for NOT_THE_CFLAGS... no
      checking whether gcc supports -fvisibility=hidden, for CFLAGS_SL_MODULE... yes
      checking whether g++ supports -fvisibility=hidden, for CXXFLAGS_SL_MODULE... yes
      checking whether g++ supports -fvisibility-inlines-hidden, for CXXFLAGS_SL_MODULE... yes
      checking whether the C compiler still works... yes
      checking how to run the C preprocessor... gcc -E
      checking for pkg-config... /usr/bin/pkg-config
      checking pkg-config is at least version 0.9.0... yes
      checking whether to build with ICU support... yes
      checking for icu-uc icu-i18n... yes
      checking whether to build with Tcl... no
      checking whether to build Perl modules... no
      checking whether to build Python modules... no
      checking whether to build with GSSAPI support... no
      checking whether to build with PAM support... no
      checking whether to build with BSD Authentication support... no
      checking whether to build with LDAP support... no
      checking whether to build with Bonjour support... no
      checking whether to build with SELinux support... no
      checking whether to build with systemd support... no
      checking whether to build with XML support... no
      checking whether to build with LZ4 support... no
      checking whether to build with ZSTD support... no
      checking for strip... strip
      checking whether it is possible to strip libraries... yes
      checking for ar... ar
      checking for a BSD-compatible install... /usr/bin/install -c
      checking for tar... /usr/bin/tar
      checking whether ln -s works... yes
      checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
      checking for bison... no
      configure: error: bison not found

Location:
   /home/developer/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cargo-pgrx-0.12.1/src/command/init.rs:458

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

   0: cargo_pgrx::command::init::download_postgres with pg_version=17beta3
      at /home/developer/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cargo-pgrx-0.12.1/src/command/init.rs:237
   1: cargo_pgrx::command::init::init_pgrx
      at /home/developer/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cargo-pgrx-0.12.1/src/command/init.rs:163
   2: cargo_pgrx::command::init::execute
      at /home/developer/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cargo-pgrx-0.12.1/src/command/init.rs:88

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
eeeebbbbrrrr commented 2 weeks ago

configure: error: bison not found

Ahh, you need to install bison. Whatever that package is for Fedora should work. I believe the postgres website or wiki documents how to do this. You may also need "flex" -- that could very well be the next upcoming error after this one.

eeeebbbbrrrr commented 2 weeks ago

configure: error: ICU library not found

Similarly for macos you'll need to brew install icu4c. I believe that's the proper package there.

bpsbits-org commented 2 weeks ago

configure: error: ICU library not found

Similarly for macos you'll need to brew install icu4c. I believe that's the proper package there.

dnf install flex & dnf install bison solved issue on Fedora.

icu4c was installed actually on macOS

Please update the documentation about new dependencies.

eeeebbbbrrrr commented 2 weeks ago

configure: error: ICU library not found

Similarly for macos you'll need to brew install icu4c. I believe that's the proper package there.

brew install icu4c does not install flex and bison

Correct. It installs the ICU packages, which will solve the error you reported for MacOS. If there are further errors on MacOS concerning flex/bison, the process to resolve those will be similar.

Please update the documentation about new dependencies.

Could we talk you into submitting a PR once you've got them figured out?

Clearly this isn't something we realized ourselves -- which is easy to escape the notice of the people that work on pgrx every day as we have all sorts of dependencies and packages installed locally.

bpsbits-org commented 2 weeks ago

An update.

dnf install flex & dnf install bison solves issue on Fedora.

But on macOS the icu4c is installed but this does not solve the issue. pg17 install fails regardless of icu4c.

# brew ls --versions icu4c    
icu4c 74.2
eeeebbbbrrrr commented 2 weeks ago

Looks like on my Mac box I had to add this to my shell's login script:

export PKG_CONFIG_PATH="/opt/homebrew/opt/icu4c/lib/pkgconfig"

Been awhile since I've done this, but I imagine brew install icu4c probably told me I'd likely need that -- it's not something I'd have realized on my own!

bpsbits-org commented 2 weeks ago

Looks like on my Mac box I had to add this to my shell's login script:

export PKG_CONFIG_PATH="/opt/homebrew/opt/icu4c/lib/pkgconfig"

Been awhile since I've done this, but I imagine brew install icu4c probably told me I'd likely need that -- it's not something I'd have realized on my own!

Thank you!

export PKG_CONFIG_PATH="/opt/homebrew/opt/icu4c/lib/pkgconfig" && cargo pgrx init --pg17 download

Seemed to work.

Probably some additional notes are needed in the documentation for macOS users.

@eeeebbbbrrrr , thank you for helping!

eeeebbbbrrrr commented 2 weeks ago

Probably some additional notes are needed in the documentation for macOS users.

If you’d like to PR those we’d appreciate it. Get your name added to the elite list of pgrx contributors and help your fellow humans! ;)

bpsbits-org commented 2 weeks ago

Probably some additional notes are needed in the documentation for macOS users.

If you’d like to PR those we’d appreciate it. Get your name added to the elite list of pgrx contributors and help your fellow humans! ;)

Ok, I will try :)

workingjubilee commented 2 weeks ago

Thank you so much for posting an error message! So much easier to diagnose problems in.

We may want to instill cargo pgrx itself with an awareness of the build system requirements and have it pre-check in a way likely to match what cargo pgrx init will actually require, then nudge the user to run an install command if we detect a recognizable package manager. I wouldn't have it be exhaustive, though, so it might still miss a thing, just "have you fulfilled the basic needs we're aware of?"