petere / homebrew-postgresql

🐘 PostgreSQL formulae for the Homebrew package manager
296 stars 34 forks source link

postgresql@12.rb: Detects/cannot link against system Python #62

Closed al closed 3 years ago

al commented 3 years ago

Running brew install petere/postgresql/postgresql@12 on a new MBP, M1 Pro chip, running Monterey, I'm getting the following:

ld: cannot link directly with dylib/framework, your binary is not an allowed client of /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk/System/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.tbd for architecture arm64

The system Python is being detected and used for includes/libraries, and it seems unsuitable.

For what it's worth I've installed Python 2.7 via Pyenv and Python 3 via Homebrew. The configuration appears correct, but it's evidently not being detected during the build.

~ echo $PATH
#=> /Users/alan/.rbenv/shims:/Users/alan/.pyenv/shims:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin:/opt/homebrew/opt/fzf/bin

~ which python
#=> /Users/alan/.pyenv/shims/python

~ python --version
#=> Python 2.7.18

Any suggestions?

Excerpt from output:

~ brew install petere/postgresql/postgresql@12 --verbose
==> Downloading https://ftp.postgresql.org/pub/source/v12.8/postgresql-12.8.tar.bz2
Already downloaded: /Users/alan/Library/Caches/Homebrew/downloads/5323b6d7ce5556b4bdd026584b36ffbaf42c5457b208ce2914521a420429c605--postgresql-12.8.tar.bz2
==> Verifying checksum for '5323b6d7ce5556b4bdd026584b36ffbaf42c5457b208ce2914521a420429c605--postgresql-12.8.tar.bz2'
==> Installing postgresql@12 from petere/postgresql
tar --extract --no-same-owner --file /Users/alan/Library/Caches/Homebrew/downloads/5323b6d7ce5556b4bdd026584b36ffbaf42c5457b208ce2914521a420429c605--postgresql-12.8.tar.bz2 --directory /private/tmp/d20211031-25133-1x78ap
cp -pR /private/tmp/d20211031-25133-1x78ap/postgresql-12.8/. /private/tmp/postgresqlA12-20211031-25133-15wfto6/postgresql-12.8
chmod -Rf +w /private/tmp/d20211031-25133-1x78ap
==> ./configure --prefix=/opt/homebrew/Cellar/postgresql@12/12.8 --enable-dtrace --enable-nls --with-bonjour --with-gssapi --with-icu --with-ldap --with-libxml --with-libxslt --with-openssl --with-uuid=e2fs --with-pam --with-perl --with-python --with-tcl XML2_CONFIG=: --with-includes=/opt/homebrew/opt/gettext/include:/opt/homebrew/opt/icu4c/include:/opt/homebrew/opt/openldap/include:/opt/homebrew/opt/openssl@1.1/include:/opt/homebrew/opt/readline/include:/opt/homebrew/opt/tcl-tk/include --with-libraries=/opt/homebrew/opt/gettext/lib:/opt/homebrew/opt/icu4c/lib:/opt/homebrew/opt/openldap/lib:/opt/homebrew/opt/openssl@1.1/lib:/opt/homebrew/opt/readline/lib:/opt/homebrew/opt/tcl-tk/lib --with-extra-version= (Homebrew petere/postgresql)

...

checking for python... /usr/bin/python
configure: using python 2.7.18 (default, Oct  2 2021, 04:20:38)
checking for Python distutils module... yes
checking Python configuration directory... /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config
checking Python include directories... -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7
checking how to link an embedded Python application... -L/System/Library/Frameworks/Python.framework/Versions/2.7/lib -lpython2.7 -ldl  -framework CoreFoundation

...

clang -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -O2  -bundle -multiply_defined suppress -o plperl.so plperl.o SPI.o Util.o  -L../../../src/port -L../../../src/common  -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.0.sdk   -L/opt/homebrew/opt/gettext/lib -L/opt/homebrew/opt/icu4c/lib -L/opt/homebrew/opt/openldap/lib -L/opt/homebrew/opt/openssl@1.1/lib -L/opt/homebrew/opt/readline/lib -L/opt/homebrew/opt/tcl-tk/lib -Wl,-dead_strip_dylibs   -fstack-protector-strong  -L/System/Library/Perl/5.30/darwin-thread-multi-2level/CORE -lperl -bundle_loader ../../../src/backend/postgres
ld: cannot link directly with dylib/framework, your binary is not an allowed client of /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk/System/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.tbd for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [plpython2.so] Error 1
make[2]: *** [install-plpython-recurse] Error 2
make[2]: *** Waiting for unfinished jobs....
clang -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -O2  -bundle -multiply_defined suppress -o plpgsql.so pl_gram.o pl_handler.o pl_comp.o pl_exec.o pl_funcs.o pl_scanner.o  -L../../../../src/port -L../../../../src/common  -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.0.sdk   -L/opt/homebrew/opt/gettext/lib -L/opt/homebrew/opt/icu4c/lib -L/opt/homebrew/opt/openldap/lib -L/opt/homebrew/opt/openssl@1.1/lib -L/opt/homebrew/opt/readline/lib -L/opt/homebrew/opt/tcl-tk/lib -Wl,-dead_strip_dylibs   -lintl -bundle_loader ../../../../src/backend/postgres
/usr/bin/install -c -m 755  plperl.so '/opt/homebrew/Cellar/postgresql@12/12.8/lib/plperl.so'
/usr/bin/install -c -m 755  plpgsql.so '/opt/homebrew/Cellar/postgresql@12/12.8/lib/plpgsql.so'
make[1]: *** [install-pl-recurse] Error 2
make: *** [install-world-src-recurse] Error 2

==> Formula
Tap: petere/postgresql
Path: /opt/homebrew/Library/Taps/petere/homebrew-postgresql/postgresql@12.rb
==> Configuration
HOMEBREW_VERSION: 3.3.2
ORIGIN: https://github.com/Homebrew/brew
HEAD: 9daf5e2d28396c7a4379605e89e029d9fccb41e6
Last commit: 15 hours ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 63c932f6434e9194154927bac6e8e4fb53e59f16
Core tap last commit: 21 minutes ago
Core tap branch: master
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_CORE_GIT_REMOTE: https://github.com/Homebrew/homebrew-core
HOMEBREW_EDITOR: mate
HOMEBREW_MAKE_JOBS: 10
Homebrew Ruby: 2.6.8 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: 10-core 64-bit arm_firestorm_icestorm
Clang: 13.0.0 build 1300
Git: 2.33.1 => /opt/homebrew/bin/git
Curl: 7.77.0 => /usr/bin/curl
macOS: 12.0.1-arm64
CLT: 13.1.0.0.1.1633545042
Xcode: 13.1
Rosetta 2: false
==> ENV
HOMEBREW_CC: clang
HOMEBREW_CXX: clang++
MAKEFLAGS: -j10
CMAKE_PREFIX_PATH: /opt/homebrew/opt/icu4c:/opt/homebrew/opt/openssl@1.1:/opt/homebrew/opt/openldap:/opt/homebrew/opt/readline:/opt/homebrew/opt/tcl-tk:/opt/homebrew
CMAKE_INCLUDE_PATH: /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk/System/Library/Frameworks/OpenGL.framework/Versions/Current/Headers
CMAKE_LIBRARY_PATH: /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk/System/Library/Frameworks/OpenGL.framework/Versions/Current/Libraries
PKG_CONFIG_PATH: /opt/homebrew/opt/icu4c/lib/pkgconfig:/opt/homebrew/opt/openssl@1.1/lib/pkgconfig:/opt/homebrew/opt/openldap/lib/pkgconfig:/opt/homebrew/opt/readline/lib/pkgconfig:/opt/homebrew/opt/tcl-tk/lib/pkgconfig
PKG_CONFIG_LIBDIR: /usr/lib/pkgconfig:/opt/homebrew/Library/Homebrew/os/mac/pkgconfig/12
HOMEBREW_GIT: git
HOMEBREW_SDKROOT: /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk
ACLOCAL_PATH: /opt/homebrew/share/aclocal
PATH: /opt/homebrew/Library/Homebrew/shims/mac/super:/opt/homebrew/opt/pkg-config/bin:/opt/homebrew/opt/gettext/bin:/opt/homebrew/opt/icu4c/bin:/opt/homebrew/opt/openssl@1.1/bin:/opt/homebrew/opt/openldap/bin:/opt/homebrew/opt/tcl-tk/bin:/usr/bin:/bin:/usr/sbin:/sbin

Error: petere/postgresql/postgresql@12 12.8 did not build
Logs:
     /Users/alan/Library/Logs/Homebrew/postgresql@12/00.options.out
     /Users/alan/Library/Logs/Homebrew/postgresql@12/01.configure.cc
     /Users/alan/Library/Logs/Homebrew/postgresql@12/02.make.cc
     /Users/alan/Library/Logs/Homebrew/postgresql@12/config.log
     /Users/alan/Library/Logs/Homebrew/postgresql@12/01.configure
     /Users/alan/Library/Logs/Homebrew/postgresql@12/02.make
Do not report this issue to Homebrew/brew or Homebrew/core!

These open issues may also help:
postgresql 14 fails to start after upgrading from postgresql 13 https://github.com/petere/homebrew-postgresql/issues/61
While installing postgresql@11: configure: error: header file <perl.h> is required for Perl https://github.com/petere/homebrew-postgresql/issues/44
petere commented 3 years ago

Yeah, I just noticed that happening, also outside of brew. I'm not sure yet what the cause is.

al commented 3 years ago

I don't suppose exposing the --without-python option (in the same way --with-cassert is) would be an acceptable short term workaround?

al commented 3 years ago

Looking inside the seemingly problematic text-based dylib file:

~ head -n30 /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk/System/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.tbd
--- !tapi-tbd
tbd-version:     4
targets:         [ x86_64-macos, arm64-macos, arm64e-macos ]
uuids:
  - target:          x86_64-macos
    value:           EA4FEBF4-8D7C-3D3D-BBCB-B084B95BFBF5
  - target:          arm64-macos
    value:           00000000-0000-0000-0000-000000000000
  - target:          arm64e-macos
    value:           4C205BA2-1934-3B13-AB55-9DBA891C1B13
install-name:    '/System/Library/Frameworks/Python.framework/Versions/2.7/Python'
current-version: 2.7.16
compatibility-version: 2.7
allowable-clients:
  - targets:         [ x86_64-macos, arm64-macos, arm64e-macos ]
    clients:         [ Accounts, CPAN, CoreAnalytics_executables, CoreGraphicsBindings,
                       CoreMLTools, CoreNLPFramework, CreateMLTest, CreateML_tests,
                       DataDetectorsCore_executables, Feldspar, GPUDriversIntel,
                       HIDSensingPipeline, HSComposer, LanguageModeling, Mica, MobileBluetooth_DevTools,
                       MobileQuickLook_iosmac, PencilKit_swiftoverlay_iosmac, Photos_Swift,
                       Picholine, Raft, RaftFramework, RaftFrameworkExternal, SpeechRecognitionCore_executables,
                       Stocks, TimeSync_exec, UIKit_swiftoverlay, accounts_tool,
                       appletrace_bindings-python2, ausiss, caffeconverter, cgdump,
                       coremlpython, disasm, duallinkanalyse, eoslib, fastsim, gxtools,
                       lldbPluginScriptInterpreterPython2, lldb_host, mrecfpu, pdsasm,
                       pmrec, pmrec.tmp, py_dasm, pydis, pydis-s2a8, pyobjc, python.exe,
                       python_modules, pythonw, pytrace, quarantine_test, rogueasm,
                       sim, sim_bindings-python2, testbotctl, testbotd, testbotd-base,
                       testbottask, timesyncanalyse, timesynclogextract ]

Again the error message is:

clang -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -O2  -bundle -multiply_defined suppress -o plpython2.so plpy_cursorobject.o plpy_elog.o plpy_exec.o plpy_main.o plpy_planobject.o plpy_plpymodule.o plpy_procedure.o plpy_resultobject.o plpy_spi.o plpy_subxactobject.o plpy_typeio.o plpy_util.o  -L../../../src/port -L../../../src/common  -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.0.sdk   -L/opt/homebrew/opt/gettext/lib -L/opt/homebrew/opt/icu4c/lib -L/opt/homebrew/opt/openldap/lib -L/opt/homebrew/opt/openssl@1.1/lib -L/opt/homebrew/opt/readline/lib -L/opt/homebrew/opt/tcl-tk/lib -Wl,-dead_strip_dylibs   -L/System/Library/Frameworks/Python.framework/Versions/2.7/lib -lpython2.7 -ldl  -framework CoreFoundation -lintl -bundle_loader ../../../src/backend/postgres
/bin/sh ../../../../config/install-sh -c -d '/opt/homebrew/Cellar/postgresql@12/12.8/share/extension'
/bin/sh ../../../../config/install-sh -c -d '/opt/homebrew/Cellar/postgresql@12/12.8/include/server'
/usr/bin/install -c -m 644 ./plperl.h ./ppport.h ./plperl_helpers.h '/opt/homebrew/Cellar/postgresql@12/12.8/include/server'
/usr/bin/install -c -m 644 ./plpgsql.control ./plpgsql--1.0.sql ./plpgsql--unpackaged--1.0.sql '/opt/homebrew/Cellar/postgresql@12/12.8/share/extension/'
/usr/bin/install -c -m 644 './plpgsql.h' '/opt/homebrew/Cellar/postgresql@12/12.8/include/server'

ld: cannot link directly with dylib/framework, your binary is not an allowed client of /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk/System/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.tbd for architecture arm64

clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [plpython2.so] Error 1
make[2]: *** [install-plpython-recurse] Error 2
make[2]: *** Waiting for unfinished jobs....

So am I right in saying we're failing to build plpython2 because it's not explicitly listed as an allowable client of the Python dylib?

Looking on an older machine (MacOSX11.1.sdk) the corresponding file has no allowable-clients section.

petere commented 3 years ago

Yeah, it looks like Apple changed it so that you just can't build against the system Python installation anymore.

The way forward is probably to change everything to build against brewed Python.

al commented 3 years ago

I tried altering the formula to make it use a brewed version of Python but had no joy, so I offer two awful workarounds for anyone else stuck on this...

  1. Hack the TBD File

This is a terrible and probably dangerous idea, but open /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk/System/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.tbd in your favourite text editor and add plpython2, ltree_plpython, jsonb_plpython2, hstore_plpython2 to the clients array under allowable-clients.

or

  1. Hack the Formula and Install from Local Repo

Fork and clone this repo then symlink it to your Taps directory, e.g.:

~ ln -s /path/to/homebrew-postgresql `brew --repo`/Library/Taps/al/homebrew-postgresql

Verify with brew tap:

~ brew tap
al/postgresql
homebrew/cask
homebrew/core

Edit the formula so that it skips Python:

diff --git a/postgresql@12.rb b/postgresql@12.rb
index b74cc77..1f4ef01 100644
--- a/postgresql@12.rb
+++ b/postgresql@12.rb
@@ -50,7 +50,7 @@ class PostgresqlAT12 < Formula
       --with-uuid=e2fs
       --with-pam
       --with-perl
-      --with-python
+      --without-python
       --with-tcl
       XML2_CONFIG=:
     ]

Install as normal, e.g.:

~ brew install al/postgresql/postgresql-common -v
~ brew install al/postgresql/postgresql@12 -v
al commented 3 years ago

👍