citusdata / cstore_fdw

Columnar storage extension for Postgres built as a foreign data wrapper. Check out https://github.com/citusdata/citus for a modernized columnar storage implementation built as a table access method.
Apache License 2.0
1.76k stars 172 forks source link

Installing on Mac OS X 10.14: missing headers #207

Closed jankatins closed 4 years ago

jankatins commented 5 years ago

On a up to date Mac OS X Mojave (new install), I needed to add the following fixes, to make it compile:

  1. Install header files via open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg (From https://github.com/neovim/neovim/issues/9050#issuecomment-424417456)
  2. Add PG_CPPFLAGS += -I/usr/include to the makefile (I use the develop_v1x branch, because it missed an assert.h header
mtuncer commented 5 years ago

That's odd I have been using Mojave for some time and did not have this problem.

Following is my build statement when I issue make.

Note : I build PG 11 from sources. You will see that -isysroot parameter is added to compile params. That is coming from pg_config --cppflags

gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -g -O0 -g -gdwarf-2  --std=c99 -I/usr/local/include -I. -I./ -I/opt/pgsql/11/include/server -I/opt/pgsql/11/include/internal  -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk   -I/usr/local/opt/openssl/include  -c -o cstore.pb-c.o cstore.pb-c.c

I configured PG 11 build with

./configure --prefix=/opt/pgsql/11 --enable-cassert --enable-debug CFLAGS="-O0 -g -gdwarf-2" --with-openssl --with-includes=/usr/local/opt/openssl/include --with-libraries=/usr/local/opt/openssl/lib --no-create --no-recursion

Could you check that again try again ?

Thanks

jankatins commented 5 years ago

I didn't build/configured postgresql, I used brew to install it (basically brew install postgresql + brew services start postgresql as a normal user which starts the service as a user).

This is the output of pg_config:

pg_config --cppflags
-I/usr/local/Cellar/icu4c/63.1/include -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk -I/usr/local/opt/openssl/include -I/usr/local/opt/readline/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/libxml2
mtuncer commented 4 years ago

we should be fine after recent fixes

jankatins commented 4 years ago

I still have the problem with a brew installed postgresql (v11) on a up to date mac os x.

λ make
protoc-c --c_out=. cstore.proto
clang -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -O2  --std=c99 -I/usr/local/include -I. -I./ -I/usr/local/opt/postgresql@11/include/server -I/usr/local/opt/postgresql@11/include/internal -I/usr/local/Cellar/icu4c/64.2/include -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -I/usr/local/opt/openssl@1.1/include -I/usr/local/opt/readline/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/libxml2   -c -o cstore.pb-c.o cstore.pb-c.c
clang: warning: no such sysroot directory: '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk' [-Wmissing-sysroot]
In file included from cstore.pb-c.c:9:
In file included from ./cstore.pb-c.h:7:
/usr/local/include/protobuf-c/protobuf-c.h:199:10: fatal error: 'assert.h' file not found
#include <assert.h>
         ^~~~~~~~~~
1 error generated.
make: *** [cstore.pb-c.o] Error 1

The important part seems to be the clang warning:

clang: warning: no such sysroot directory: '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk' [-Wmissing-sysroot]

The problem seems to be that the brew postgresql version is compiled with a full xcode install while I only have some commandline stuff installed. The difference is that my sysroot is /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk while the 'full xcode install' has /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk. Unfortunately this path is hardcoded as PG_SYSROOT in /usr/local/opt/postgresql@11/lib/pgxs/src/Makefile.global which is included from the pgxs makefile . This results in clang not finding any headers (assert.h, stdio.h,...). Using make PG_SYSROOT=$(xcrun --show-sdk-path) fixed that problem. (No idea if brew should fix this or if the pgxs.mk file should "reset" that on mac)

Next problem was that it couldn't find the brew installed protobuf lib.

clang -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -O2  -bundle -multiply_defined suppress -o cstore_fdw.so cstore.pb-c.o cstore_fdw.o cstore_writer.o cstore_reader.o cstore_metadata_serialization.o cstore_compression.o /usr/local/lib/ -L/usr/local/opt/postgresql@11/lib -L/usr/local/opt/openssl@1.1/lib -L/usr/local/opt/readline/lib  -Wl,-dead_strip_dylibs   -lprotobuf-c -bundle_loader /usr/local/Cellar/postgresql@11/11.7/bin/postgres
ld: library not found for -lprotobuf-c
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Using PG_LDFLAGS=-L/usr/local/lib/ fixed that. Not sure why it worked before.

Together:

make PG_SYSROOT=$(xcrun --show-sdk-path) PG_LDFLAGS=-L/usr/local/lib/

This comment and the whole issue brought me to the right track about the first issue: https://github.com/neovim/neovim/issues/9050#issuecomment-424425017