search

adjust / parquet_fdw

Parquet foreign data wrapper for PostgreSQL
PostgreSQL License
333 stars 37 forks source link

MAC OS: error Make Install | STDLIB.H Fatal Error #14

Open dfirence opened 4 years ago

dfirence commented 4 years ago

Hello, I want to use your extensions but am unable to get past the #1 step - make install.

I have Mojave 10.14, all XCODE tools are installed

Here's what I get

me@[parquet_fdw]
 >> make install
clang++ -std=c++11 -O3 -I. -I./ -I/usr/local/Cellar/postgresql/11.5_1/include/server -I/usr/local/Cellar/postgresql/11.5_1/include/internal -I/usr/local/Cellar/icu4c/64.2/include -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk -I/usr/local/opt/openssl@1.1/include -I/usr/local/opt/readline/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/libxml2   parquet_impl.cpp  -c -fPIC -L/usr/local/lib  -L/usr/local/opt/openssl@1.1/lib -L/usr/local/opt/readline/lib  -Wl,-dead_strip_dylibs   -lpgcommon -lpgport -lxslt -lxml2 -lpam -lssl -lcrypto -lgssapi_krb5 -lz -lreadline -lm  -o parquet.o
clang: warning: -Wl,-dead_strip_dylibs: 'linker' input unused [-Wunused-command-line-argument]
clang: warning: -lpgcommon: 'linker' input unused [-Wunused-command-line-argument]
clang: warning: -lpgport: 'linker' input unused [-Wunused-command-line-argument]
clang: warning: -lxslt: 'linker' input unused [-Wunused-command-line-argument]
clang: warning: -lxml2: 'linker' input unused [-Wunused-command-line-argument]
clang: warning: -lpam: 'linker' input unused [-Wunused-command-line-argument]
clang: warning: -lssl: 'linker' input unused [-Wunused-command-line-argument]
clang: warning: -lcrypto: 'linker' input unused [-Wunused-command-line-argument]
clang: warning: -lgssapi_krb5: 'linker' input unused [-Wunused-command-line-argument]
clang: warning: -lz: 'linker' input unused [-Wunused-command-line-argument]
clang: warning: -lreadline: 'linker' input unused [-Wunused-command-line-argument]
clang: warning: -lm: 'linker' input unused [-Wunused-command-line-argument]
clang: warning: no such sysroot directory: '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk' [-Wmissing-sysroot]
clang: warning: argument unused during compilation: '-L/usr/local/lib' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-L/usr/local/opt/openssl@1.1/lib' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-L/usr/local/opt/readline/lib' [-Wunused-command-line-argument]
In file included from parquet_impl.cpp:5:
In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/list:173:
In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/memory:653:
In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/typeinfo:61:
In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/exception:82:
In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/cstdlib:86:
/Library/Developer/CommandLineTools/usr/include/c++/v1/stdlib.h:94:15: fatal error: 'stdlib.h' file not found
#include_next <stdlib.h>
              ^~~~~~~~~~
1 error generated.
make: *** [parquet.o] Error 1
zilder commented 4 years ago

Hi @dfirence,

After quick googling i see that there were some changes in Mojave regarding location of the standard headers (see release notes and 'Command Line Tools > New features' section particularly). I also see similar issues on github for other software. The common recommendation i found is to run:

open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

I don't have Mojave so can't really verify the issue and the solution. Please let me know it that helps.

dfirence commented 4 years ago

Hey @zilder

What versions of PgSQL you support. I have a debian box and can try to install there again. Yesterday I was able to compile it in Debian Stretch with PGSQL 11.5.

However in step #2

CREATE EXTENSION ....

Here's the output of Debian 9 Stretch with PGSQL 12

mvtest=# CREATE EXTENSION parquet_fdw;

ERROR:  could not load library "/usr/lib/postgresql/12/lib/parquet_fdw.so": /usr/lib/x86_64-linux-gnu/libarrow.so.15: undefined symbol: _ZN6google8protobuf8internal17AssignDescriptorsERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKNS1_15MigrationSchemaEPKPKNS0_7MessageEPKjPNS0_8MetadataEPPKNS0_14EnumDescriptorEPPKNS0_17ServiceDescriptorE
mvtest=#

It fails by throwing a SYMBOL ERROR Not Found.

dfirence commented 4 years ago

I then installed PGSQL 11.6 on Debian Stretch

Compiled FDW for 11.6


> sudo make install
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -g -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -fno-omit-frame-pointer -fPIC -shared -o parquet_fdw.so parquet.o parquet_fdw.o  -L/usr/lib/x86_64-linux-gnu  -Wl,-z,relro -Wl,-z,now -L/usr/lib/llvm-6.0/lib  -L/usr/lib/x86_64-linux-gnu/mit-krb5 -Wl,--as-needed  -lm -lstdc++ -lparquet -larrow
/bin/mkdir -p '/usr/lib/postgresql/11/lib'
/bin/mkdir -p '/usr/share/postgresql/11/extension'
/bin/mkdir -p '/usr/share/postgresql/11/extension'
/usr/bin/install -c -m 755  parquet_fdw.so '/usr/lib/postgresql/11/lib/parquet_fdw.so'
/usr/bin/install -c -m 644 .//parquet_fdw.control '/usr/share/postgresql/11/extension/'
/usr/bin/install -c -m 644 .//parquet_fdw--0.1.sql  '/usr/share/postgresql/11/extension/'
/bin/mkdir -p '/usr/lib/postgresql/11/lib/bitcode/parquet_fdw'
/bin/mkdir -p '/usr/lib/postgresql/11/lib/bitcode'/parquet_fdw/
/usr/bin/install -c -m 644 parquet.bc '/usr/lib/postgresql/11/lib/bitcode'/parquet_fdw/./
/usr/bin/install -c -m 644 parquet_fdw.bc '/usr/lib/postgresql/11/lib/bitcode'/parquet_fdw/./
cd '/usr/lib/postgresql/11/lib/bitcode' && /usr/lib/llvm-6.0/bin/llvm-lto -thinlto -thinlto-action=thinlink -o parquet_fdw.index.bc parquet_fdw/parquet.bc parquet_fdw/parquet_fdw.bc

Attempted to CREATE EXTENSION in PGSQL 11.6

mvtest=# CREATE EXTENSION parquet_fdw;
ERROR:  could not load library "/usr/lib/postgresql/11/lib/parquet_fdw.so": /usr/lib/x86_64-linux-gnu/libarrow.so.15: undefined symbol: _ZN6google8protobuf8internal17AssignDescriptorsERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPKNS1_15MigrationSchemaEPKPKNS0_7MessageEPKjPNS0_8MetadataEPPKNS0_14EnumDescriptorEPPKNS0_17ServiceDescriptorE
dfirence commented 4 years ago

Any update on this one?

chriszrc commented 1 year ago

I'm on Ventura, 13.3.1, I installed arrow through homebrew, and this is the output from make:

| ~/github/parquet_fdw | (master) =>$ make install
g++ -Wall -Wpointer-arith -Werror=unguarded-availability-new -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv  -mmacosx-version-min=10.12 -arch arm64 -arch x86_64 -std=c++17 -O3  -I. -I./ -I/Applications/Postgres.app/Contents/Versions/14/include/postgresql/server -I/Applications/Postgres.app/Contents/Versions/14/include/postgresql/internal -I/Applications/Postgres.app/Contents/Versions/14/share/icu -I/Applications/Postgres.app/Contents/Versions/14/include/libxml2 -I/Applications/Postgres.app/Contents/Versions/14/include  -I/Applications/Postgres.app/Contents/Versions/14/include  -c -o src/common.o src/common.cpp
In file included from src/common.cpp:1:
In file included from src/common.hpp:7:
In file included from /usr/local/include/arrow/api.h:22:
In file included from /usr/local/include/arrow/array.h:41:
In file included from /usr/local/include/arrow/array/array_base.h:26:
In file included from /usr/local/include/arrow/array/data.h:28:
/usr/local/include/arrow/type.h:1805:36: error: 'get<arrow::FieldPath, arrow::FieldPath, std::string, std::vector<arrow::FieldRef>>' is unavailable: introduced in macOS 10.13
    if (IsFieldPath()) return std::get<FieldPath>(impl_).indices().size() > 1;
                                   ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/variant:1556:22: note: 'get<arrow::FieldPath, arrow::FieldPath, std::string, std::vector<arrow::FieldRef>>' has been explicitly marked unavailable here
constexpr const _Tp& get(const variant<_Types...>& __v) {
                     ^
In file included from src/common.cpp:1:
In file included from src/common.hpp:7:
In file included from /usr/local/include/arrow/api.h:22:
In file included from /usr/local/include/arrow/array.h:41:
In file included from /usr/local/include/arrow/array/array_base.h:26:
In file included from /usr/local/include/arrow/array/data.h:28:
/usr/local/include/arrow/type.h:1810:34: error: 'get<arrow::FieldPath, arrow::FieldPath, std::string, std::vector<arrow::FieldRef>>' is unavailable: introduced in macOS 10.13
    return IsFieldPath() ? &std::get<FieldPath>(impl_) : NULLPTR;
                                 ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/variant:1556:22: note: 'get<arrow::FieldPath, arrow::FieldPath, std::string, std::vector<arrow::FieldRef>>' has been explicitly marked unavailable here
constexpr const _Tp& get(const variant<_Types...>& __v) {
                     ^
In file included from src/common.cpp:1:
In file included from src/common.hpp:7:
In file included from /usr/local/include/arrow/api.h:22:
In file included from /usr/local/include/arrow/array.h:41:
In file included from /usr/local/include/arrow/array/array_base.h:26:
In file included from /usr/local/include/arrow/array/data.h:28:
/usr/local/include/arrow/type.h:1813:29: error: 'get<std::string, arrow::FieldPath, std::string, std::vector<arrow::FieldRef>>' is unavailable: introduced in macOS 10.13
    return IsName() ? &std::get<std::string>(impl_) : NULLPTR;
                            ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/variant:1556:22: note: 'get<std::string, arrow::FieldPath, std::string, std::vector<arrow::FieldRef>>' has been explicitly marked unavailable here
constexpr const _Tp& get(const variant<_Types...>& __v) {
                     ^
In file included from src/common.cpp:1:
In file included from src/common.hpp:7:
In file included from /usr/local/include/arrow/api.h:22:
In file included from /usr/local/include/arrow/array.h:41:
In file included from /usr/local/include/arrow/array/array_base.h:26:
In file included from /usr/local/include/arrow/array/data.h:28:
/usr/local/include/arrow/type.h:1817:24: error: 'get<std::vector<arrow::FieldRef>, arrow::FieldPath, std::string, std::vector<arrow::FieldRef>>' is unavailable: introduced in macOS 10.13
               ? &std::get<std::vector<FieldRef>>(impl_)
                       ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/variant:1556:22: note: 'get<std::vector<arrow::FieldRef>, arrow::FieldPath, std::string, std::vector<arrow::FieldRef>>' has been explicitly marked unavailable here
constexpr const _Tp& get(const variant<_Types...>& __v) {
                     ^
In file included from src/common.cpp:1:
In file included from src/common.hpp:7:
In file included from /usr/local/include/arrow/api.h:30:
/usr/local/include/arrow/datum.h:150:17: error: 'get<std::shared_ptr<arrow::ArrayData>, arrow::Datum::Empty, std::shared_ptr<arrow::Scalar>, std::shared_ptr<arrow::ArrayData>, std::shared_ptr<arrow::ChunkedArray>, std::shared_ptr<arrow::RecordBatch>, std::shared_ptr<arrow::Table>>' is unavailable: introduced in macOS 10.13
    return std::get<std::shared_ptr<ArrayData>>(this->value);
                ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/variant:1556:22: note: 'get<std::shared_ptr<arrow::ArrayData>, arrow::Datum::Empty, std::shared_ptr<arrow::Scalar>, std::shared_ptr<arrow::ArrayData>, std::shared_ptr<arrow::ChunkedArray>, std::shared_ptr<arrow::RecordBatch>, std::shared_ptr<arrow::Table>>' has been explicitly marked unavailable here
constexpr const _Tp& get(const variant<_Types...>& __v) {
                     ^
In file included from src/common.cpp:1:
In file included from src/common.hpp:7:
In file included from /usr/local/include/arrow/api.h:30:
/usr/local/include/arrow/datum.h:163:17: error: 'get<std::shared_ptr<arrow::ChunkedArray>, arrow::Datum::Empty, std::shared_ptr<arrow::Scalar>, std::shared_ptr<arrow::ArrayData>, std::shared_ptr<arrow::ChunkedArray>, std::shared_ptr<arrow::RecordBatch>, std::shared_ptr<arrow::Table>>' is unavailable: introduced in macOS 10.13
    return std::get<std::shared_ptr<ChunkedArray>>(this->value);
                ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/variant:1556:22: note: 'get<std::shared_ptr<arrow::ChunkedArray>, arrow::Datum::Empty, std::shared_ptr<arrow::Scalar>, std::shared_ptr<arrow::ArrayData>, std::shared_ptr<arrow::ChunkedArray>, std::shared_ptr<arrow::RecordBatch>, std::shared_ptr<arrow::Table>>' has been explicitly marked unavailable here
constexpr const _Tp& get(const variant<_Types...>& __v) {
                     ^
In file included from src/common.cpp:1:
In file included from src/common.hpp:7:
In file included from /usr/local/include/arrow/api.h:30:
/usr/local/include/arrow/datum.h:167:17: error: 'get<std::shared_ptr<arrow::RecordBatch>, arrow::Datum::Empty, std::shared_ptr<arrow::Scalar>, std::shared_ptr<arrow::ArrayData>, std::shared_ptr<arrow::ChunkedArray>, std::shared_ptr<arrow::RecordBatch>, std::shared_ptr<arrow::Table>>' is unavailable: introduced in macOS 10.13
    return std::get<std::shared_ptr<RecordBatch>>(this->value);
                ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/variant:1556:22: note: 'get<std::shared_ptr<arrow::RecordBatch>, arrow::Datum::Empty, std::shared_ptr<arrow::Scalar>, std::shared_ptr<arrow::ArrayData>, std::shared_ptr<arrow::ChunkedArray>, std::shared_ptr<arrow::RecordBatch>, std::shared_ptr<arrow::Table>>' has been explicitly marked unavailable here
constexpr const _Tp& get(const variant<_Types...>& __v) {
                     ^
In file included from src/common.cpp:1:
In file included from src/common.hpp:7:
In file included from /usr/local/include/arrow/api.h:30:
/usr/local/include/arrow/datum.h:171:17: error: 'get<std::shared_ptr<arrow::Table>, arrow::Datum::Empty, std::shared_ptr<arrow::Scalar>, std::shared_ptr<arrow::ArrayData>, std::shared_ptr<arrow::ChunkedArray>, std::shared_ptr<arrow::RecordBatch>, std::shared_ptr<arrow::Table>>' is unavailable: introduced in macOS 10.13
    return std::get<std::shared_ptr<Table>>(this->value);
                ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/variant:1556:22: note: 'get<std::shared_ptr<arrow::Table>, arrow::Datum::Empty, std::shared_ptr<arrow::Scalar>, std::shared_ptr<arrow::ArrayData>, std::shared_ptr<arrow::ChunkedArray>, std::shared_ptr<arrow::RecordBatch>, std::shared_ptr<arrow::Table>>' has been explicitly marked unavailable here
constexpr const _Tp& get(const variant<_Types...>& __v) {
                     ^
In file included from src/common.cpp:1:
In file included from src/common.hpp:7:
In file included from /usr/local/include/arrow/api.h:30:
/usr/local/include/arrow/datum.h:175:17: error: 'get<std::shared_ptr<arrow::Scalar>, arrow::Datum::Empty, std::shared_ptr<arrow::Scalar>, std::shared_ptr<arrow::ArrayData>, std::shared_ptr<arrow::ChunkedArray>, std::shared_ptr<arrow::RecordBatch>, std::shared_ptr<arrow::Table>>' is unavailable: introduced in macOS 10.13
    return std::get<std::shared_ptr<Scalar>>(this->value);
                ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/variant:1556:22: note: 'get<std::shared_ptr<arrow::Scalar>, arrow::Datum::Empty, std::shared_ptr<arrow::Scalar>, std::shared_ptr<arrow::ArrayData>, std::shared_ptr<arrow::ChunkedArray>, std::shared_ptr<arrow::RecordBatch>, std::shared_ptr<arrow::Table>>' has been explicitly marked unavailable here
constexpr const _Tp& get(const variant<_Types...>& __v) {
                     ^
9 errors generated.
make: *** [src/common.o] Error 1

Any advice on how to proceed?