search

tds-fdw / tds_fdw

A PostgreSQL foreign data wrapper to connect to TDS databases (Sybase and Microsoft SQL Server)
Other
377 stars 101 forks source link

Cannot compile against PG11 #192

Closed fluca1978 closed 5 years ago

fluca1978 commented 5 years ago

On CentOS 7 64 bit, PostgreSQL 11.1 I cannot compile version 2.0.0-alpha.2:

make
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC -I./include/ -fvisibility=hidden -I. -I./ -I/usr/pgsql-11/include/server -I/usr/pgsql-11/include/internal  -D_GNU_SOURCE -I/usr/include/libxml2  -I/usr/include  -c -o src/tds_fdw.o src/tds_fdw.c
src/tds_fdw.c: In function ‘tdsGetColumnMetadata’:
src/tds_fdw.c:1395:72: error: invalid type argument of ‘->’ (have ‘FormData_pg_attribute’)
       column->attr_oid = festate->attinmeta->tupdesc->attrs[local_ncol]->atttypid;
                                                                        ^
src/tds_fdw.c:1404:65: error: invalid type argument of ‘->’ (have ‘FormData_pg_attribute’)
      local_name = festate->attinmeta->tupdesc->attrs[local_ncol]->attname.data;
                                                                 ^
src/tds_fdw.c:1417:72: error: invalid type argument of ‘->’ (have ‘FormData_pg_attribute’)
       column->attr_oid = festate->attinmeta->tupdesc->attrs[local_ncol]->atttypid;
                                                                        ^
src/tds_fdw.c:1443:63: error: invalid type argument of ‘->’ (have ‘FormData_pg_attribute’)
    column->attr_oid = festate->attinmeta->tupdesc->attrs[ncol]->atttypid;
                                                               ^
In file included from /usr/pgsql-11/include/server/postgres.h:47:0,
                 from src/tds_fdw.c:32:
src/tds_fdw.c:1461:47: error: invalid type argument of ‘->’ (have ‘FormData_pg_attribute’)
       festate->attinmeta->tupdesc->attrs[ncol]->attname.data)
                                               ^
/usr/pgsql-11/include/server/utils/elog.h:107:14: note: in definition of macro ‘ereport_domain’
    errfinish rest; \
              ^
src/tds_fdw.c:1457:5: note: in expansion of macro ‘ereport’
     ereport(WARNING,
     ^
make: *** [src/tds_fdw.o] Error 1
fluca1978 commented 5 years ago

I've pulled the commit 3719a995b0ae3fc4c4b390dd8a2820d54b88e18a from the repository, and it seems a little better but I've something wrong with clang or llvm:

$ make
/opt/rh/llvm-toolset-7/root/usr/bin/clang -Wno-ignored-attributes -fno-strict-aliasing -fwrapv -O2  -I./include/ -fvisibility=hidden -I. -I./ -I/usr/pgsql-11/include/server -I/usr/pgsql-11/include/internal  -D_GNU_SOURCE -I/usr/include/libxml2  -I/usr/include -flto=thin -emit-llvm -c -o src/tds_fdw.bc src/tds_fdw.c
make: /opt/rh/llvm-toolset-7/root/usr/bin/clang: Command not found
make: *** [src/tds_fdw.bc] Error 127

I cannot get llvm-toolset from the yum repository, so cannot try. Suggestions?

SudoerWithAnOpinion commented 5 years ago

This sounds like a package availability issues, not a tds_fdw issue.

Are you are sure you asked yum to install the right package? sudo yum install llvm-toolset-7

juliogonzalez commented 5 years ago

I am able to reproduce the problem on a clean CentOS7, but not on CentOS6.

clang is provided by clang package, but even after get it installed it is still complaining about /opt/rh/llvm-toolset-7/root/usr/bin/clang not being available.

So something is trying to use clang only on CentOS7 and not from the distribution, as AFAIK standard CentOS packages are not using /opt

juliogonzalez commented 5 years ago

So for RHEL7: https://developers.redhat.com/blog/2018/07/07/yum-install-gcc7-clang/

And for CentOS7: https://stackoverflow.com/a/48103599/4055246

And after getting llvm-toolset-7 and llvm5.0 installed, tds_fdw builds again.

But I wonder why this is suddenly required. I am pretty sure this is something required by something external (freetds? make?)

juliogonzalez commented 5 years ago

OK:

[root@f793aad80d0c tds_fdw]# grep -ir '/opt/rh/llvm-toolset-7' /usr
/usr/pgsql-11/lib/pgxs/src/Makefile.global:configure_args =  '--enable-rpath' '--prefix=/usr/pgsql-11' '--includedir=/usr/pgsql-11/include' '--mandir=/usr/pgsql-11/share/man' '--datadir=/usr/pgsql-11/share' '--with-icu' 'CLANG=/opt/rh/llvm-toolset-7/root/usr/bin/clang' 'LLVM_CONFIG=/usr/lib64/llvm5.0/bin/llvm-config' '--with-llvm' '--with-perl' '--with-python' '--with-tcl' '--with-tclconfig=/usr/lib64' '--with-openssl' '--with-pam' '--with-gssapi' '--with-includes=/usr/include' '--with-libraries=/usr/lib64' '--enable-nls' '--enable-dtrace' '--with-uuid=e2fs' '--with-libxml' '--with-libxslt' '--with-ldap' '--with-selinux' '--with-systemd' '--with-system-tzdata=/usr/share/zoneinfo' '--sysconfdir=/etc/sysconfig/pgsql' '--docdir=/usr/pgsql-11/doc' '--htmldir=/usr/pgsql-11/doc/html' 'CFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic' 'LDFLAGS=-Wl,--as-needed' 'PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig'
/usr/pgsql-11/lib/pgxs/src/Makefile.global:CLANG = /opt/rh/llvm-toolset-7/root/usr/bin/clang
[root@f793aad80d0c tds_fdw]# rpm -qf /usr/pgsql-11/lib/pgxs/src/Makefile.global
postgresql11-devel-11.1-1PGDG.rhel7.x86_64

So it's neither freetds or make, but postgresql11-devel!

And it seems not only tds_fdw is affected. Here is the explanation: https://github.com/theory/pg-semver/issues/35#issuecomment-440671985

It seems we will need to update the instructions for CentOS, to add this special step for CentOS7, and I will need to keep it in mind when I create the CI for CentOS7.

juliogonzalez commented 5 years ago

Even with llvm-toolset-7 it is still possible to compile with:

make USE_PGXS=1 CLANG=true install

As instructed at https://github.com/powa-team/pg_stat_kcache/issues/15#issuecomment-422691873

However the install breaks, with our without -k parameter:

[root@ea24bc4f805d tds_fdw]# PATH=/usr/pgsql-11/bin:$PATH make -k USE_PGXS=1 CLANG=true install
true -Wno-ignored-attributes -fno-strict-aliasing -fwrapv -O2  -I./include/ -fvisibility=hidden -I. -I./ -I/usr/pgsql-11/include/server -I/usr/pgsql-11/include/internal  -D_GNU_SOURCE -I/usr/include/libxml2  -I/usr/include -flto=thin -emit-llvm -c -o src/tds_fdw.bc src/tds_fdw.c
true -Wno-ignored-attributes -fno-strict-aliasing -fwrapv -O2  -I./include/ -fvisibility=hidden -I. -I./ -I/usr/pgsql-11/include/server -I/usr/pgsql-11/include/internal  -D_GNU_SOURCE -I/usr/include/libxml2  -I/usr/include -flto=thin -emit-llvm -c -o src/options.bc src/options.c
true -Wno-ignored-attributes -fno-strict-aliasing -fwrapv -O2  -I./include/ -fvisibility=hidden -I. -I./ -I/usr/pgsql-11/include/server -I/usr/pgsql-11/include/internal  -D_GNU_SOURCE -I/usr/include/libxml2  -I/usr/include -flto=thin -emit-llvm -c -o src/deparse.bc src/deparse.c
/usr/bin/mkdir -p '/usr/pgsql-11/lib'
/usr/bin/mkdir -p '/usr/pgsql-11/share/extension'
/usr/bin/mkdir -p '/usr/pgsql-11/share/extension'
/usr/bin/mkdir -p '/usr/pgsql-11/doc/extension'
/usr/bin/install -c -m 755  tds_fdw.so '/usr/pgsql-11/lib/tds_fdw.so'
/usr/bin/install -c -m 644 .//tds_fdw.control '/usr/pgsql-11/share/extension/'
/usr/bin/install -c -m 644 .//sql/tds_fdw--2.0.0-alpha.2.sql  '/usr/pgsql-11/share/extension/'
/usr/bin/install -c -m 644 .//README.tds_fdw.md '/usr/pgsql-11/doc/extension/'
/usr/bin/mkdir -p '/usr/pgsql-11/lib/bitcode/tds_fdw'
/usr/bin/mkdir -p '/usr/pgsql-11/lib/bitcode'/tds_fdw/src/
/usr/bin/install -c -m 644 src/tds_fdw.bc '/usr/pgsql-11/lib/bitcode'/tds_fdw/src/
/usr/bin/install: cannot stat 'src/tds_fdw.bc': No such file or directory
make: *** [install] Error 1
[root@ea24bc4f805d tds_fdw]# PATH=/usr/pgsql-11/bin:$PATH make USE_PGXS=1 CLANG=true install
true -Wno-ignored-attributes -fno-strict-aliasing -fwrapv -O2  -I./include/ -fvisibility=hidden -I. -I./ -I/usr/pgsql-11/include/server -I/usr/pgsql-11/include/internal  -D_GNU_SOURCE -I/usr/include/libxml2  -I/usr/include -flto=thin -emit-llvm -c -o src/tds_fdw.bc src/tds_fdw.c
true -Wno-ignored-attributes -fno-strict-aliasing -fwrapv -O2  -I./include/ -fvisibility=hidden -I. -I./ -I/usr/pgsql-11/include/server -I/usr/pgsql-11/include/internal  -D_GNU_SOURCE -I/usr/include/libxml2  -I/usr/include -flto=thin -emit-llvm -c -o src/options.bc src/options.c
true -Wno-ignored-attributes -fno-strict-aliasing -fwrapv -O2  -I./include/ -fvisibility=hidden -I. -I./ -I/usr/pgsql-11/include/server -I/usr/pgsql-11/include/internal  -D_GNU_SOURCE -I/usr/include/libxml2  -I/usr/include -flto=thin -emit-llvm -c -o src/deparse.bc src/deparse.c
/usr/bin/mkdir -p '/usr/pgsql-11/lib'
/usr/bin/mkdir -p '/usr/pgsql-11/share/extension'
/usr/bin/mkdir -p '/usr/pgsql-11/share/extension'
/usr/bin/mkdir -p '/usr/pgsql-11/doc/extension'
/usr/bin/install -c -m 755  tds_fdw.so '/usr/pgsql-11/lib/tds_fdw.so'
/usr/bin/install -c -m 644 .//tds_fdw.control '/usr/pgsql-11/share/extension/'
/usr/bin/install -c -m 644 .//sql/tds_fdw--2.0.0-alpha.2.sql  '/usr/pgsql-11/share/extension/'
/usr/bin/install -c -m 644 .//README.tds_fdw.md '/usr/pgsql-11/doc/extension/'
/usr/bin/mkdir -p '/usr/pgsql-11/lib/bitcode/tds_fdw'
/usr/bin/mkdir -p '/usr/pgsql-11/lib/bitcode'/tds_fdw/src/
/usr/bin/install -c -m 644 src/tds_fdw.bc '/usr/pgsql-11/lib/bitcode'/tds_fdw/src/
/usr/bin/install: cannot stat 'src/tds_fdw.bc': No such file or directory
make: *** [install] Error 1

I guess we probablyt need to change something at the source code, but I am not sure what exactly.

@GeoffMontee any idea?

juliogonzalez commented 5 years ago

After reviewing more doc, it seems CLANG parameter just allows compiling by simulating that clang is available by calling the true command, but of course then PostgreSQL still tries to get the bytecode generated by clang and it's not found (it tries to get it even when CLANG=true is present).

I am not sure if there's anything we can do on our side, other than modifying the README for CentOS, to explain this.

juliogonzalez commented 5 years ago

And BTW I see this was also explained at https://github.com/tds-fdw/tds_fdw/issues/182#issuecomment-433183362 by @donnorman, so I will submit a PR for the doc and will let @GeoffMontee decide how to proceed.

fluca1978 commented 5 years ago

Just for the record:

$ sudo yum install centos-release-scl
$ sudo yum install llvm-toolset-7
$ sudo yum install llvm-5.0

that made me compile and install without any make option (no USE_PGXS, no CLANG).