Ubuntu 16, Postgres 11 - "make USE_PGXS=1" install fails

[bharathregal]

Gone thru bunch of troubleshooting steps to create missing symlinks for Clang, LLVM-lto to appropriate directory structure and finally got blocked with below error message:

Specs: OS - Ubuntu 16 PG - PG11 Clang - 6.0 LLVM - 6.0

/usr/bin/mkdir -p '/opt/postgres/bigsql/pg11/lib/postgresql' /usr/bin/mkdir -p '/opt/postgres/bigsql/pg11/share/postgresql/extension' /usr/bin/mkdir -p '/opt/postgres/bigsql/pg11/share/postgresql/extension' /usr/bin/mkdir -p '/opt/postgres/bigsql/pg11/share/doc/postgresql/extension' /usr/bin/install -c -m 755 tds_fdw.so '/opt/postgres/bigsql/pg11/lib/postgresql/tds_fdw.so' /usr/bin/install -c -m 644 .//tds_fdw.control '/opt/postgres/bigsql/pg11/share/postgresql/extension/' /usr/bin/install -c -m 644 .//sql/tds_fdw--2.0.0-alpha.3.sql '/opt/postgres/bigsql/pg11/share/postgresql/extension/' /usr/bin/install -c -m 644 .//README.tds_fdw.md '/opt/postgres/bigsql/pg11/share/doc/postgresql/extension/' /usr/bin/mkdir -p '/opt/postgres/bigsql/pg11/lib/postgresql/bitcode/tds_fdw' /usr/bin/mkdir -p '/opt/postgres/bigsql/pg11/lib/postgresql/bitcode'/tds_fdw/src/ /usr/bin/install -c -m 644 src/tds_fdw.bc '/opt/postgres/bigsql/pg11/lib/postgresql/bitcode'/tds_fdw/src/ /usr/bin/install -c -m 644 src/options.bc '/opt/postgres/bigsql/pg11/lib/postgresql/bitcode'/tds_fdw/src/ /usr/bin/install -c -m 644 src/deparse.bc '/opt/postgres/bigsql/pg11/lib/postgresql/bitcode'/tds_fdw/src/ cd '/opt/postgres/bigsql/pg11/lib/postgresql/bitcode' && /opt/rh/llvm-toolset-7/root/usr/bin/llvm-lto -thinlto -thinlto-action=thinlink -o tds_fdw.index.bc tds_fdw/src/tds_fdw.bc tds_fdw/src/options.bc tds_fdw/src/deparse.bc **error: can't create module summary index for buffer: Invalid Summary Block: version expected LLVM ERROR: ThinLink didn't create an index /opt/postgres/bigsql/pg11/lib/postgresql/pgxs/src/makefiles/pgxs.mk:229: recipe for target 'install' failed make: * [install] Error 1

[SudoerWithAnOpinion]

What commit are you building with? And what is the minor version of Postgres (11.4/11.5)?

[bharathregal]

We are building on top of Master branch and we are running PG11.5

[bharathregal]

Tried to install with CLANG 3.9, LLVM-3.9 and got below error msg.

cd '/opt/postgres/bigsql/pg11/lib/postgresql/bitcode' && /opt/rh/llvm-toolset-7/root/usr/bin/llvm-lto -thinlto -thinlto-action=thinlink -o tds_fdw.index.bc tds_fdw/src/tds_fdw.bc tds_fdw/src/options.bc tds_fdw/src/deparse.bc Invalid Summary Block: version expected error: can't create ModuleSummaryIndexObjectFile for buffer: Corrupted bitcode 0 libLLVM-3.9.so.1 0x00007f9105c22fe8 llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 56 1 libLLVM-3.9.so.1 0x00007f9105c2115e llvm::sys::RunSignalHandlers() + 62 2 libLLVM-3.9.so.1 0x00007f9105c212ac 3 libc.so.6 0x00007f9104e8c4b0 4 libLLVM-3.9.so.1 0x00007f91061d29d4 5 libLLVM-3.9.so.1 0x00007f91061e64ff llvm::WriteIndexToFile(llvm::ModuleSummaryIndex const&, llvm::raw_ostream&, std::map<std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::map<unsigned long, llvm::GlobalValueSummary, std::less, std::allocator<std::pair<unsigned long const, llvm::GlobalValueSummary> > >, std::less<std::cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::cxx11::basic_string<char, std::char_traits, std::allocator > const, std::map<unsigned long, llvm::GlobalValueSummary, std::less, std::allocator<std::pair<unsigned long const, llvm::GlobalValueSummary> > > > > >*) + 415 6 llvm-lto 0x0000000000418164 7 llvm-lto 0x000000000041936a 8 llvm-lto 0x000000000040cfc3 9 libc.so.6 0x00007f9104e77830 libc_start_main + 240 10 llvm-lto 0x000000000040eba9 Stack dump:

0. Program arguments: /opt/rh/llvm-toolset-7/root/usr/bin/llvm-lto -thinlto -thinlto-action=thinlink -o tds_fdw.index.bc tds_fdw/src/tds_fdw.bc tds_fdw/src/options.bc tds_fdw/src/deparse.bc Segmentation fault (core dumped) Use "fg" to return to nano.ib/postgresql/pgxs/src/makefiles/pgxs.mk:229: recipe for target 'install' failed make: *** [install] Error 139 [1]+ Stopped nano Makefileano Makefile

[juliogonzalez]

Note for me: time to switch our CI to Ubuntu 16.04.

PostgreSQL dev team is no longer updating repositories for Ubuntu 14.04, so it stayed with PostgreSQL 11.3 at our CI.

However PostgreSQL 11.5 on top of CentOS6 seems to be working fine: https://jenkins.juliogonzalez.es/view/tds_fdw-jobs/job/tds_fdw-build/DISTRO=centos6,PG_VER=11,label=docker/1389/console

@bharathregal are you using the official packages from the PostgreSQL dev team? It seems to me that you are using https://www.bigsql.org/

What happens if you try to reproduce the issue with the official packages?

[bharathregal]

Yes @juliogonzalez , We are using 3rd party distribution of Postgres. We have tested TDS FDW on same distribution of PG11 on Ubuntu14 and it worked fine with no issues(except for missing links for clang and llvm).

However, Let me test this using official release.

[carlosolivasio]

Seeing this when trying to run with core-postgresql packages, looks like a compiling issue with LLVM?

/bin/mkdir -p '/usr/lib/postgresql/11/lib' /bin/mkdir -p '/usr/share/postgresql/11/extension' /bin/mkdir -p '/usr/share/postgresql/11/extension' /bin/mkdir -p '/usr/share/doc/postgresql-doc-11/extension' /usr/bin/install -c -m 755 tds_fdw.so '/usr/lib/postgresql/11/lib/tds_fdw.so' /usr/bin/install -c -m 644 .//tds_fdw.control '/usr/share/postgresql/11/extension/' /usr/bin/install -c -m 644 .//sql/tds_fdw--2.0.0-alpha.3.sql '/usr/share/postgresql/11/extension/' /usr/bin/install -c -m 644 .//README.tds_fdw.md '/usr/share/doc/postgresql-doc-11/extension/' /bin/mkdir -p '/usr/lib/postgresql/11/lib/bitcode/tds_fdw' /bin/mkdir -p '/usr/lib/postgresql/11/lib/bitcode'/tds_fdw/src/ /usr/bin/install -c -m 644 src/tds_fdw.bc '/usr/lib/postgresql/11/lib/bitcode'/tds_fdw/src/ /usr/bin/install -c -m 644 src/options.bc '/usr/lib/postgresql/11/lib/bitcode'/tds_fdw/src/ /usr/bin/install -c -m 644 src/deparse.bc '/usr/lib/postgresql/11/lib/bitcode'/tds_fdw/src/ cd '/usr/lib/postgresql/11/lib/bitcode' && /usr/lib/llvm-6.0/bin/llvm-lto -thinlto -thinlto-action=thinlink -o tds_fdw.index.bc tds_fdw/src/tds_fdw.bc tds_fdw/src/options.bc tds_fdw/src/deparse.bc error: can't create module summary index for buffer: Invalid Summary Block: version expected LLVM ERROR: ThinLink didn't create an index /usr/lib/postgresql/11/lib/pgxs/src/makefiles/pgxs.mk:229: recipe for target 'install' failed make: *** [install] Error 1

[SudoerWithAnOpinion]

I have rebuilt the extension in Alpine Linux (in docker) using the latest postgres (11.5) and it functions properly. It may be a dependency issue specific to Ubuntu 16

[SudoerWithAnOpinion]

# OS   : Ubuntu 16.04
# PG   : PG 11.5
# Clang: 6.0
# LLVM : 6.0
# *Fresh Ubuntu 16.04 install (logged in as root)*
# *Add Postgresql APT*
apt-get install postgresql-11 postgresql-server-dev-11 libsybdb5 freetds-dev freetds-common
git clone https://github.com/tds-fdw/tds_fdw.git
cd tds_fdw
make USE_PGXS=1
make USE_PGXS=1 install

I'm not seeing any errors so it must be an issue specific to your system or Postgres install

[juliogonzalez]

I am now testing Ubuntu 16.04, and I can confirm I can't reproduce the issues.

https://jenkins.juliogonzalez.es/view/tds_fdw-jobs/job/tds_fdw-build-free/DISTRO=ubuntu16.04,PG_VER=11,label=docker/62/console.

01:05:47 cd '/usr/lib/postgresql/11/lib/bitcode' && /usr/lib/llvm-6.0/bin/llvm-lto -thinlto -thinlto-action=thinlink -o tds_fdw.index.bc tds_fdw/src/tds_fdw.bc tds_fdw/src/options.bc tds_fdw/src/deparse.bc

While for @bharathregal I see:

cd '/opt/postgres/bigsql/pg11/lib/postgresql/bitcode' && /opt/rh/llvm-toolset-7/root/usr/bin/llvm-lto -thinlto -thinlto-action=thinlink -o tds_fdw.index.bc tds_fdw/src/tds_fdw.bc tds_fdw/src/options.bc tds_fdw/src/deparse.bc

For @carlosolivasio I see the same paths as in my case, so I am not sure of what's going on, but it has to be something specific on that system, as neither @TheRevenantStar or myself (using our CI based on docker containers) can't reproduce.

For both @bharathregal and @carlosolivasio I suspect that somehow CLANG and LLVM versions are not matching: https://www.postgresql-archive.org/Build-fails-with-different-versions-of-clang-and-LLVM-td6018263.html https://stackoverflow.com/questions/15836430/how-stable-is-the-llvm-assembly-language

So I am 99.99% sure it's something specific on those systems having something "extra" that does not come with Ubuntu 16.04.

[juliogonzalez]

I now added CLANG and LLVM version detection at the CI, so it's clear what's happening:

https://jenkins.juliogonzalez.es/job/tds_fdw-build-pr/59/DISTRO=ubuntu16.04,PG_VER=11,label=docker/console

14:09:37 =================================================================================
14:09:37                                     INFO                                         
14:09:37 =================================================================================
14:09:41 DIST RELEASE: Ubuntu 16.04.6 LTS
14:09:41 **** PACKAGE VERSIONS ****
14:09:41 POSTGRESQL-SERVER: 11.5-1.pgdg16.04+1
14:09:41 POSTGRESQL-DEV:    11.5-1.pgdg16.04+1
14:09:41 FREETDS:           0.91-6.1build1
14:09:41 FREETDS-DEV:       0.91-6.1build1
14:09:41 LLVM:              1:6.0-1ubuntu2~16.04.1
14:09:41 CLANG:             1:6.0-1ubuntu2~16.04.1

As you can see, with the default LLVM and CLANG Ubuntu packages, and the official PostgreSQL 11.5, things are working as expected.

[MUrquizo]

Thanks @juliogonzalez . I am pretty sure it was due to mismatching CLANG and LLVM as we had several versions. We also had to install postgresql-server-dev-all, I did not see where to get postgresql-server-dev-11

This is the final setup that worked for @bharathregal and me on Ubuntu 16.04.5 LTS

$ sudo apt  list --installed | grep 'llvm\|clang\|post'
clang-6.0/unknown,now 1:6.0.1~svn334776-1~exp1~20190309042730.123 amd64 [installed]
libclang-common-6.0-dev/unknown,now 1:6.0.1~svn334776-1~exp1~20190309042730.123 amd64 [installed,automatic]
libclang1-6.0/unknown,now 1:6.0.1~svn334776-1~exp1~20190309042730.123 amd64 [installed,automatic]
libllvm6.0/unknown,now 1:6.0.1~svn334776-1~exp1~20190309042730.123 amd64 [installed,automatic]
llvm-6.0/unknown,now 1:6.0.1~svn334776-1~exp1~20190309042730.123 amd64 [installed,automatic]
llvm-6.0-dev/unknown,now 1:6.0.1~svn334776-1~exp1~20190309042730.123 amd64 [installed,automatic]
llvm-6.0-runtime/unknown,now 1:6.0.1~svn334776-1~exp1~20190309042730.123 amd64 [installed,automatic]
postgresql-client-common/xenial-updates,now 173ubuntu0.2 all [installed,automatic]
postgresql-common/xenial-updates,now 173ubuntu0.2 all [installed,automatic]
postgresql-server-dev-9.5/xenial-updates,xenial-security,now 9.5.19-0ubuntu0.16.04.1 amd64 [installed,automatic]
postgresql-server-dev-all/xenial-updates,now 173ubuntu0.2 all [installed]

[bharathregal]

Thanks Everyone. Closing this thread as we were able to resolve the issue as stated in above comment.