pip install trako fails

[tashrifbillah]

    gmake[2]: *** [CMakeFiles/draco_compression_attributes_dec.dir/src/draco/compression/attributes/kd_tree_attributes_decoder.cc.o] Error 1
    gmake[1]: *** [CMakeFiles/draco_compression_attributes_dec.dir/all] Error 2
    gmake: *** [all] Error 2
    Traceback (most recent call last):
      File "/tmp/pip-build-env-gm55xlip/lib/python3.7/site-packages/skbuild/setuptools_wrap.py", line 577, in setup
        cmkr.make(make_args, env=env)
      File "/tmp/pip-build-env-gm55xlip/lib/python3.7/site-packages/skbuild/cmaker.py", line 482, in make
        os.path.abspath(CMAKE_BUILD_DIR())))

    An error occurred while building with CMake.
      Command:
        "cmake" "--build" "." "--target" "install" "--config" "Release" "--"
      Source directory:
        /tmp/pip-install-evsbjt8l/TrakoDracoPy
      Working directory:
        /tmp/pip-install-evsbjt8l/TrakoDracoPy/_skbuild/linux-x86_64-3.7/cmake-build
    Please see CMake's output for more information.

    ----------------------------------------
Command "/tmp/miniconda3_2/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-evsbjt8l/TrakoDracoPy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-ata12qwi/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-evsbjt8l/TrakoDracoPy/

[tashrifbillah]

cc @yrathi

[haehn]

Thanks, @tashrifbillah ! I am not sure about this error but maybe CMake's output helps?

[tashrifbillah]

@haehn , please see the log that you asked for:

trako_install.log

By the way, when opening the issue, I didn't realize there was a python setup.py install step after creating the conda environment named TRAKO. Later, I tried that step. The step gives same error as that of pip install trako.

That being said, I have not been able to install trako in the following environment at all:

Linux pnl-z840-2 3.10.0-957.27.2.el7.x86_64 #1 SMP Mon Jul 29 17:46:05 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

(edited my issue description accordingly)

[haehn]

pip install trako should just do it (on Ubuntu 18.04 and Mac).. the python setup.py install is only for the developer installation.

Which OS are you running? Which compiler? The log file points to some C++ errors in the Draco library (overloading) so I expect some version mismatch.

[haehn]

And you said it worked with python=3.6?

[tashrifbillah]

And you said it worked with python=3.6?

I didn't.

Please see my earlier comment with OS specification. It is a CentOS 7.

You can send me the cmake/gmake build commands. Maybe I can also help in developing a robust installation scheme.

[haehn]

I unfortunately don't know how to build it on CentOS 7.

The pip install trako is a general source distribution and builds on most modern systems.

[haehn]

Which Anaconda version are you using?

[tashrifbillah]

The pip install trako is a general source distribution and builds on most modern systems.

Which is why it is wired that it failed on my machine.

Which Anaconda version are you using?

We build all our software from a skeletal miniconda3

[pieper]

I can replicate the issue on a google cloud vm running centos7

Starting from a fresh instance:

$ uname -a
Linux instance-1 3.10.0-1062.18.1.el7.x86_64 #1 SMP Tue Mar 17 23:49:17 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
$ sudo yum -y update && sudo yum -y install python3 python3-pip gcc gcc-c++
$ sudo pip3 install scikit-build
$ sudo pip3 install trako

and I get the same build errors as in the log. Looks like a TrakoDracoPy level c++ issue with an overloaded method.

[tashrifbillah]

From the log I posted:

CMake Error: CMake was unable to find a build program corresponding to "Ninja". CMAKE_MAKE_PROGRAM is not set. You probably need to select a different build tool. -- Configuring incomplete, errors occurred!

Doesn't this bother us?

[pieper]

@haehn how can I build just TrakoDracoPy from source if I check out your repo?

[tashrifbillah]

@haehn how can I build just TrakoDracoPy from source if I check out your repo?

I am already working on it!

[pieper]

Thanks @tashrifbillah - let us know what you find! I wouldn't worry about the ninja error since it appears to fall back to makefiles.

[tashrifbillah]

@haehn, it appears that https://pypi.org/project/TrakoDracoPy/ installation fails in first place. With pip install TrakoDracoPy, I get the same error that I reported here.

After I took out TrakoDracoPy from setup.py of this repo:

'six>=1.13.0',
'stringcase>=1.2.0',
# 'TrakoDracoPy',
'typing-extensions>=3.7.4.1',
'typing-inspect>=0.5.0',

python setup.py install of TRAKO works fine.

Based on the above, I believe the error traces its root in https://github.com/seung-lab/draco/ which is what you try to build here.

I can try to build draco from source and try to sort out the issue.

[haehn]

Thanks, @tashrifbillah and @pieper !

[tashrifbillah]

Hi @pieper , I would like to share my experiment. Please see the hierarchy below:

Google Draco (succeeds)
   |-Seung-lab Draco (fails)
      |-TrakoDracoPy (fails)

• https://github.com/google/draco is the root of all, builds fine

• https://github.com/seung-lab/draco, which is a fork of Google Draco, doesn't build. I didn't find the output log helpful. Posting it below: CMakeOutput.log

• Since @haehn has forked from seung-lab, it makes sense that his fork would fail too.

[pieper]

Hi @tashrifbillah - thanks for digging in to this 👍

I can confirm that the google version builds for me and the seung version fails.

But I do not get weird cmake errors like you did. Maybe because I installed the very latest release. Here's what I get for the seung version:

INFO|Draco: ignoring empty build type, forcing release mode.
-- The C compiler identification is GNU 4.8.5
-- The CXX compiler identification is GNU 4.8.5
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc - works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ - works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
Checking CXX compiler flag support for: -std=c++11
-- Performing Test HAVE_CXX_FLAG
-- Performing Test HAVE_CXX_FLAG - Success
Checking CXX compiler flag support for: -fPIC
-- Found Git: /usr/bin/git (found version "1.8.3.1") 
-- Configuring done
-- Generating done
-- Build files have been written to: /home/pieper/seung/draco-build

But then I get the same compile errors for the method overrides that you saw originally in trako.

Let me see if I can come up with a fix.

[pieper]

Hmm, looks like the seung version stripped out some DRACO_OLD_GCC blocks and did some other reformats. I didn't see an obvious fix for gcc.

But clang works for building draco - @tashrifbillah would clang be an option for you?

This works:

sudo yum install clang
CC=clang CXX=clang++ cmake ../draco; make

But this fails:

CC=clang CXX=clang++ pip3 install trako

@haehn would you be able to look and see if you can see what's going on in the pip part?

Here's the error message:

    clang -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I_skbuild/linux-x86_64-3.6/cmake-install/include/ -I/usr/include/python3.6m -c ./src/TrakoDracoPy.cpp -o _skbuild/linux-x86_64-3.6/setuptools/temp.linux-x86_64-3.6/./src/TrakoDracoPy.o -std=c++11 -O3
    clang: error: unknown argument: '-fstack-protector-strong'
    error: command 'clang' failed with exit status 1

[tashrifbillah]

Edit on October 28, 2020

Please follow this recipe for building. Two notes--

• a separate conda environment seemed to misbehave
• python setup.py install seemed to fail

---

Hi @pieper ,

CC=clang CXX=clang++ cmake ../draco; make works for seung-lab/draco

But contrary to what you observed, both works for me:

CC=clang CXX=clang++ pip install trako

# and

conda create -y --name TRAKO python=3.6
conda activate TRAKO
cd haen/draco
CC=clang CXX=clang++ python setup.py install

At least the build went fine.

I am going to try with pip install clang because not everyone will have sudo.

[haehn]

Ok - great. We can also maybe try to build against upstream draco rather than Seung's lab.

[tashrifbillah]

Hello @haehn , I have been exploring this idea for a while. If you change this line to google/draco and then try pip install . for TrakoDracoPy, google/draco builds fine but the following error appears in the linking stage:

  /tmp/miniconda3-TrDrGoogle/envs/TRAKO/compiler_compat/ld: /tmp/miniconda3-TrDrGoogle/envs/TRAKO/lib/libdraco.a(geometry_metadata.cc.o): relocation R_X86_64_32S against symbol `_ZNSs4_Rep20_S_empty_rep_storageE@@GLIBCXX_3.4' can not be used when making a shared object; recompile with -fPIC
  /tmp/miniconda3-TrDrGoogle/envs/TRAKO/compiler_compat/ld: /tmp/miniconda3-TrDrGoogle/envs/TRAKO/lib/libdraco.a(dynamic_integer_points_kd_tree_decoder.cc.o): relocation R_X86_64_32 against `.bss' can not be used when making a shared object; recompile with -fPIC
  /tmp/miniconda3-TrDrGoogle/envs/TRAKO/compiler_compat/ld: /tmp/miniconda3-TrDrGoogle/envs/TRAKO/lib/libdraco.a(float_points_tree_decoder.cc.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
  /tmp/miniconda3-TrDrGoogle/envs/TRAKO/compiler_compat/ld: /tmp/miniconda3-TrDrGoogle/envs/TRAKO/lib/libdraco.a(attributes_decoder.cc.o): relocation R_X86_64_32S against symbol `_ZTVN5draco17AttributesDecoderE' can not be used when making a shared object; recompile with -fPIC
  /tmp/miniconda3-TrDrGoogle/envs/TRAKO/compiler_compat/ld: /tmp/miniconda3-TrDrGoogle/envs/TRAKO/lib/libdraco.a(sequential_attribute_decoder.cc.o): relocation R_X86_64_32S against symbol `_ZTVN5draco26SequentialAttributeDecoderE' can not be used when making a shared object; recompile with -fPIC
  collect2: error: ld returned 1 exit status
  error: command 'g++' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for TrakoDracoPy
Failed to build TrakoDracoPy
ERROR: Could not build wheels for TrakoDracoPy which use PEP 517 and cannot be installed directly

I traced the source of error to this line:

extra_link_args = ['-L{0}'.format(lib_dir)] + library_link_args

If you remove library_link_args, the pip install . works fine.

Given the above, maybe @pieper can help us finding a solution?

I think trying to build from google/draco is a good idea since we haven't come this far with seung/draco.

[pieper]

@tashrifbillah maybe try adding -fPIC to the end of extra_link_args?

[tashrifbillah]

I tried adding here:

extra_compile_args=[
  '-std=c++11','-O3'
],

Again, I tried adding where you said, but getting the same error.

[pieper]

@tashrifbillah are you allowed to use singularity? We could build a trako image.

[tashrifbillah]

Sure, containerization is a way around with clang. But it's not an option on Partners HPC yet, so no.

[pieper]

Do they specifically prohibit singularity or just not provide it? I was able to install it from source on centos7 on my vm (I don't use ERIS, or at least not for years).

[tashrifbillah]

The first requirement is Centos7. Partners has only 3 compute nodes running Centos7. The rest are running Centos6. Maybe this is why they haven't installed it.

On the other hand, I was under the impression that I need sudo to install singularity, no?

[pieper]

You probably need to have some prerequisite packages, but then you can install singularity itself to your own userspace. Might be able to install the prereqs too I guess. I actually haven't tried it much but it seems like a possible workaround.

Probably better to just get the link error fixed. Let's see what @haehn says.

[haehn]

Yes, I agree let's try to get the linking fixed. But I have no idea how to fix it :)

[haehn]

Maybe some of the sysadmins know this error since they are CentOS experts?

[tashrifbillah]

Maybe some of the sysadmins know this error since they are CentOS experts?

Except it is not their job to help us develop software, so I wouldn't hold my breath. The riddle to solve here should be: google/draco builds fine from source while python wrapped source fails.

The problem addressed by this software and given solution sound promising, which is why I am just trying to make it usable by more people. In my experience, most Linux users are RHEL compared to Ubuntu where you may have perfected it. Anyways, when I get some more time, I might be able to find a workaround.

[haehn]

The sysadmins helped me before because of the older CentOS versions so I would ask them. I appreciate you trying to debug - thank you.

[tashrifbillah]

Way to reproduce error:

cd TrakoDracoPy
pip install . --no-clean
cd /tmp/build/directory/from/above

gcc -pthread -B /tmp/miniconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I_skbuild/linux-x86_64-3.7/cmake-install/include/ -I/tmp/miniconda3/include/python3.7m -c ./src/TrakoDracoPy.cpp -o _skbuild/linux-x86_64-3.7/setuptools/temp.linux-x86_64-3.7/./src/TrakoDracoPy.o -std=c++11 -O3

Notice the already included -fPIC flag in bold. The above runs fine but the next step runs into error:

g++ -pthread -shared -B /tmp/miniconda3/compiler_compat -L/tmp/miniconda3/lib -Wl,-rpath=/tmp/miniconda3/lib -Wl,--no-as-needed -Wl,--sysroot=/ _skbuild/linux-x86_64-3.7/setuptools/temp.linux-x86_64-3.7/./src/TrakoDracoPy.o -o _skbuild/linux-x86_64-3.7/setuptools/lib.linux-x86_64-3.7/TrakoDracoPy.cpython-37m-x86_64-linux-gnu.so -L/tmp/pip-req-build-j45qy2bj/_skbuild/linux-x86_64-3.7/cmake-install/lib -l:libdracoenc.a -l:libdraco.a -l:libdracodec.a

[tashrifbillah]

A more proper title of this issue should be: skbuild.setup() fails in linking libraries to executables which originates in https://github.com/haehn/TrakoDracoPy/blob/master/setup.py#L58

[tashrifbillah]

It looks like it is a known issue originating from skbuild: https://github.com/scikit-build/scikit-build/pull/121#issuecomment-238642797

[tashrifbillah]

Hi @pieper , our @yrathi increased the priority of TRAKO installation on our HPC. Accordingly, I have made this Singularity container and tested on a few cases. It seems to work fine.

Hi @haehn , if PR is welcome, I can integrate my Singularity recipe in a folder with TRACO.

[pieper]

That is awesome @tashrifbillah ! I had no idea that singularity was an option on the cluster, but it sounds like an ideal solution. Will it satisfy the use case long-term? If so we don't need to work on the back porting project.

[tashrifbillah]

I had no idea that singularity was an option on the cluster

Hi @pieper , it was not until recently. We pushed for its availability and got it.

Will it satisfy the use case long-term? If so we don't need to work on the back porting project.

It should. I agree, we may not need Kitware support for fixing this anymore (not sure if that is what you meant).

[pieper]

That's great @tashrifbillah. Yes, I was referring to the project we were working on with @jcfr to change the compilation of trako to work with older OS versions. We can drop that project now.

[jcfr]

I was still planing on updating TrakoDracoPy so that wheels are published on all three platforms.