conda / conda-build

Commands and tools for building conda packages
https://docs.conda.io/projects/conda-build/
Other
382 stars 423 forks source link

libgcc not complete #782

Closed basnijholt closed 6 years ago

basnijholt commented 8 years ago

I compiled a package for kwant where libgcc is a runtime dep (here is the yml).

However when installing kwant afterwards libgcc is installed, but not complete.

For one when I try:

In [1]: import kwant
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-2e77c8a8f09d> in <module>()
----> 1 import kwant

/home/basnijholt/anaconda3/lib/python3.5/site-packages/kwant/__init__.py in <module>()
     12
     13 try:
---> 14     from . import _system
     15 except ImportError:
     16     import inspect

kwant/graph/core.pxd in init kwant._system (kwant/_system.c:24514)()

/home/basnijholt/anaconda3/lib/python3.5/site-packages/kwant/graph/__init__.py in <module>()
     12 __all__ = []
     13 for module in ['core', 'defs', 'slicer', 'utils']:
---> 14     exec('from . import {0}'.format(module))
     15     exec('from .{0} import *'.format(module))
     16     exec('__all__.extend({0}.__all__)'.format(module))

/home/basnijholt/anaconda3/lib/python3.5/site-packages/kwant/graph/__init__.py in <module>()

ImportError: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.15' not found (required by /home/basnijholt/anaconda3/lib/python3.5/site-packages/kwant/graph/slicer.cpython-35m-x86_64-linux-gnu.so)

Then I try:

[basnijholt@hpc05:~]$ conda install libgcc
Using Anaconda Cloud api site https://api.anaconda.org
Fetching package metadata: ....
Solving package specifications: ....................
# All requested packages already installed.
# packages in environment at /home/basnijholt/anaconda3:
#
libgcc                    4.8.5                         1

So libgcc is installed. When I uninstall libgcc and install it again via conda everything suddenly works.

This only happens when there is a gcc version already installed on the machine. Happens for both Mac OS X and Linux.

msarahan commented 8 years ago

This indicates that your original source of libgcc was too new for your system. That error generally points to your version of glibc (which is linked against, but not included in libgcc) being older than the version of glibc that your package provider compiled with.

It is generally frowned on to ship glibc directly. Instead, people should be compiling on old platforms with old glibc (but with new gcc). It's a sticky situation. We have a docker image in testing that should facilitate this: https://hub.docker.com/r/msarahan/conda_builder_linux/

basnijholt commented 8 years ago

I compile with the gcc of conda and the problem happens on the machine where I compiled.

It also happens for Mac OS X, how could I solve that?

basnijholt commented 8 years ago

In the Docker image I can't compile with conda gcc?

requirements:
  build:
    - gcc
  run:
    - libgcc

should be changed into:

requirements:
  build:
    - gcc [osx]
  run:
    - libgcc

?

msarahan commented 8 years ago

In the docker image, you should not need to. It is CentOS5 (which should be very compatible), and GCC 5.2. You'll need an updated libgcc package to go with it. For now, this is available on my anaconda.org channel:

conda install -c msarahan libgcc
basnijholt commented 8 years ago

One problem arises when compiling with the gcc in your Docker image:

[dev@ac48b9770452 conda-recipes]$ conda build scotch5;
Removing old build environment
Removing old work directory
BUILD START: scotch5-5.1.12b-1
Removing old work directory
Source cache directory is: /home/dev/miniconda/conda-bld/src_cache
Found source in cache: scotch_5.1.12b_esmumps.tar.gz
Extracting download
Package: scotch5-5.1.12b-1
source tree in: /home/dev/miniconda/conda-bld/work/scotch_5.1.12_esmumps
+ cd src/
+ echo 'prefix = $PREFIX'
+ echo ''
++ uname
+ '[' Linux == Darwin ']'
+ cp Make.inc/Makefile.inc.x86-64_pc_linux2 Makefile.inc
+ sed -i 's@CLIBFLAGS\t=@CLIBFLAGS\t= -fPIC@g' Makefile.inc
+ sed -i 's#-l$(SCOTCHLIB)errexit#-l$(SCOTCHLIB)errexit -lm#g' esmumps/Makefile
+ sed -i 's#-lz -lm -lrt#-lz -lm -lrt -lpthread#g' Makefile.inc
+ make
+ tee make.log
mkdir ../bin
mkdir ../include
mkdir ../lib
(cd libscotch ;      make VERSION=5 RELEASE=1 PATCHLEVEL=12 scotch && make install)
make[1]: Entering directory `/home/dev/miniconda/conda-bld/work/scotch_5.1.12_esmumps/src/libscotch'
rm -f *~ *.o lib*.a parser_yy.c parser_ly.h parser_ll.c *scotch.h *scotchf.h y.output dummysizes
make CC="gcc" CCD="gcc" \
                    scotch.h                \
                    scotchf.h               \
                    libscotch.a             \
                    libscotcherr.a          \
                    libscotcherrexit.a
make[2]: Entering directory `/home/dev/miniconda/conda-bld/work/scotch_5.1.12_esmumps/src/libscotch'
gcc -O3 -DCOMMON_FILE_COMPRESS_GZ -DCOMMON_PTHREAD -DCOMMON_RANDOM_FIXED_SEED -DSCOTCH_RENAME -DSCOTCH_RENAME_PARSER -DSCOTCH_PTHREAD -Drestrict=__restrict -DIDXSIZE64 -DSCOTCH_VERSION=5 -DSCOTCH_RELEASE=1 -DSCOTCH_PATCHLEVEL=12 dummysizes.c -o dummysizes -lz -lm -lrt -lpthread
/usr/local/lib/gcc/x86_64-redhat-linux-gnu/5.2.0/../../../../x86_64-redhat-linux-gnu/bin/ld: cannot find -lz
collect2: error: ld returned 1 exit status
make[2]: *** [dummysizes] Error 1
make[2]: Leaving directory `/home/dev/miniconda/conda-bld/work/scotch_5.1.12_esmumps/src/libscotch'
make[1]: *** [scotch] Error 2
make[1]: Leaving directory `/home/dev/miniconda/conda-bld/work/scotch_5.1.12_esmumps/src/libscotch'
make: *** [scotch] Error 2
+ cd ..
+ mkdir -p /home/dev/miniconda/envs/_build/lib/
+ cp 'lib/*' /home/dev/miniconda/envs/_build/lib/
cp: cannot stat `lib/*': No such file or directory
Command failed: /bin/bash -x -e /conda-recipes/scotch5/build.sh
msarahan commented 8 years ago

Thanks for trying it out. It looks like zlib is somehow missing or not being found. I'll look into why this is happening.

msarahan commented 8 years ago

I think I left it out because we generally explicitly list zlib as a dependency in conda. In your meta.yaml:

requirements:
  build:
    - zlib

If you need it at runtime, too (shared libs):

requirements:
  build:
    - zlib
  run:
    - zlib

Doing things this way (rather than having zlib in the docker image) should make your recipe more buildable on more systems.

stuarteberg commented 8 years ago

This appears to be a duplicate of conda/conda-recipes#558, although I'm concerned to hear that you're seeing problems on Mac OSX, too.

msarahan commented 6 years ago

compiler libraries have been completely reworked in the Anaconda 5 series of packages (anything on the main subchannel of defaults.) Closing.

github-actions[bot] commented 2 years ago

Hi there, thank you for your contribution!

This issue has been automatically locked because it has not had recent activity after being closed.

Please open a new issue if needed.

Thanks!