pip install error mentioning relocation R_X86_64_32S

[cdeil]

I'm trying to install Fermipy into ScienceTools-v10r0p5-fssc-20150518-x86_64-unknown-linux-gnu-libc2.12 on a server with Scientific Linux 6. For Numpy and other packages with C extensions, I'm running into this error:

gcc -pthread -shared -L/Home/lhea2/jasercio/sulacco/20150518/WithRoot/ScienceTools/tcltk/tcl/unix -L/Home/lhea2/jasercio/sulacco/20150518/WithRoot/ScienceTools/tcltk/tk/unix build/temp.linux-x86_64-2.7/numpy/core/src/dummymodule.o -L/Home/lhea2/jasercio/sulacco/20150518/WithRoot/ScienceTools/external/x86_64-unknown-linux-gnu-libc2.12/lib -Lbuild/temp.linux-x86_64-2.7 -lm -lpython2.7 -o build/lib.linux-x86_64-2.7/numpy/core/_dummy.so
    /usr/bin/ld: /usr/local/lib/libpython2.7.a(modsupport.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
    /usr/local/lib/libpython2.7.a: could not read symbols: Bad value
    collect2: ld returned 1 exit status
    /usr/bin/ld: /usr/local/lib/libpython2.7.a(modsupport.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
    /usr/local/lib/libpython2.7.a: could not read symbols: Bad value
    collect2: ld returned 1 exit status
    error: Command "gcc -pthread -shared -L/Home/lhea2/jasercio/sulacco/20150518/WithRoot/ScienceTools/tcltk/tcl/unix -L/Home/lhea2/jasercio/sulacco/20150518/WithRoot/ScienceTools/tcltk/tk/unix build/temp.linux-x86_64-2.7/numpy/core/src/dummymodule.o -L/Home/lhea2/jasercio/sulacco/20150518/WithRoot/ScienceTools/external/x86_64-unknown-linux-gnu-libc2.12/lib -Lbuild/temp.linux-x86_64-2.7 -lm -lpython2.7 -o build/lib.linux-x86_64-2.7/numpy/core/_dummy.so" failed with exit status 1

Full log: https://gist.github.com/cdeil/f81a893dc0e690a8bb7c

It looks to me like the numpy build is misconfigured, it tries to link against /usr/local/lib/libpython2.7.a instead of the Python from the ScienceTools?

Is this a known issue? Is there a workaround?

@kialio I think pip-install of packages with C extensions into the ScienceTools will always be brittle / only work sometimes. Shipping Fermipy and it's dependencies with the ScienceTools would help. Conda packages for ScienceTools would probably be the best solution for Python power-users, but also maybe a lot of work to build and maintain?

[cdeil]

I forgot to say: this is not really a Fermipy issue ... feel free to close if you don't want to handle ScienceTools install issues here and let me know if you think it's useful or not to ask about this via the fermi help form.

[woodmd]

Since these issues are somewhat fermipy specific (i.e. they are connected to how fermipy is trying to install its dependencies) I think it's useful to have a record of them here. That said we've already been in communication with the ST support people about getting the STs to work better with pip/conda so I think the relevant experts are already aware of these issues..

Unfortunately I don't think there's really a good workaround for the errors you were seeing. If you're using a ST binary distribution you could try the alternate installation instructions described here which use an external anaconda python instead of the STs python. I've verified that this works on both Scientific Linux 6 and Ubuntu Trusty. Long term it would probably be better for us to develop our own STs conda package but this might be an acceptable solution in the short term.

[woodmd]

Did you ever try the anaconda installation instructions? It would be a useful data point to know if these are working for other people.

[cdeil]

@robertazanin asked me to install Fermipy today on our Linux Server at work and I tried pip install astropy and pip install healpy again and got the same linking error. (because I had forgotten about this issue).

@woodmd - No, I haven't tried installing the Fermi Science Tools or Fermipy with conda.

@robertazanin - I don't have time today, could you give conda a try? http://fermipy.readthedocs.org/en/latest/install.html#installing-with-anaconda-python

[robertazanin]

I have tried to install fermypy with conda. After having installed the latest version of healpy, I have these warnings:

from fermipy.gtanalysis import GTAnalysis Warning in TUnixSystem::SetDisplay: DISPLAY not set, setting it to hin-1512a.dhcp.mpi-hd.mpg.de:0.0 healpy/pixelfunc.py:105: UserWarning: Warning: cannot import _healpy_pixel_lib module warnings.warn('Warning: cannot import _healpy_pixel_lib module') healpy/sphtfunc.py:34: UserWarning: Warning: cannot import _healpy_pixel_lib module warnings.warn('Warning: cannot import _healpy_pixel_lib module') healpy/sphtfunc.py:38: UserWarning: Warning: cannot import _healpy_fitsio_lib module warnings.warn('Warning: cannot import _healpy_fitsio_lib module') healpy/sphtfunc.py:42: UserWarning: Warning: cannot import _sphtools module warnings.warn('Warning: cannot import _sphtools module') healpy/init.py:58: UserWarning: Warning: cannot import query disc module warnings.warn('Warning: cannot import query disc module') healpy/init.py:62: UserWarning: Warning: cannot import pixelfunc module warnings.warn('Warning: cannot import pixelfunc module') healpy/init.py:69: UserWarning: Warning: cannot import _sphtools module warnings.warn('Warning: cannot import _sphtools module') healpy/init.py:77: UserWarning: Warning: cannot import pixel lib module warnings.warn('Warning: cannot import pixel lib module') healpy/projaxes.py:36: UserWarning: Warning: cannot import _healpy_pixel_lib module warnings.warn('Warning: cannot import _healpy_pixel_lib module') Plotter is MatPlotlib

[cdeil]

@woodmd - How important is healpy for fermipy? Is is possible / worth the effort to make it an optional dependency so that most analyses can be run without it?

@robertazanin - You could try and install one of the binary healpy conda packages that are available: https://anaconda.org/search?q=healpy I don't know if this will work, but you could try:

conda install -c https://conda.anaconda.org/lsst lsst-healpy

[woodmd]

Making this an optional dependency is possible but would require some non-negligible work to reorganize the code. We're also planning to add full support for healpix-based analysis in the near future so in the long term I think it would be better if we can try to resolve these issues with the healpy installation. Just out of curiosity is there some reason that astropy doesn't have built-in healpix support? It would seem very natural given that healpix is so widely used in astronomy.

I'm surprised that the pip install of healpy isn't working for you. I tested the pip installation on both ubuntu and SL6 and it's also how I install healpy for the travis builds. Can you give any additional details about your build environment? This is the same SL machine you were testing on before?

[cdeil]

is there some reason that astropy doesn't have built-in healpix support?

It's partly a licensing issue. HEALPIX is GPL, Astropy is BSD.

There's some very recent lobbying asking HEALPIX to reconsider: https://mail.scipy.org/pipermail/astropy/2016-January/004084.html It doesn't look like HEALPIX will become available under a liberal license though: http://sourceforge.net/p/healpix/mailman/message/34777818/

Another option might be to use HEALPIX functionality from WCSLIB 5 (which is LGPL and is used by Astropy) and expose that in astropy.wcs or reproject or some other Astropy core or affiliated Python package. Or to re-implement the HEALPIX functionality we need from scratch. See https://github.com/astrofrog/reproject/issues/83 if you're interested in this avenue.

Can you give any additional details about your build environment? This is the same SL machine you were testing on before?

As I said in the comment at the start, it's a SL 6 machine and the full build log is linked to from there.

Probably conda is the more promising solution to focus on than pip. Having healpy packages via https://anaconda.org/astropy/packages would be a good solution instead of starting our own conda build pipeline, no? See https://github.com/astropy/conda-builder-affiliated/issues/48

[woodmd]

It would be great to have healpy available through the astropy channel. As a proof of principle I created a fermipy channel that provides both healpy and fermipy packages (only linux-64). With this setup you should be able to install fermipy with:

conda config --add channels astropy conda config --add channels fermipy conda install fermipy

I created the conda packages on ubuntu-trusty but unfortunately the healpy conda packages doesn't seem to work if I switch to a different flavor of Linux (SL6). I'm a complete conda novice so I probably have some fundamental misunderstand about how conda builds work (I just followed the tutorial for making a conda skeleton from a pypi package).

[cdeil]

@robertazanin - Can you try if conda install -c fermipy healpy works, i.e. if after you can do python -c 'import healpy'?

@woodmd - Building binaries for Linux that work for different flavors is a bit of a dark art. Masters (e.g. the official conda packages from Continuum) use something like http://phusion.github.io/holy-build-box/ and hand-curated recipes how to build and link.

This is not what https://github.com/astropy/conda-builder-affiliated is doing though ... they build on the Ubuntu from travis-ci. Do you know if the Linux packages from the astropy channel work with the distros you have available for testing?

In any case ... I think if at all possible we should try and use https://github.com/astropy/conda-builder-affiliated or pool with some other existing efforts to build packages instead of starting to build our own, so that we don't have to learn all the conda build tricks and spend a lot of time ourselves. I'm sure that you could add fermipy and other dependencies it needs to https://github.com/astropy/conda-builder-affiliated .

[woodmd]

I agree that it would be better to have this done in a fully-automated way so if the astropy conda builder is sufficient for our purposes then we should certainly use it. If I want to add fermipy to conda-builder-affiliated do I need to anything more than make a PR or is there someone I need to contact first about getting fermipy officially designated as an astropy-affiliated package? If the astropy channel builds are all produced on travis-ci how do the windows and OSX builds get created?

I haven't seen any issues with packages from the astropy channel. However the only package I've really tested is wcsaxes which I believe is pure python and thus wouldn't necessarily expose these sorts of issues. Here's the traceback I get when I try to import the healpy conda build I made under SL6:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/vagrant/miniconda/envs/fermi-env/lib/python2.7/site-packages/healpy/__init__.py", line 42, in <module>
    from .sphtfunc import (anafast, map2alm,
  File "/home/vagrant/miniconda/envs/fermi-env/lib/python2.7/site-packages/healpy/sphtfunc.py", line 31, in <module>
    from . import _healpy_sph_transform_lib as sphtlib
ImportError: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /home/vagrant/miniconda/envs/fermi-env/lib/python2.7/site-packages/healpy/_healpy_sph_transform_lib.so)

I think this just reflects an incompatibility between the version of GLIBC that comes with ubuntu trusty vs. SL6 but I could be wrong.

[cdeil]

If I want to add fermipy to conda-builder-affiliated do I need to anything more than make a PR or is there someone I need to contact first about getting fermipy officially designated as an astropy-affiliated package?

There's some info here if you're considering to apply with fermipy to be an astropy-affiliated package: http://www.astropy.org/affiliated/

https://github.com/mwcraig has set up and is maintaining https://github.com/astropy/conda-builder-affiliated . He's super-helpful, I don't think it matters whether fermipy is an official astropy-affiliated package or not ... if you want a few packages built and are willing to help get the build set up via pull requests to that repo it's a good way to go. The first step is healpy though, right? That's already work in progress: https://github.com/astropy/conda-builder-affiliated/issues/48 If there's other packages you need as dependencies, you could start separate PRs in parallel.

If the astropy channel builds are all produced on travis-ci how do the windows and OSX builds get created?

Linux and Mac builds are generated on travis-ci. Windows builds on Appveyor (for the packages that support Windows).

You can see the setup here and check out one of the closed pull requests to see the builds that ran: https://github.com/astropy/conda-builder-affiliated

[cdeil]

One more thing. Triggered by this recent message https://mail.scipy.org/pipermail/numpy-discussion/2016-January/074560.html there is now an effort to build Linux binaries for pip (called wheels) and distribute them via PyPI: https://mail.scipy.org/pipermail/numpy-discussion/2016-February/074841.html

You could test and see if those work for you. My understanding is that the experts agree that conda is the better solution though, so probably it makes sense to try hard to get conda to work.

[woodmd]

Ok I'll wait for the healpy conda package before I take any further action with fermipy. In any event creating a fermipy conda package should be pretty trivial once we have its dependencies available as conda packages.

I was aware of the wheel format and in fact I think pip is already installing these automatically if the package provides them. For instance if I pip install healpy on linux-64 it appears to be installing a wheel rather than building from source.

I found that installing healpy with pip appears to give a working healpy build on both ubuntu trusty and SL6 so I've updated the fermipy installation instructions to use pip for both the healpy and fermipy part of the installation. I think this should finally address the issue you had with getting fermipy installed on a SL6 machine. Note that you have to be careful not to install any of the other package dependencies with pip (astropy, numpy, etc.) as pip will attempt (and fail) to build these packages from source. If you run the install script it should take care of installing everything in the correct order.

[cdeil]

Healpy is now available here via conda:

conda install -c openastronomy healpy

See https://github.com/OpenAstronomy/conda-channel and https://github.com/OpenAstronomy/conda-channel/pull/3

@woodmd - Add fermipy there now? Or wait until ScienceTools are available via conda?

[woodmd]

I gave it a try but unfortunately it looks there's no python 2.7 build (see https://anaconda.org/OpenAstronomy/healpy/files). I've opened an issue here: OpenAstronomy/conda-channel#5

Once healpy is working I think we should just move forward with a fermipy conda package. Creating a ST conda package is a fairly big undertaking (lots of complicated external dependencies) so I don't see it happening for another 3-6 months at the earliest. We also still need to figure out who would be responsible for the maintenance and testing of such a package.

How do you recommend that I go about making a fermipy conda package? Should I submit a PR to the OpenAstronomy repo?

[cdeil]

How do you recommend that I go about making a fermipy conda package? Should I submit a PR to the OpenAstronomy repo?

Yes.

[woodmd]

I'm closing this issue now since it should be mostly addressed by the improvements we've made to the installation process over the last year. Linux users can follow these instructions for using the STs with anaconda python. Given an existing ST+anaconda installation fermipy can be installed from the conda-forge channel with:

conda config --append channels conda-forge
conda install fermipy

This doesn't really solve the larger issue of being able to install the STs with conda. We're working on this but realistically it will probably be another 6 months to a year before we have a ST conda package.