landriu commented 2 years ago

Hi all,

I would like to install fermi-2.2.0 using mamba, with the goal to install fermipy afterwards, with following command :

**mamba create --name fermi-2.2.0-mamba -c conda-forge -c fermi python=3.9 fermitools=2.2.0**

but after the installation was completed, I did not get a working environment :

[landriu@sappcfermi] ~ > conda activate fermi-2.2.0-mamba You have LD_LIBRARY_PATH set. This might interfere with the correct functioning of conda and the Fermitools. You have PYTHONPATH set. This might interfere with the correct functioning of conda and the Fermitools. (fermi-2.2.0-mamba) [landriu@sappcfermi] ~ > python Python 3.9.13 | packaged by conda-forge | (main, May 27 2022, 16:58:50) [GCC 10.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. Using startup saving python history in ~/.pyhistory

import UnbinnedAnalysis Traceback (most recent call last): File "", line 1, in File "/dsm/manip/glast/anaconda3/envs/fermi-2.2.0-mamba/lib/python3.9/site-packages/fermitools/UnbinnedAnalysis.py", line 12, in import pyLikelihood as pyLike File "/dsm/manip/glast/anaconda3/envs/fermi-2.2.0-mamba/lib/python3.9/site-packages/fermitools/pyLikelihood.py", line 15, in import _pyLikelihood ImportError: libgsl.so.25: cannot open shared object file: No such file or directory

and command "mamba list gsl" gives :

[landriu@sappcfermi] ~ > mamba list gsl

              __    __    __    __
             /  \  /  \  /  \  /  \
            /    \/    \/    \/    \

███████████████/ /██/ /██/ /██/ /████████████████████████ / / \ / \ / \ / \ _ / / _/ _/ _/ \ o _, / _/ ___/ ` |/ ███╗ ███╗ █████╗ ███╗ ███╗██████╗ █████╗ ████╗ ████║██╔══██╗████╗ ████║██╔══██╗██╔══██╗ ██╔████╔██║███████║██╔████╔██║██████╔╝███████║ ██║╚██╔╝██║██╔══██║██║╚██╔╝██║██╔══██╗██╔══██║ ██║ ╚═╝ ██║██║ ██║██║ ╚═╝ ██║██████╔╝██║ ██║ ╚═╝ ╚═╝╚═╝ ╚═╝╚═╝ ╚═╝╚═════╝ ╚═╝ ╚═╝

    mamba (0.5.1) supported by @QuantStack

    GitHub:  https://github.com/TheSnakePit/mamba
    Twitter: https://twitter.com/QuantStack

█████████████████████████████████████████████████████████████

packages in environment at /dsm/manip/glast/anaconda3/envs/fermi-2.2.0-mamba:

#

Name Version Build Channel

gsl 2.7.1 h917eeb2_1
(fermi-2.2.0-mamba)

I wonder why library libgsl.so.25 is still mentionned ...

Can you please check what is happening ?

Areustle commented 2 years ago

Hey David, I suspect your pre-set LD_LIBRARY_PATH or PYTHONPATH is causing your issue.

You have LD_LIBRARY_PATH set. This might interfere with the correct functioning of conda and the Fermitools.
You have PYTHONPATH set. This might interfere with the correct functioning of conda and the Fermitools.

The GSL version 2.7 package on conda provides the library libgsl.25.{dylib,so}.

$  tar tvf gsl-2.7-h6e638da_0.tar.bz2 | grep libgsl                                                                                                                                                                                                                            
lrwxrwxr-x  0 501    20          0 Jul 15  2021 lib/libgsl.dylib -> libgsl.25.dylib
lrwxrwxr-x  0 501    20          0 Jul 15  2021 lib/libgslcblas.dylib -> libcblas.3.dylib
lrwxrwxr-x  0 501    20          0 Jul 15  2021 lib/libgslcblas.0.dylib -> libcblas.3.dylib
-rwxrwxr-x  0 501    20    2468784 Jul 15  2021 lib/libgsl.25.dylib
...

Which is linked against by libLikelihood

$ otool -L $CONDA_PREFIX/lib/libLikelihood.dylib                                                                                                                                                                                          
/miniconda3-latest/envs/fermi-2.2.0/lib/libLikelihood.dylib:
        ...
    @rpath/libgsl.25.dylib (compatibility version 27.0.0, current version 27.0.0)
        ...

Thus your linking problem doesn't lie with gsl or the fermitools, but with your environment. If python can't find a libgsl to link against that suggests you've got path pollution obfuscating the linker's view of the available libs.

landriu commented 2 years ago

Hi Alex,

I redid the installation, with , this time :

conda deactivate

unset PYTHONPATH

unset LD_LIBRARY_PATH

and then

mamba create --name fermi-2.2.0-mamba -c conda-forge -c fermi python=3.9 fermitools=2.2.0

.../...

@. ~ > conda activate fermi-2.2.0-mamba (fermi-2.2.0-mamba) @. ~ > python Python 3.9.13 | packaged by conda-forge | (main, May 27 2022, 16:58:50) [GCC 10.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. Using startup saving python history in ~/.pyhistory

import UnbinnedAnalysis Traceback (most recent call last): File "", line 1, in File "/dsm/manip/glast/anaconda3/envs/fermi-2.2.0-mamba/lib/python3.9/site-packages/fermitools/UnbinnedAnalysis.py", line 12, in import pyLikelihood as pyLike File "/dsm/manip/glast/anaconda3/envs/fermi-2.2.0-mamba/lib/python3.9/site-packages/fermitools/pyLikelihood.py", line 15, in import _pyLikelihood ImportError: libgsl.so.25: cannot open shared object file: No such file or directory

and I find the same error message, though I have before and after the installation :

(fermi-2.2.0-mamba) @. ~ > env | grep PYTHONPATH (fermi-2.2.0-mamba) @. ~ > env | grep LD_LIBRARY_PATH (fermi-2.2.0-mamba) @.*** ~ >

De : Alex Reustle @.***> Envoyé : jeudi 30 juin 2022 13:42:57 À : fermi-lat/Fermitools-conda Cc : Landriu David; Author Objet : Re: [fermi-lat/Fermitools-conda] Trying to install fermitools when using mamba (Issue #122)

Hey David, I suspect your pre-set LD_LIBRARY_PATH or PYTHONPATH is causing your issue.

You have LD_LIBRARY_PATH set. This might interfere with the correct functioning of conda and the Fermitools. You have PYTHONPATH set. This might interfere with the correct functioning of conda and the Fermitools.

The GSL version 2.7 package on condahttps://anaconda.org/conda-forge/gsl/files provides the library libgsl.25.{dylib,so}.

$ tar tvf gsl-2.7-h6e638da_0.tar.bz2 | grep libgsl lrwxrwxr-x 0 501 20 0 Jul 15 2021 lib/libgsl.dylib -> libgsl.25.dylib lrwxrwxr-x 0 501 20 0 Jul 15 2021 lib/libgslcblas.dylib -> libcblas.3.dylib lrwxrwxr-x 0 501 20 0 Jul 15 2021 lib/libgslcblas.0.dylib -> libcblas.3.dylib -rwxrwxr-x 0 501 20 2468784 Jul 15 2021 lib/libgsl.25.dylib ...

Which is linked against by libLikelihood

$ otool -L $CONDA_PREFIX/lib/libLikelihood.dylib /miniconda3-latest/envs/fermi-2.2.0/lib/libLikelihood.dylib: ... @rpath/libgsl.25.dylib (compatibility version 27.0.0, current version 27.0.0) ...

Thus your linking problem doesn't lie with gsl or the fermitools, but with your environment. If python can't find a libgsl to link against that suggests you've got path pollution obfuscating the linker's view of the available libs.

— Reply to this email directly, view it on GitHubhttps://github.com/fermi-lat/Fermitools-conda/issues/122#issuecomment-1171115587, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFN3RXZ5NT53ORHQWICIPZ3VRWB4DANCNFSM52IQA7YA. You are receiving this because you authored the thread.Message ID: @.***>

donhorner commented 2 years ago

Can you reinstall Fermitools 2.2.0 but add the “--override-channels” option as described here: https://docs.conda.io/projects/conda/en/latest/user-guide/concepts/channels.html#specifying-channels-when-installing-packages

That has helped in the past. The other thing that has fixed similar issues in reinstalling conda.

Areustle commented 2 years ago

I think I see what the issue is: Your GSL is from Anaconda, not conda-forge.

Mamba got you gsl 2.7.1 h917eeb2_1, from https://anaconda.org/anaconda/gsl/files, as opposed to gsl 2.7 he838d99_0 from https://anaconda.org/conda-forge/gsl/files (looks like you were one of only 7 people to download that tarball, including me!).

You can confirm this by checking for the library find $CONDA_PREFIX -name "libgsl*"

I think this is a bug in mamba, because the channel declarations -c conda-forge -c fermi are supposed to prioritize those channels when downloading new packages, and that gsl package is only 2 days old, though I suppose it's possible that your .condarc has some custom configuration that prioritizes the anaconda channels?

@donhorner is right, the --override-channels flag should fix this for this issue and any other packages that were also taken from anaconda.

landriu commented 2 years ago

Hi,

thanks to you both for these indications :

find $CONDA_PREFIX -name "libgsl*" gives :

@.** ~ > find $CONDA_PREFIX -name "libgsl" /dsm/manip/glast/anaconda3/envs/fermi-2.0.18/lib/libgslcblas.so.0.0.0 /dsm/manip/glast/anaconda3/envs/fermi-2.0.18/lib/libgslcblas.so /dsm/manip/glast/anaconda3/envs/fermi-2.0.18/lib/libgsl.so /dsm/manip/glast/anaconda3/envs/fermi-2.0.18/lib/libgsl.a /dsm/manip/glast/anaconda3/envs/fermi-2.0.18/lib/libgsl.so.25 /dsm/manip/glast/anaconda3/envs/fermi-2.0.18/lib/libgsl.so.25.0.0 /dsm/manip/glast/anaconda3/envs/fermi-2.0.18/lib/libgslcblas.so.0 /dsm/manip/glast/anaconda3/envs/fermi-2.0.18/lib/libgslcblas.a

I'll try tomorrow to use this override-channels.

Cheers,

David

De : Alex Reustle @.***> Envoyé : jeudi 30 juin 2022 15:59:29 À : fermi-lat/Fermitools-conda Cc : Landriu David; Author Objet : Re: [fermi-lat/Fermitools-conda] Trying to install fermitools when using mamba (Issue #122)

I think I see what the issue is: Your GSL is from Anaconda, not conda-forge.

Mamba got you gsl 2.7.1 h917eeb2_1, from https://anaconda.org/anaconda/gsl/files, as opposed to gsl 2.7 he838d99_0 from https://anaconda.org/conda-forge/gsl/files (looks like you were one of only 7 people to download that tarball, including me!).

You can confirm this by checking for the library find $CONDA_PREFIX -name "libgsl*"

I think this is a bug in mamba, because the channel declarations -c conda-forge -c fermi are supposed to prioritize those channels when downloading new packages, and that gsl package is only 2 days old, though I suppose it's possible that your .condarc has some custom configuration that prioritizes the anaconda channels?

@donhornerhttps://github.com/donhorner is right, the --override-channels command should fix this for this issue and any other packages that were also taken from anaconda.

— Reply to this email directly, view it on GitHubhttps://github.com/fermi-lat/Fermitools-conda/issues/122#issuecomment-1171256159, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFN3RX77SH7DJW7HGUX7XX3VRWR4DANCNFSM52IQA7YA. You are receiving this because you authored the thread.Message ID: @.***>

Areustle commented 2 years ago

Activate your fermi-2.2.0-mamba environment before running the find command, or alternatively run it as find /dsm/manip/glast/anaconda3/envs/fermi-2.2.0-mamba/lib -name "libgsl*"

landriu commented 2 years ago

This time, I got :

@.** ~ > find /dsm/manip/glast/anaconda3/envs/fermi-2.2.0-mamba/lib -name "libgsl" /dsm/manip/glast/anaconda3/envs/fermi-2.2.0-mamba/lib/libgslcblas.so.0.0.0 /dsm/manip/glast/anaconda3/envs/fermi-2.2.0-mamba/lib/libgsl.so.27.0.0 /dsm/manip/glast/anaconda3/envs/fermi-2.2.0-mamba/lib/libgslcblas.so /dsm/manip/glast/anaconda3/envs/fermi-2.2.0-mamba/lib/libgsl.so.27 /dsm/manip/glast/anaconda3/envs/fermi-2.2.0-mamba/lib/libgsl.so /dsm/manip/glast/anaconda3/envs/fermi-2.2.0-mamba/lib/libgsl.a /dsm/manip/glast/anaconda3/envs/fermi-2.2.0-mamba/lib/libgslcblas.so.0 /dsm/manip/glast/anaconda3/envs/fermi-2.2.0-mamba/lib/libgslcblas.a

De : Alex Reustle @.***> Envoyé : jeudi 30 juin 2022 17:17:32 À : fermi-lat/Fermitools-conda Cc : Landriu David; Author Objet : Re: [fermi-lat/Fermitools-conda] Trying to install fermitools when using mamba (Issue #122)

Activate your fermi-2.2.0-mamba environment before running the find command, or alternatively run it as find /dsm/manip/glast/anaconda3/envs/fermi-2.2.0-mamba/lib -name "libgsl*"

— Reply to this email directly, view it on GitHubhttps://github.com/fermi-lat/Fermitools-conda/issues/122#issuecomment-1171349717, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFN3RX37CFJB6UDTOGSO2HDVRW3AZANCNFSM52IQA7YA. You are receiving this because you authored the thread.Message ID: @.***>

Areustle commented 2 years ago

Yep, that's the library.

landriu commented 2 years ago

Hi Alex and Don,

I re-created fermi-2.2.0-mamba using the command : mamba create --name fermi-2.2.0-mamba -c conda-forge -c fermi python=3.9 fermitools=2.2.0 --override-channels

and could have this time :

import Unbinned

without errors.

For info, I could go further in fermipy installation doing then : mamba install -c conda-forge -c fermi healpy gammapy and pip install fermipy

Thank you again.

Cheers

fermi-lat / Fermitools-conda

Trying to install fermitools when using mamba #122

packages in environment at /dsm/manip/glast/anaconda3/envs/fermi-2.2.0-mamba:

Name Version Build Channel