search

fermi-lat / Fermitools-conda

Conda recipe files for the Fermi Sciencetools software analysis package: Fermitools
https://fermi.gsfc.nasa.gov/ssc/data/analysis/
BSD 3-Clause "New" or "Revised" License
34 stars 17 forks source link

import error in pyLike #100

Closed grburgess closed 3 years ago

grburgess commented 3 years ago

I am running some basic tests with Fermipy and upon import of pyLike, I get the following error:

~/work/threeML/threeML/threeML/plugins/FermipyLike.py in _expensive_imports_hook()
85
    213 def _expensive_imports_hook():
86
    214 
87
--> 215     from fermipy.gtanalysis import GTAnalysis
88
    216     from GtBurst.LikelihoodComponent import findGalacticTemplate, findIsotropicTemplate
89
    217 
90

91
/usr/share/miniconda/envs/test_env/lib/python3.7/site-packages/ROOT/_facade.py in _importhook(name, *args, **kwds)
92
    119                 except Exception:
93
    120                     pass
94
--> 121             return _orig_ihook(name, *args, **kwds)
95
    122         __builtin__.__import__ = _importhook
96
    123 
97

98
/usr/share/miniconda/envs/test_env/lib/python3.7/site-packages/fermipy/gtanalysis.py in <module>
99
     19 import fermipy.wcs_utils as wcs_utils
100
     20 import fermipy.fits_utils as fits_utils
101
---> 21 import fermipy.gtutils as gtutils
102
     22 import fermipy.srcmap_utils as srcmap_utils
103
     23 import fermipy.skymap as skymap
104

105
/usr/share/miniconda/envs/test_env/lib/python3.7/site-packages/ROOT/_facade.py in _importhook(name, *args, **kwds)
106
    119                 except Exception:
107
    120                     pass
108
--> 121             return _orig_ihook(name, *args, **kwds)
109
    122         __builtin__.__import__ = _importhook
110
    123 
111

112
/usr/share/miniconda/envs/test_env/lib/python3.7/site-packages/fermipy/gtutils.py in <module>
113
      4 from functools import wraps
114
      5 import numpy as np
115
----> 6 import pyLikelihood as pyLike
116
      7 from SrcModel import SourceModel
117
      8 from AnalysisBase import AnalysisBase
118

119
/usr/share/miniconda/envs/test_env/lib/python3.7/site-packages/ROOT/_facade.py in _importhook(name, *args, **kwds)
120
    119                 except Exception:
121
    120                     pass
122
--> 121             return _orig_ihook(name, *args, **kwds)
123
    122         __builtin__.__import__ = _importhook
124
    123 
125

126
/usr/share/miniconda/envs/test_env/lib/python3.7/site-packages/fermitools/pyLikelihood.py in <module>
127
     13     from . import _pyLikelihood
128
     14 else:
129
---> 15     import _pyLikelihood
130
     16 
131
     17 try:
132

133
/usr/share/miniconda/envs/test_env/lib/python3.7/site-packages/ROOT/_facade.py in _importhook(name, *args, **kwds)
134
    119                 except Exception:
135
    120                     pass
136
--> 121             return _orig_ihook(name, *args, **kwds)
137
    122         __builtin__.__import__ = _importhook
138
    123 
139

140
ImportError: libCLHEP-2.4.1.3.so: cannot open shared object file: No such file or directory
141
ImportError: libCLHEP-2.4.1.3.so: cannot open shared object file: No such file or directory
142

My guess is a ROOT library is missing. This is run on a clean install of Ubuntu via conda.

grburgess commented 3 years ago

Also, this is a python 3.7 install

ldivenere commented 3 years ago

I have the same error when importing pyLikelihood. Also in my case, I have a fresh instalation on Ubuntu. It seems that CLHEP is present, but with the wrong version. I can find version 2.4.4.0, while 2.4.1.3 is required.

(ft2-0-0) divenere@jupyterhub:~$ ls $CONDA_PREFIX/lib/libCLHEP*
/home/divenere/anaconda3/envs/ft2-0-0/lib/libCLHEP-2.4.4.0.a
/home/divenere/anaconda3/envs/ft2-0-0/lib/libCLHEP-2.4.4.0.so
/home/divenere/anaconda3/envs/ft2-0-0/lib/libCLHEP-Cast-2.4.4.0.a
/home/divenere/anaconda3/envs/ft2-0-0/lib/libCLHEP-Cast-2.4.4.0.so
/home/divenere/anaconda3/envs/ft2-0-0/lib/libCLHEP-Evaluator-2.4.4.0.a
/home/divenere/anaconda3/envs/ft2-0-0/lib/libCLHEP-Evaluator-2.4.4.0.so
/home/divenere/anaconda3/envs/ft2-0-0/lib/libCLHEP-Exceptions-2.4.4.0.a
/home/divenere/anaconda3/envs/ft2-0-0/lib/libCLHEP-Exceptions-2.4.4.0.so
/home/divenere/anaconda3/envs/ft2-0-0/lib/libCLHEP-GenericFunctions-2.4.4.0.a
/home/divenere/anaconda3/envs/ft2-0-0/lib/libCLHEP-GenericFunctions-2.4.4.0.so
/home/divenere/anaconda3/envs/ft2-0-0/lib/libCLHEP-Geometry-2.4.4.0.a
/home/divenere/anaconda3/envs/ft2-0-0/lib/libCLHEP-Geometry-2.4.4.0.so
/home/divenere/anaconda3/envs/ft2-0-0/lib/libCLHEP-Matrix-2.4.4.0.a
/home/divenere/anaconda3/envs/ft2-0-0/lib/libCLHEP-Matrix-2.4.4.0.so
/home/divenere/anaconda3/envs/ft2-0-0/lib/libCLHEP-Random-2.4.4.0.a
/home/divenere/anaconda3/envs/ft2-0-0/lib/libCLHEP-Random-2.4.4.0.so
/home/divenere/anaconda3/envs/ft2-0-0/lib/libCLHEP-RandomObjects-2.4.4.0.a
/home/divenere/anaconda3/envs/ft2-0-0/lib/libCLHEP-RandomObjects-2.4.4.0.so
/home/divenere/anaconda3/envs/ft2-0-0/lib/libCLHEP-RefCount-2.4.4.0.a
/home/divenere/anaconda3/envs/ft2-0-0/lib/libCLHEP-RefCount-2.4.4.0.so
/home/divenere/anaconda3/envs/ft2-0-0/lib/libCLHEP-Vector-2.4.4.0.a
/home/divenere/anaconda3/envs/ft2-0-0/lib/libCLHEP-Vector-2.4.4.0.so
/home/divenere/anaconda3/envs/ft2-0-0/lib/libCLHEP.a
/home/divenere/anaconda3/envs/ft2-0-0/lib/libCLHEP.so

In this installation I have ROOT 6.22/06:

(ft2-0-0) divenere@jupyterhub:~$ conda list root
# packages in environment at /home/divenere/anaconda3/envs/ft2-0-0:
#
# Name                    Version                   Build  Channel
root                      6.22.6           py37h07c7eef_0    conda-forge
root-binaries             6.22.6           py37h07c7eef_0    conda-forge
root-dependencies         6.22.6           py37h07c7eef_0    conda-forge
root_base                 6.22.6           py37h5ed0ae1_0    conda-forge

In an old installation (same fermitools version, but installed one month ago), I have ROOT 6.22/02, with libCLHEP-2.4.1.3.

grburgess commented 3 years ago

@ldivenere is this a clean install with the latest conda version as well?

ldivenere commented 3 years ago

yes, it was a clean install with the latest conda version. These are the details of the conda installed and of the linux system:

(ft2-0-0) divenere@jupyterhub:~$ conda --version
conda 4.9.2
(ft2-0-0) divenere@jupyterhub:~$ uname -a
Linux jupyterhub 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
grburgess commented 3 years ago

Thanks. Yeah, still cannot find a way to debug this..

paulray commented 3 years ago

conda is relatively new to me, but the more I use it the more I hate it.

kerrm commented 3 years ago

Doesn't solve the issue, but if you just conda install that particular version of CLHEP, it seems to work. (At least I was able to run basic command line fermi tools.)

ldivenere commented 3 years ago

Also in my case, I tried to downgrade both ROOT to version 6.22.2 and CLHEP to 2.4.1.3. I get the same errors as in issue #101, but fermitools and fermipy seem to work.

Areustle commented 3 years ago

I have built version 2.0.7.0 of the Fermitools to link against CLHEP 2.4.4.0 . the dev build is available on anaconda cloud. If @paulray is growing to hate conda, I suggest downloading and using mamba in it's place for faster dependency resolution and acquisition. Mamba implements a sane SAT solver algorithm in C++ rather than the slow loop-based python implementation that conda uses.

mamba create -n fermi -c conda-forge -c fermi/label/dev fermitools=2.0.7.0 python=3

paulray commented 3 years ago

I haven't tried mamba, but the 2.0.7 build does seem to work. Thanks! When will a version after 2.0 be released on the non-dev channel?

donhorner commented 3 years ago

I'm hoping we can get a release out soon with at least this patch and a few others. However, we're effectively short staffed at the moment and have some non-Fermitools related deadlines for the AAS meeting next week that we're dealing with, so I don't want to make any promises.

BordeauxDave commented 3 years ago

Hello y'all. I'm new at this. Installed miniconda and fermitools this afternoon following https://github.com/fermi-lat/Fermitools-conda/wiki/Installation-Instructions and the first thing I tried to run was gtselect, and it failed with

error while loading shared libraries: libCLHEP-2.4.1.3.so: cannot open shared object file: No such file or directory

So I simply found that I already had an old 2.4.1.0/lib/libCLHEP.so on my disk, and created a link calling it 2.4.1.3 and then defined LD_LIBRARY_PATH to point to the link. Inelegant, but gtselect is running. I'm only telling you this in the hope that confirmation of a different manifestation of the bug might help you.

nmirabal commented 3 years ago

Until we release a new version. The following appears to be working conda create -n fermi -c conda-forge -c fermi fermitools python=3 clhep=2.4.1.3

BordeauxDave commented 3 years ago

I was nervous about what "create" would do when I already had a "fermi", so first I did conda remove --name fermi --all and then your command, and now gtselect works without my LD_LIBRARY_PATH kluge.

donhorner commented 3 years ago

We've released Fermitools 2.0.8 which should fix this issue and several other things. Please check the Release Notes.

If you have the Fermitools 2 already installed, you can update to the new version using the command: conda update -c conda-forge  -c fermi fermitools and then reactivate your fermi environment for the changes to take effect.

grburgess commented 3 years ago

Thanks for that !