search

kevin218 / Eureka

Eureka! is a data reduction and analysis pipeline intended for time-series observations with JWST.
https://eurekadocs.readthedocs.io/
MIT License
56 stars 43 forks source link

[Bug]: Problem with theano and blas_info flag after fresh update #626

Closed rluquer closed 4 months ago

rluquer commented 4 months ago

FAQ check

Instrument

NIRSpec (Stages 1-3)

What happened?

After updating the code to the most recent version in the main branch, I get the following error related with theano and numpy. I tried patching myself replacing line 2621 in theano/link/c/cmodule.py with

blas_info = np.distutils.config.blas_ilp64_opt_info

but that raised additional errors related with mc3.

Additionally, I tried adding this to the first line of run_eureka.py, but did not work either

`import numpy as np

try: np.distutils.config.blas_opt_info = np.distutils.config.blas_ilp64_opt_info except Exception: pass `

Error traceback output

`(eureka) rafael@dune:~/WORK/JWST/GO3263/NRS1/controls$ python run_eureka.py Traceback (most recent call last): File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/theano/configparser.py", line 238, in fetch_val_for_key return self._theano_cfg.get(section, option) File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/configparser.py", line 781, in get d = self._unify_values(section, vars) File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/configparser.py", line 1152, in _unify_values raise NoSectionError(section) from None configparser.NoSectionError: No section: 'blas'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/theano/configparser.py", line 354, in get val_str = cls.fetch_val_for_key(self.name, delete_key=delete_key) File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/theano/configparser.py", line 242, in fetch_val_for_key raise KeyError(key) KeyError: 'blas__ldflags'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/rafael/WORK/JWST/GO3263/NRS1/controls/run_eureka.py", line 2, in import eureka.lib.plots File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/eureka/init.py", line 28, in from . import S5_lightcurve_fitting File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/eureka/S5_lightcurve_fitting/init.py", line 6, in from . import differentiable_models File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/eureka/S5_lightcurve_fitting/differentiable_models/init.py", line 1, in from .PyMC3Models import PyMC3Model, CompositePyMC3Model File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/eureka/S5_lightcurve_fitting/differentiable_models/PyMC3Models.py", line 6, in import theano File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/theano/init.py", line 83, in from theano import scalar, tensor File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/theano/tensor/init.py", line 20, in from theano.tensor import nnet # used for softmax, sigmoid, etc. File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/theano/tensor/nnet/init.py", line 3, in from . import opt File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/theano/tensor/nnet/opt.py", line 32, in from theano.tensor.nnet.conv import ConvOp, conv2d File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/theano/tensor/nnet/conv.py", line 20, in from theano.tensor import blas File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/theano/tensor/blas.py", line 163, in from theano.tensor.blas_headers import blas_header_text, blas_header_version File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/theano/tensor/blas_headers.py", line 1016, in if not config.blasldflags: File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/theano/configparser.py", line 358, in get val_str = self.default() File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/theano/link/c/cmodule.py", line 2621, in default_blas_ldflags blas_info = numpy.distutils.config.blas_opt_info AttributeError: module 'numpy.distutils.config__' has no attribute 'blas_opt_info'`

What operating system are you using?

Operating System: Ubuntu 22.04.3 LTS Kernel: Linux 6.2.0-39-generic Architecture: x86-64

What version of Python are you running?

Python 3.9.7

What Python packages do you have installed?

packages in environment at /home/rafael/anaconda3/envs/eureka:

Name Version Build Channel _libgcc_mutex 0.1 main
_openmp_mutex 5.1 1_gnu
aesara-theano-fallback 0.1.0 pypi_0 pypi alabaster 0.7.16 pypi_0 pypi arviz 0.12.1 pypi_0 pypi asciitree 0.3.3 pypi_0 pypi asdf 3.0.1 pypi_0 pypi asdf-astropy 0.5.0 pypi_0 pypi asdf-coordinates-schemas 0.2.0 pypi_0 pypi asdf-standard 1.0.3 pypi_0 pypi asdf-transform-schemas 0.4.0 pypi_0 pypi asdf-unit-schemas 0.1.0 pypi_0 pypi asdf-wcs-schemas 0.3.0 pypi_0 pypi asteval 0.9.31 pypi_0 pypi astraeus 0.3 pypi_0 pypi astropy 5.3.4 pypi_0 pypi astropy-healpix 1.0.2 pypi_0 pypi astroquery 0.4.6 pypi_0 pypi astroscrappy 1.1.0 pypi_0 pypi asttokens 2.4.1 pypi_0 pypi attrs 23.2.0 pypi_0 pypi babel 2.14.0 pypi_0 pypi batman-package 2.4.9 pypi_0 pypi bayesicfitting 3.2.0 pypi_0 pypi beautifulsoup4 4.12.3 pypi_0 pypi bokeh 2.4.3 pypi_0 pypi bottleneck 1.3.7 pypi_0 pypi ca-certificates 2023.12.12 h06a4308_0
cachetools 5.3.2 pypi_0 pypi ccdproc 2.4.1 pypi_0 pypi celerite 0.4.2 pypi_0 pypi celerite2 0.3.0 pypi_0 pypi certifi 2024.2.2 pypi_0 pypi cffi 1.16.0 pypi_0 pypi cftime 1.6.3 pypi_0 pypi charset-normalizer 3.3.2 pypi_0 pypi click 8.1.7 pypi_0 pypi cloudpickle 3.0.0 pypi_0 pypi contourpy 1.2.0 pypi_0 pypi corner 2.2.2 pypi_0 pypi crds 11.17.16 pypi_0 pypi cryptography 42.0.3 pypi_0 pypi cycler 0.12.1 pypi_0 pypi dask 2024.2.0 pypi_0 pypi decorator 5.1.1 pypi_0 pypi deprecat 2.1.1 pypi_0 pypi dill 0.3.8 pypi_0 pypi docutils 0.20.1 pypi_0 pypi drizzle 1.15.0 pypi_0 pypi dynesty 2.1.3 pypi_0 pypi emcee 3.1.4 pypi_0 pypi eureka 0.11.dev124+gfb1aaf3b pypi_0 pypi exceptiongroup 1.2.0 pypi_0 pypi executing 2.0.1 pypi_0 pypi exoplanet 0.5.3 pypi_0 pypi exoplanet-core 0.1.2 pypi_0 pypi exotic-ld 3.0.0 pypi_0 pypi fasteners 0.19 pypi_0 pypi fastprogress 1.0.3 pypi_0 pypi filelock 3.13.1 pypi_0 pypi fonttools 4.49.0 pypi_0 pypi fsspec 2024.2.0 pypi_0 pypi future 0.18.3 pypi_0 pypi george 0.4.1 pypi_0 pypi gwcs 0.20.0 pypi_0 pypi h5netcdf 1.3.0 pypi_0 pypi h5py 3.10.0 pypi_0 pypi html5lib 1.1 pypi_0 pypi idna 3.6 pypi_0 pypi imageio 2.34.0 pypi_0 pypi imagesize 1.4.1 pypi_0 pypi importlib-metadata 7.0.1 pypi_0 pypi importlib-resources 6.1.1 pypi_0 pypi iniconfig 2.0.0 pypi_0 pypi ipython 8.18.1 pypi_0 pypi jaraco-classes 3.3.1 pypi_0 pypi jedi 0.19.1 pypi_0 pypi jeepney 0.8.0 pypi_0 pypi jinja2 3.1.3 pypi_0 pypi jmespath 1.0.1 pypi_0 pypi jsonschema 4.17.3 pypi_0 pypi jwst 1.12.2 pypi_0 pypi keyring 24.3.0 pypi_0 pypi kiwisolver 1.4.5 pypi_0 pypi ld_impl_linux-64 2.38 h1181459_1
libffi 3.3 he6710b0_2
libgcc-ng 11.2.0 h1234567_1
libgomp 11.2.0 h1234567_1
libstdcxx-ng 11.2.0 h1234567_1
lmfit 1.2.2 pypi_0 pypi locket 1.0.0 pypi_0 pypi markupsafe 2.1.5 pypi_0 pypi matplotlib 3.8.3 pypi_0 pypi matplotlib-inline 0.1.6 pypi_0 pypi mc3 3.1.3 pypi_0 pypi more-itertools 10.2.0 pypi_0 pypi ncurses 6.4 h6a678d5_0
netcdf4 1.6.5 pypi_0 pypi networkx 3.2.1 pypi_0 pypi numcodecs 0.12.1 pypi_0 pypi numpy 1.22.4 pypi_0 pypi numpydoc 1.6.0 pypi_0 pypi opencv-python 4.7.0.72 pypi_0 pypi opencv-python-headless 4.7.0.72 pypi_0 pypi openssl 1.1.1w h7f8727e_0
packaging 23.2 pypi_0 pypi pandas 2.0.3 pypi_0 pypi parsley 1.3 pypi_0 pypi parso 0.8.3 pypi_0 pypi partd 1.4.1 pypi_0 pypi patsy 0.5.6 pypi_0 pypi pexpect 4.9.0 pypi_0 pypi photutils 1.8.0 pypi_0 pypi pillow 10.2.0 pypi_0 pypi pip 23.3.1 py39h06a4308_0
pluggy 1.4.0 pypi_0 pypi poppy 1.1.1 pypi_0 pypi prompt-toolkit 3.0.43 pypi_0 pypi psutil 5.9.8 pypi_0 pypi ptyprocess 0.7.0 pypi_0 pypi pure-eval 0.2.2 pypi_0 pypi pycparser 2.21 pypi_0 pypi pyerfa 2.0.1.1 pypi_0 pypi pygments 2.17.2 pypi_0 pypi pymc3 3.11.5 pypi_0 pypi pymc3-ext 0.1.1 pypi_0 pypi pyparsing 3.1.1 pypi_0 pypi pyrsistent 0.20.0 pypi_0 pypi pysynphot 2.0.0 pypi_0 pypi pytest 8.0.1 pypi_0 pypi python 3.9.7 h12debd9_1
pymc3-ext 0.1.1 pypi_0 pypi pyparsing 3.1.1 pypi_0 pypi pyrsistent 0.20.0 pypi_0 pypi pysynphot 2.0.0 pypi_0 pypi pytest 8.0.1 pypi_0 pypi python 3.9.7 h12debd9_1
python-dateutil 2.8.2 pypi_0 pypi pytz 2024.1 pypi_0 pypi pyvo 1.5 pypi_0 pypi pywavelets 1.4.1 pypi_0 pypi pyyaml 6.0.1 pypi_0 pypi readline 8.2 h5eee18b_0
reproject 0.13.0 pypi_0 pypi requests 2.31.0 pypi_0 pypi scikit-image 0.19.3 pypi_0 pypi scipy 1.7.3 pypi_0 pypi secretstorage 3.3.3 pypi_0 pypi semantic-version 2.10.0 pypi_0 pypi semver 3.0.2 pypi_0 pypi setuptools 68.2.2 py39h06a4308_0
setuptools-scm 8.0.4 pypi_0 pypi six 1.16.0 pypi_0 pypi snowballstemmer 2.2.0 pypi_0 pypi soupsieve 2.5 pypi_0 pypi spherical-geometry 1.3.1 pypi_0 pypi sphinx 7.2.6 pypi_0 pypi sphinxcontrib-applehelp 1.0.8 pypi_0 pypi sphinxcontrib-devhelp 1.0.6 pypi_0 pypi sphinxcontrib-htmlhelp 2.0.5 pypi_0 pypi sphinxcontrib-jsmath 1.0.1 pypi_0 pypi sphinxcontrib-qthelp 1.0.7 pypi_0 pypi sphinxcontrib-serializinghtml 1.1.10 pypi_0 pypi sqlite 3.41.2 h5eee18b_0
stack-data 0.6.3 pypi_0 pypi starry 1.2.0 pypi_0 pypi stcal 1.4.4 pypi_0 pypi stdatamodels 1.8.4 pypi_0 pypi stpipe 0.5.1 pypi_0 pypi stsci-image 2.3.5 pypi_0 pypi stsci-imagestats 1.7.0 pypi_0 pypi stsci-stimage 0.2.6 pypi_0 pypi svo-filters 0.4.4 pypi_0 pypi tabulate 0.9.0 pypi_0 pypi theano-pymc 1.1.2 pypi_0 pypi tifffile 2024.2.12 pypi_0 pypi tk 8.6.12 h1ccaba5_0
tomli 2.0.1 pypi_0 pypi toolz 0.12.1 pypi_0 pypi tornado 6.4 pypi_0 pypi tqdm 4.66.2 pypi_0 pypi traitlets 5.14.1 pypi_0 pypi tweakwcs 0.8.6 pypi_0 pypi typing-extensions 4.9.0 pypi_0 pypi tzdata 2024.1 pypi_0 pypi uncertainties 3.1.7 pypi_0 pypi urllib3 2.2.1 pypi_0 pypi wcwidth 0.2.13 pypi_0 pypi webencodings 0.5.1 pypi_0 pypi wheel 0.41.2 py39h06a4308_0
wiimatch 0.3.2 pypi_0 pypi wrapt 1.16.0 pypi_0 pypi xarray 2023.9.0 pypi_0 pypi xarray-einstats 0.6.0 pypi_0 pypi xz 5.4.5 h5eee18b_0
zarr 2.17.0 pypi_0 pypi zipp 3.17.0 pypi_0 pypi zlib 1.2.13 h5eee18b_0

Code of Conduct

taylorbell57 commented 4 months ago

Can you edit your message to include the full list of installed packages? It looks like you only got packages starting with p onward and are missing the first chunk of the alphabet (including numpy). I have an idea what's causing this, but need those version numbers to know for sure

rluquer commented 4 months ago

done, sorry i did not realize

rluquer commented 4 months ago

in case this is useful:

np.show_config() openblas64_info: libraries = ['openblas64', 'openblas64_'] library_dirs = ['/usr/local/lib'] language = c define_macros = [('HAVE_CBLAS', None), ('BLAS_SYMBOLSUFFIX', '64'), ('HAVE_BLAS_ILP64', None)] runtime_library_dirs = ['/usr/local/lib'] blas_ilp64_optinfo: libraries = ['openblas64', 'openblas64_'] library_dirs = ['/usr/local/lib'] language = c define_macros = [('HAVE_CBLAS', None), ('BLAS_SYMBOLSUFFIX', '64'), ('HAVE_BLAS_ILP64', None)] runtime_library_dirs = ['/usr/local/lib'] openblas64__lapackinfo: libraries = ['openblas64', 'openblas64_'] library_dirs = ['/usr/local/lib'] language = c define_macros = [('HAVE_CBLAS', None), ('BLAS_SYMBOLSUFFIX', '64'), ('HAVE_BLAS_ILP64', None), ('HAVE_LAPACKE', None)] runtime_library_dirs = ['/usr/local/lib'] lapack_ilp64_optinfo: libraries = ['openblas64', 'openblas64_'] library_dirs = ['/usr/local/lib'] language = c define_macros = [('HAVE_CBLAS', None), ('BLAS_SYMBOLSUFFIX', '64'), ('HAVE_BLAS_ILP64', None), ('HAVE_LAPACKE', None)] runtime_library_dirs = ['/usr/local/lib'] Supported SIMD extensions in this NumPy install: baseline = SSE,SSE2,SSE3 found = SSSE3,SSE41,POPCNT,SSE42,AVX,F16C,FMA3,AVX2 not found = AVX512F,AVX512CD,AVX512_KNL,AVX512_KNM,AVX512_SKX,AVX512_CLX,AVX512_CNL,AVX512_ICL

rluquer commented 4 months ago

and the error from mc3 if I change line 2621 in theano as above:

(eureka) rafael@dune:~$ python run_eureka.py WARN: Could not locate executable armflang Traceback (most recent call last): File "/home/rafael/run_eureka.py", line 2, in import eureka.lib.plots File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/eureka/init.py", line 28, in from . import S5_lightcurve_fitting File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/eureka/S5_lightcurve_fitting/init.py", line 12, in from . import fitters File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/eureka/S5_lightcurve_fitting/fitters.py", line 21, in from . import plots_s5 as plots File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/eureka/S5_lightcurve_fitting/plots_s5.py", line 5, in from mc3.stats import time_avg File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/mc3/init.py", line 4, in from .sampler_driver import File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/mc3/sampler_driver.py", line 20, in from .fit_driver import fit File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/mc3/fit_driver.py", line 11, in from . import stats as ms File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/mc3/stats/init.py", line 5, in from .stats import File "/home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/mc3/stats/stats.py", line 31, in import _binarray as ba ImportError: /home/rafael/anaconda3/envs/eureka/lib/python3.9/site-packages/mc3/lib/_binarray.cpython-39-x86_64-linux-gnu.so: undefined symbol: _ZGVbN2v_exp

rluquer commented 4 months ago

If I run Stage1 in my laptop (macOS Monterey 12.6.5) after a fresh git pull this problem does not occur.

taylorbell57 commented 4 months ago

Hmm, so I don't know how you installed things earlier, but if you've installed theano then you should have an upper-limit on numpy of <1.22 while you have 1.22.4 installed. The issue with line 2621 in theano/link/c/cmodule.py that you pointed out is a result of theano no longer being supported, and so numpy versions >=1.22 break theano. I appreciate you trying to troubleshoot the issue though, and perhaps there's a way for me to make a fork of theano that's compatible with more recent versions of numpy.

But as for your issue, you shouldn't encounter this if you do pip install 'eureka[pymc3]@git+https://github.com/kevin218/Eureka.git@main' which should impose all of the correct upper and lower limits on package version numbers. If that doesn't solve your issue, please be sure to let me know

rluquer commented 4 months ago

All good now! Not sure what happened either. You may close the issue, thank you so much Taylor!