Empty BLAS flags in mamba-installed v2.18.1 environment under Docker

michaelosthege commented 11 months ago

Description

We have a Dockerfile based on ubuntu:22.04 which creates a micromamba base environment and updates it with a buch of packages.

Some (including internal) packages are pip-installed, but I was careful to avoid uninstalls/reinstalls/downgrades of any compilation/NumPy/blas-related dependencies.

This is the result of a mamba list just prior to running a test script:

mamba list

``` Python 3.10.10 # packages in environment at /opt/mamba: # # Name Version Build Channel _libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 2_kmp_llvm conda-forge aiobotocore 1.1.2 py_0 conda-forge aiohttp 3.8.4 py310h1fa729e_0 conda-forge aioitertools 0.11.0 pyhd8ed1ab_0 conda-forge aiosignal 1.3.1 pyhd8ed1ab_0 conda-forge alembic 1.10.2 pypi_0 pypi alsa-lib 1.2.8 h166bdaf_0 conda-forge amqp 5.1.1 pypi_0 pypi anyio 3.6.2 pypi_0 pypi aom 3.5.0 h27087fc_0 conda-forge apache-airflow 2.5.2 pypi_0 pypi apache-airflow-flowitems 1.0.2 pypi_0 pypi apache-airflow-providers-amazon 7.3.0 pypi_0 pypi apache-airflow-providers-celery 3.1.0 pypi_0 pypi apache-airflow-providers-common-sql 1.3.4 pypi_0 pypi apache-airflow-providers-docker 3.5.1 pypi_0 pypi apache-airflow-providers-ftp 3.3.1 pypi_0 pypi apache-airflow-providers-http 4.2.0 pypi_0 pypi apache-airflow-providers-imap 3.1.1 pypi_0 pypi apache-airflow-providers-mysql 4.0.2 pypi_0 pypi apache-airflow-providers-slack 7.2.0 pypi_0 pypi apache-airflow-providers-sqlite 3.3.1 pypi_0 pypi apache-airflow-providers-ssh 3.5.0 pypi_0 pypi apispec 3.3.2 pypi_0 pypi argcomplete 3.0.5 pypi_0 pypi arviz 0.16.1 pyhd8ed1ab_1 conda-forge asgiref 3.6.0 pypi_0 pypi asn1crypto 1.5.1 pypi_0 pypi asttokens 2.4.1 pyhd8ed1ab_0 conda-forge async-timeout 4.0.2 pyhd8ed1ab_0 conda-forge atk-1.0 2.38.0 hd4edc92_1 conda-forge attr 2.5.1 h166bdaf_1 conda-forge attrs 22.2.0 pyh71513ae_0 conda-forge avro-python3 1.10.2 pypi_0 pypi babel 2.12.1 pypi_0 pypi bcrypt 4.0.1 pypi_0 pypi beautifulsoup4 4.12.0 pyha770c72_0 conda-forge betterproto 2.0.0b6 pypi_0 pypi billiard 3.6.4.0 pypi_0 pypi binutils_impl_linux-64 2.40 hf600244_0 conda-forge binutils_linux-64 2.40 hbdbef99_2 conda-forge black 23.11.0 pypi_0 pypi blas 2.116 openblas conda-forge blas-devel 3.9.0 16_linux64_openblas conda-forge bletl 1.4.1 pypi_0 pypi blinker 1.5 pypi_0 pypi boto3 1.26.104 pypi_0 pypi botocore 1.29.104 pypi_0 pypi brotli 1.1.0 hd590300_1 conda-forge brotli-bin 1.1.0 hd590300_1 conda-forge brotlipy 0.7.0 py310h5764c6d_1005 conda-forge bzip2 1.0.8 h7f98852_4 conda-forge c-ares 1.22.1 hd590300_0 conda-forge ca-certificates 2022.12.7 ha878542_0 conda-forge cached-property 1.5.2 hd8ed1ab_1 conda-forge cached_property 1.5.2 pyha770c72_1 conda-forge cachelib 0.9.0 pypi_0 pypi cachetools 5.3.2 pyhd8ed1ab_0 conda-forge cairo 1.16.0 ha61ee94_1014 conda-forge calibr8 7.1.0 pypi_0 pypi ---redacted---- 0.4.0 pypi_0 pypi cattrs 22.2.0 pypi_0 pypi celery 5.2.7 pypi_0 pypi certifi 2023.7.22 pypi_0 pypi cffi 1.15.1 py310h255011f_3 conda-forge chardet 5.1.0 py310hff52083_0 conda-forge charset-normalizer 2.0.12 pypi_0 pypi click 8.1.3 pypi_0 pypi click-didyoumean 0.3.0 pypi_0 pypi click-plugins 1.1.1 pypi_0 pypi click-repl 0.2.0 pypi_0 pypi clickclick 20.10.2 pypi_0 pypi clickhouse-driver 0.2.6 py310h2372a71_1 conda-forge cloudpickle 3.0.0 pyhd8ed1ab_0 conda-forge colorama 0.4.6 pyhd8ed1ab_0 conda-forge colorlog 4.8.0 pypi_0 pypi conda 23.1.0 py310hff52083_0 conda-forge conda-build 3.24.0 py310hff52083_0 conda-forge conda-package-handling 2.0.2 pyh38be061_0 conda-forge conda-package-streaming 0.7.0 pyhd8ed1ab_1 conda-forge configupdater 3.1.1 pypi_0 pypi connexion 2.14.2 pypi_0 pypi cons 0.4.6 pyhd8ed1ab_0 conda-forge contourpy 1.2.0 py310hd41b1e2_0 conda-forge cron-descriptor 1.2.35 pypi_0 pypi croniter 1.3.8 pypi_0 pypi cryptography 40.0.1 py310h34c0648_0 conda-forge csaps 1.1.0 pypi_0 pypi cycler 0.12.1 pyhd8ed1ab_0 conda-forge cyrus-sasl 2.1.27 h7604b24_5 conda-forge dask 2023.11.0 pypi_0 pypi dbus 1.13.6 h5008d03_3 conda-forge decorator 5.1.1 pyhd8ed1ab_0 conda-forge deprecated 1.2.13 pypi_0 pypi ---redacted---- 0.2.1 pypi_0 pypi ---redacted---- 0.8.2 pypi_0 pypi dill 0.3.6 pypi_0 pypi distributed 2023.11.0 pypi_0 pypi dnspython 2.3.0 pypi_0 pypi docker 6.0.1 pypi_0 pypi docutils 0.15.2 py310hff52083_6 conda-forge email-validator 1.3.1 pypi_0 pypi et_xmlfile 1.1.0 pyhd8ed1ab_0 conda-forge etuples 0.3.9 pyhd8ed1ab_0 conda-forge exceptiongroup 1.1.1 pypi_0 pypi executing 2.0.1 pyhd8ed1ab_0 conda-forge expat 2.5.0 hcb278e6_1 conda-forge fastprogress 1.0.3 pyhd8ed1ab_0 conda-forge ffmpeg 5.1.2 gpl_h8dda1f0_106 conda-forge fftw 3.3.10 nompi_hc118613_108 conda-forge filelock 3.10.7 pyhd8ed1ab_0 conda-forge flask 2.2.3 pypi_0 pypi flask-appbuilder 4.1.4 pypi_0 pypi flask-babel 2.0.0 pypi_0 pypi flask-caching 2.0.2 pypi_0 pypi flask-jwt-extended 4.4.4 pypi_0 pypi flask-login 0.6.2 pypi_0 pypi flask-session 0.4.0 pypi_0 pypi flask-sqlalchemy 2.5.1 pypi_0 pypi flask-wtf 1.1.1 pypi_0 pypi flower 1.2.0 pypi_0 pypi fmt 9.1.0 h924138e_0 conda-forge font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge font-ttf-inconsolata 3.000 h77eed37_0 conda-forge font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge font-ttf-ubuntu 0.83 h77eed37_1 conda-forge fontconfig 2.14.2 h14ed4e7_0 conda-forge fonts-conda-ecosystem 1 0 conda-forge fonts-conda-forge 1 0 conda-forge fonttools 4.45.1 py310h2372a71_0 conda-forge freetype 2.12.1 h267a509_2 conda-forge fribidi 1.0.10 h36c2ea0_0 conda-forge frozenlist 1.3.3 py310h5764c6d_0 conda-forge fsspec 2023.10.0 pypi_0 pypi gcc 12.3.0 h8d2909c_2 conda-forge gcc_impl_linux-64 12.3.0 he2b93b0_3 conda-forge gcc_linux-64 12.3.0 h76fc315_2 conda-forge gdk-pixbuf 2.42.10 h05c8ddd_0 conda-forge gettext 0.21.1 h27087fc_0 conda-forge giflib 5.2.1 h0b41bf4_3 conda-forge gittrail 0.1.1 pypi_0 pypi glib 2.78.1 hfc55251_1 conda-forge glib-tools 2.78.1 hfc55251_1 conda-forge glob2 0.7 py_0 conda-forge gmp 6.3.0 h59595ed_0 conda-forge gnutls 3.7.8 hf3e180e_0 conda-forge graphite2 1.3.13 h58526e2_1001 conda-forge graphviz 7.1.0 h2e5815a_0 conda-forge greenlet 2.0.2 pypi_0 pypi grpclib 0.4.6 pypi_0 pypi gst-plugins-base 1.21.3 h4243ec0_1 conda-forge gstreamer 1.21.3 h25f0c4b_1 conda-forge gstreamer-orc 0.4.34 hd590300_0 conda-forge gtk2 2.24.33 h90689f9_2 conda-forge gts 0.7.6 h977cf35_4 conda-forge gunicorn 20.1.0 pypi_0 pypi gxx 12.3.0 h8d2909c_2 conda-forge gxx_impl_linux-64 12.3.0 he2b93b0_3 conda-forge gxx_linux-64 12.3.0 h8a814eb_2 conda-forge h11 0.14.0 pypi_0 pypi h2 4.1.0 pypi_0 pypi h5netcdf 1.3.0 pyhd8ed1ab_0 conda-forge h5py 3.8.0 nompi_py310h0311031_100 conda-forge hagelkorn 1.2.3 pypi_0 pypi harfbuzz 6.0.0 h8e241bc_0 conda-forge hdf5 1.12.2 nompi_h4df4325_100 conda-forge hpack 4.0.0 pypi_0 pypi httpcore 0.16.3 pypi_0 pypi httpx 0.23.3 pypi_0 pypi humanize 4.6.0 pypi_0 pypi hyperframe 6.0.1 pypi_0 pypi icu 70.1 h27087fc_0 conda-forge idna 3.3 pypi_0 pypi imageio 2.31.5 pyh8c1a49c_0 conda-forge imageio-ffmpeg 0.4.9 pyhd8ed1ab_0 conda-forge importlib-metadata 6.8.0 pyha770c72_0 conda-forge importlib_metadata 6.8.0 hd8ed1ab_0 conda-forge inflection 0.5.1 pypi_0 pypi iniconfig 2.0.0 pypi_0 pypi ipython 8.18.0 pyh0d859eb_0 conda-forge iso3166 2.1.1 pyhd8ed1ab_0 conda-forge isort 5.12.0 pypi_0 pypi itsdangerous 2.1.2 pypi_0 pypi jack 1.9.22 h11f4161_0 conda-forge jax 0.4.19 pyhd8ed1ab_0 conda-forge jaxlib 0.4.19 cpu_py310h9ed8a0c_0 conda-forge jedi 0.19.1 pyhd8ed1ab_0 conda-forge jinja2 3.1.2 pyhd8ed1ab_1 conda-forge jmespath 0.10.0 pyh9f0ad1d_0 conda-forge joblib 1.3.2 pyhd8ed1ab_0 conda-forge jpeg 9e h0b41bf4_3 conda-forge jsonpath-ng 1.5.3 pypi_0 pypi jsonschema 4.17.3 pypi_0 pypi kernel-headers_linux-64 2.6.32 he073ed8_16 conda-forge keyutils 1.6.1 h166bdaf_0 conda-forge kiwisolver 1.4.5 py310hd41b1e2_1 conda-forge kombu 5.2.4 pypi_0 pypi krb5 1.19.3 h08a2579_0 conda-forge lame 3.100 h166bdaf_1003 conda-forge lazy-object-proxy 1.9.0 pypi_0 pypi lcms2 2.15 hfd0df8a_0 conda-forge ld_impl_linux-64 2.40 h41732ed_0 conda-forge ldap3 2.9.1 pypi_0 pypi lerc 4.0.0 h27087fc_0 conda-forge libabseil 20230802.1 cxx17_h59595ed_0 conda-forge libarchive 3.6.2 h3d51595_0 conda-forge libblas 3.9.0 16_linux64_openblas conda-forge libbrotlicommon 1.1.0 hd590300_1 conda-forge libbrotlidec 1.1.0 hd590300_1 conda-forge libbrotlienc 1.1.0 hd590300_1 conda-forge libcap 2.67 he9d0100_0 conda-forge libcblas 3.9.0 16_linux64_openblas conda-forge libclang 15.0.7 default_h7634d5b_3 conda-forge libclang13 15.0.7 default_h9986a30_3 conda-forge libcups 2.3.3 h3e49a29_2 conda-forge libcurl 7.86.0 h2283fc2_1 conda-forge libdb 6.2.32 h9c3ff4c_0 conda-forge libdeflate 1.17 h0b41bf4_0 conda-forge libdrm 2.4.114 h166bdaf_0 conda-forge libedit 3.1.20191231 he28a2e2_2 conda-forge libev 4.33 h516909a_1 conda-forge libevent 2.1.10 h28343ad_4 conda-forge libexpat 2.5.0 hcb278e6_1 conda-forge libffi 3.4.2 h7f98852_5 conda-forge libflac 1.4.3 h59595ed_0 conda-forge libgcc-devel_linux-64 12.3.0 h8bca6fd_103 conda-forge libgcc-ng 13.2.0 h807b86a_3 conda-forge libgcrypt 1.10.2 hd590300_0 conda-forge libgd 2.3.3 h5aea950_4 conda-forge libgfortran-ng 13.2.0 h69a702a_3 conda-forge libgfortran5 13.2.0 ha4646dd_3 conda-forge libglib 2.78.1 h783c2da_1 conda-forge libgomp 13.2.0 h807b86a_3 conda-forge libgpg-error 1.47 h71f35ed_0 conda-forge libgrpc 1.58.2 he06187c_0 conda-forge libhwloc 2.9.1 hd6dc26d_0 conda-forge libiconv 1.17 h166bdaf_0 conda-forge libidn2 2.3.4 h166bdaf_0 conda-forge liblapack 3.9.0 16_linux64_openblas conda-forge liblapacke 3.9.0 16_linux64_openblas conda-forge liblief 0.12.3 h27087fc_0 conda-forge libllvm14 14.0.6 hcd5def8_4 conda-forge libllvm15 15.0.7 hadd5161_1 conda-forge libmamba 1.1.0 hde2b089_3 conda-forge libmambapy 1.1.0 py310h1428755_3 conda-forge libnghttp2 1.52.0 h61bc06f_0 conda-forge libnsl 2.0.0 h7f98852_0 conda-forge libntlm 1.4 h7f98852_1002 conda-forge libogg 1.3.4 h7f98852_1 conda-forge libopenblas 0.3.21 pthreads_h78a6416_3 conda-forge libopus 1.3.1 h7f98852_1 conda-forge libpciaccess 0.17 h166bdaf_0 conda-forge libpng 1.6.39 h753d276_0 conda-forge libpq 15.1 h67c24c5_1 conda-forge libprotobuf 4.24.3 hf27288f_1 conda-forge libre2-11 2023.06.02 h7a70373_0 conda-forge librsvg 2.54.4 h7abd40a_0 conda-forge libsanitizer 12.3.0 h0f45ef3_3 conda-forge libsndfile 1.2.2 hc60ed4a_1 conda-forge libsolv 0.7.23 h3eb15da_0 conda-forge libsqlite 3.40.0 h753d276_0 conda-forge libssh2 1.10.0 hf14f497_3 conda-forge libstdcxx-devel_linux-64 12.3.0 h8bca6fd_103 conda-forge libstdcxx-ng 13.2.0 h7e041cc_3 conda-forge libsystemd0 253 h8c4010b_1 conda-forge libtasn1 4.19.0 h166bdaf_0 conda-forge libtiff 4.5.0 h6adf6a1_2 conda-forge libtool 2.4.7 h27087fc_0 conda-forge libudev1 253 h0b41bf4_1 conda-forge libunistring 0.9.10 h7f98852_0 conda-forge libuuid 2.32.1 h7f98852_1000 conda-forge libva 2.18.0 h0b41bf4_0 conda-forge libvorbis 1.3.7 h9c3ff4c_0 conda-forge libvpx 1.11.0 h9c3ff4c_3 conda-forge libwebp 1.2.4 h1daa5a0_1 conda-forge libwebp-base 1.2.4 h166bdaf_0 conda-forge libxcb 1.13 h7f98852_1004 conda-forge libxkbcommon 1.5.0 h79f4944_1 conda-forge libxml2 2.10.3 hca2bb57_4 conda-forge libzlib 1.2.13 h166bdaf_4 conda-forge linkify-it-py 2.0.0 pypi_0 pypi llvm-openmp 16.0.6 h4dfa4b3_0 conda-forge llvmlite 0.40.1 py310h1b8f574_0 conda-forge locket 1.0.0 pypi_0 pypi lockfile 0.12.2 pypi_0 pypi logical-unification 0.4.6 pyhd8ed1ab_0 conda-forge lxml 4.9.3 pypi_0 pypi lz4-c 1.9.4 hcb278e6_0 conda-forge lzo 2.10 h516909a_1000 conda-forge mako 1.2.4 pypi_0 pypi mamba 1.1.0 py310h51d5547_3 conda-forge markdown 3.4.3 pypi_0 pypi markdown-it-py 2.2.0 pypi_0 pypi markupsafe 2.1.2 py310h1fa729e_0 conda-forge marshmallow 3.19.0 pypi_0 pypi marshmallow-enum 1.5.1 pypi_0 pypi marshmallow-oneofschema 3.0.1 pypi_0 pypi marshmallow-sqlalchemy 0.26.1 pypi_0 pypi matplotlib 3.8.2 py310hff52083_0 conda-forge matplotlib-base 3.8.2 py310h62c0568_0 conda-forge matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge mcbackend 0.5.1 pypi_0 pypi mdit-py-plugins 0.3.5 pypi_0 pypi mdurl 0.1.2 pypi_0 pypi minikanren 1.0.3 pyhd8ed1ab_0 conda-forge mkl 2022.2.1 h84fe81f_16997 conda-forge mkl-service 2.4.0 py310h9263142_0 conda-forge ml_dtypes 0.3.1 py310hcc13569_2 conda-forge mpg123 1.32.3 h59595ed_0 conda-forge ---redacted---- 0.1.4 pypi_0 pypi msgpack 1.0.7 pypi_0 pypi multidict 6.0.4 py310h1fa729e_0 conda-forge multipledispatch 0.6.0 py_0 conda-forge munkres 1.1.4 pyh9f0ad1d_0 conda-forge murefi 5.3.0 pypi_0 pypi mypy-boto3-appflow 1.26.78 pypi_0 pypi mypy-boto3-rds 1.26.102 pypi_0 pypi mypy-boto3-redshift-data 1.26.88 pypi_0 pypi mypy-extensions 1.0.0 pypi_0 pypi mysql-common 8.0.33 hf1915f5_2 conda-forge mysql-connector-python 8.0.32 pypi_0 pypi mysql-libs 8.0.33 hca2cd23_2 conda-forge mysqlclient 2.1.1 pypi_0 pypi ncurses 6.3 h27087fc_1 conda-forge ndg-httpsclient 0.5.1 py_1 conda-forge nest-asyncio 1.5.8 pypi_0 pypi nettle 3.8.1 hc379101_1 conda-forge nspr 4.35 h27087fc_0 conda-forge nss 3.89 he45b914_0 conda-forge numba 0.57.1 py310h0f6aa51_0 conda-forge numpy 1.24.4 py310ha4c1d20_0 conda-forge numpyro 0.13.2 pyhd8ed1ab_1 conda-forge openblas 0.3.21 pthreads_h320a7e8_3 conda-forge openh264 2.3.1 hcb278e6_2 conda-forge openjpeg 2.5.0 hfec8fc6_2 conda-forge openldap 2.4.59 hc311a53_0 conda-forge openpyxl 3.1.2 py310h2372a71_1 conda-forge openssl 3.2.0 hd590300_0 conda-forge opt-einsum 3.3.0 hd8ed1ab_2 conda-forge opt_einsum 3.3.0 pyhc1e730c_2 conda-forge p11-kit 0.24.1 hc5aa10d_0 conda-forge packaging 23.0 pyhd8ed1ab_0 conda-forge pandas 1.5.3 py310h9b08913_1 conda-forge pango 1.50.14 hd33c08f_0 conda-forge paramiko 3.1.0 pypi_0 pypi parso 0.8.3 pyhd8ed1ab_0 conda-forge partd 1.4.1 pypi_0 pypi patch 2.7.6 h7f98852_1002 conda-forge patchelf 0.17.2 h58526e2_0 conda-forge pathspec 0.9.0 pypi_0 pypi patsy 0.5.3 pyhd8ed1ab_0 conda-forge pcre2 10.42 hcad00b1_0 conda-forge ---redacted---- 0.6.1 pypi_0 pypi pendulum 2.1.2 pypi_0 pypi pexpect 4.8.0 pyh1a96a4e_2 conda-forge pickleshare 0.7.5 py_1003 conda-forge pillow 9.4.0 py310h023d228_1 conda-forge pip 23.3.1 pyhd8ed1ab_0 conda-forge pixman 0.42.2 h59595ed_0 conda-forge pkginfo 1.9.6 pyhd8ed1ab_0 conda-forge platformdirs 4.0.0 pypi_0 pypi pluggy 1.0.0 pyhd8ed1ab_5 conda-forge ply 3.11 pypi_0 pypi prison 0.2.1 pypi_0 pypi prometheus-client 0.16.0 pypi_0 pypi prompt-toolkit 3.0.38 pypi_0 pypi prompt_toolkit 3.0.41 hd8ed1ab_0 conda-forge protobuf 3.20.3 pypi_0 pypi psutil 5.9.4 py310h5764c6d_0 conda-forge psycopg2-binary 2.9.9 pypi_0 pypi pthread-stubs 0.4 h36c2ea0_1001 conda-forge ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge pulseaudio 16.1 hcb278e6_3 conda-forge pulseaudio-client 16.1 h5195f5e_3 conda-forge pulseaudio-daemon 16.1 ha8d29e2_3 conda-forge pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge py-lief 0.12.3 py310hd8f1fbe_0 conda-forge pyasn1 0.4.8 py_0 conda-forge pyasn1-modules 0.2.7 py_0 conda-forge pybind11-abi 4 hd8ed1ab_3 conda-forge pycosat 0.6.4 py310h5764c6d_1 conda-forge pycparser 2.21 pyhd8ed1ab_0 conda-forge pygments 2.14.0 pypi_0 pypi pyjwt 2.6.0 pypi_0 pypi pymc 5.10.0 hd8ed1ab_0 conda-forge pymc-base 5.10.0 pyhd8ed1ab_0 conda-forge pynacl 1.5.0 pypi_0 pypi pyopenssl 23.1.1 pyhd8ed1ab_0 conda-forge pyparsing 3.1.1 pyhd8ed1ab_0 conda-forge pyqt 5.15.7 py310hab646b1_3 conda-forge pyqt5-sip 12.11.0 py310heca2aa9_3 conda-forge pyrff 2.0.2 pypi_0 pypi pyrsistent 0.19.3 pypi_0 pypi pysocks 1.7.1 pyha2e5f31_6 conda-forge pytensor 2.18.1 py310hc6cd4ac_0 conda-forge pytensor-base 2.18.1 py310hcc13569_0 conda-forge pytensor-federated 1.0.1 pypi_0 pypi pytest 7.4.3 pypi_0 pypi python 3.10.10 he550d4f_0_cpython conda-forge python-daemon 3.0.1 pypi_0 pypi python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge python-dotenv 1.0.0 pypi_0 pypi python-graphviz 0.20.1 pyh22cad53_0 conda-forge python-ldap 3.4.0 py310h1fa729e_3 conda-forge python-libarchive-c 4.0 py310hff52083_2 conda-forge python-nvd3 0.15.0 pypi_0 pypi python-slugify 8.0.1 pypi_0 pypi python_abi 3.10 3_cp310 conda-forge pytz 2022.1 pypi_0 pypi pytzdata 2020.1 pypi_0 pypi pyyaml 6.0 py310h5764c6d_5 conda-forge qt-main 5.15.6 hafeba50_4 conda-forge re2 2023.06.02 h2873b5e_0 conda-forge readline 8.2 h8228510_1 conda-forge redis 3.5.3 pypi_0 pypi redshift-connector 2.0.910 pypi_0 pypi reproc 14.2.4 h0b41bf4_0 conda-forge reproc-cpp 14.2.4 hcb278e6_0 conda-forge requests 2.31.0 pypi_0 pypi requests-toolbelt 0.10.1 pypi_0 pypi retl 0.3.3 pypi_0 pypi rfc3339-validator 0.1.4 pypi_0 pypi rfc3986 1.5.0 pypi_0 pypi rich 13.3.3 pypi_0 pypi ripgrep 13.0.0 h2f28480_2 conda-forge robotools 1.7.3 pypi_0 pypi ruamel.yaml 0.17.21 py310h1fa729e_3 conda-forge ruamel.yaml.clib 0.2.7 py310h1fa729e_1 conda-forge s3fs 2021.7.0 pyhd8ed1ab_0 conda-forge s3transfer 0.6.0 pypi_0 pypi scikit-learn 1.3.2 py310h1fdf081_1 conda-forge scipy 1.11.4 py310hb13e2d6_0 conda-forge scramp 1.4.4 pypi_0 pypi setproctitle 1.3.2 pypi_0 pypi setuptools 65.6.3 pyhd8ed1ab_0 conda-forge sip 6.7.12 py310hc6cd4ac_0 conda-forge six 1.16.0 pyh6c4a22f_0 conda-forge slack-sdk 3.20.2 pypi_0 pypi sniffio 1.3.0 pypi_0 pypi sortedcontainers 2.4.0 pypi_0 pypi soupsieve 2.3.2.post1 pyhd8ed1ab_0 conda-forge sqlalchemy 1.4.46 pypi_0 pypi sqlalchemy-jsonfield 1.0.1.post0 pypi_0 pypi sqlalchemy-redshift 0.8.13 pypi_0 pypi sqlalchemy-utils 0.40.0 pypi_0 pypi sqlparse 0.4.3 pypi_0 pypi sshtunnel 0.4.0 pypi_0 pypi stack_data 0.6.2 pyhd8ed1ab_0 conda-forge statsmodels 0.14.0 py310h1f7b6fc_2 conda-forge stumpy 1.12.0 pypi_0 pypi svt-av1 1.4.1 hcb278e6_0 conda-forge sysroot_linux-64 2.12 he073ed8_16 conda-forge tabulate 0.9.0 pypi_0 pypi tbb 2021.9.0 hf52228f_0 conda-forge tblib 3.0.0 pypi_0 pypi tenacity 8.2.2 pypi_0 pypi termcolor 2.2.0 pypi_0 pypi text-unidecode 1.3 pypi_0 pypi threadpoolctl 3.2.0 pyha21a80b_0 conda-forge tk 8.6.13 noxft_h4845f30_101 conda-forge toml 0.10.2 pyhd8ed1ab_0 conda-forge tomli 2.0.1 pyhd8ed1ab_0 conda-forge toolz 0.12.0 pyhd8ed1ab_0 conda-forge tornado 6.2 pypi_0 pypi tqdm 4.65.0 pyhd8ed1ab_1 conda-forge traitlets 5.13.0 pyhd8ed1ab_0 conda-forge tsfresh 0.20.1 pypi_0 pypi typing 3.7.4.3 pypi_0 pypi typing-extensions 4.1.0 pypi_0 pypi tzdata 2023b h71feb2d_0 conda-forge tzlocal 5.2 py310hff52083_0 conda-forge uc-micro-py 1.0.1 pypi_0 pypi unicodecsv 0.14.1 pypi_0 pypi unicodedata2 15.1.0 py310h2372a71_0 conda-forge urllib3 1.26.8 pypi_0 pypi vine 5.0.0 pypi_0 pypi watchtower 2.0.1 pypi_0 pypi watermark 2.4.3 pyhd8ed1ab_0 conda-forge wcwidth 0.2.6 pypi_0 pypi websocket-client 1.5.1 pypi_0 pypi werkzeug 2.2.3 pypi_0 pypi wheel 0.40.0 pyhd8ed1ab_0 conda-forge wrapt 1.15.0 py310h1fa729e_0 conda-forge wtforms 3.0.1 pypi_0 pypi x264 1!164.3095 h166bdaf_2 conda-forge x265 3.5 h924138e_3 conda-forge xarray 2023.11.0 pyhd8ed1ab_0 conda-forge xarray-einstats 0.6.0 pyhd8ed1ab_0 conda-forge xcb-util 0.4.0 h516909a_0 conda-forge xcb-util-image 0.4.0 h166bdaf_0 conda-forge xcb-util-keysyms 0.4.0 h516909a_0 conda-forge xcb-util-renderutil 0.3.9 h166bdaf_0 conda-forge xcb-util-wm 0.4.1 h516909a_0 conda-forge xkeyboard-config 2.38 h0b41bf4_0 conda-forge xorg-fixesproto 5.0 h7f98852_1002 conda-forge xorg-kbproto 1.0.7 h7f98852_1002 conda-forge xorg-libice 1.0.10 h7f98852_0 conda-forge xorg-libsm 1.2.3 hd9c2040_1000 conda-forge xorg-libx11 1.8.4 h0b41bf4_0 conda-forge xorg-libxau 1.0.11 hd590300_0 conda-forge xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge xorg-libxext 1.3.4 h0b41bf4_2 conda-forge xorg-libxfixes 5.0.3 h7f98852_1004 conda-forge xorg-libxrender 0.9.10 h7f98852_1003 conda-forge xorg-renderproto 0.11.1 h7f98852_1002 conda-forge xorg-xextproto 7.3.0 h0b41bf4_1003 conda-forge xorg-xproto 7.0.31 h7f98852_1007 conda-forge xz 5.2.6 h166bdaf_0 conda-forge yaml 0.2.5 h7f98852_2 conda-forge yaml-cpp 0.7.0 h27087fc_2 conda-forge yarl 1.8.2 py310h5764c6d_0 conda-forge zict 3.0.0 pypi_0 pypi zipp 3.17.0 pyhd8ed1ab_0 conda-forge zlib 1.2.13 h166bdaf_4 conda-forge zstandard 0.19.0 py310hdeb6495_1 conda-forge zstd 1.5.2 h3eb15da_6 conda-forge ```

The test script tries this:

def test_pymc_sampling():
    """Checking that standard NUTS sampling works."""
    with pymc.Model():
        pymc.Normal("n")
        pymc.sample(tune=10, draws=20, compute_convergence_checks=False)
    print("✔ Can sample with NUTS.")
    return

if __name__ == "__main__":
    test_pymc_sampling()

and produces this error:

[2023-11-27 09:55:38,166] {blas_headers.py:1016} WARNING - Using NumPy C-API based implementation for BLAS functions.
[2023-11-27 09:55:41,284] {mcmc.py:677} WARNING - Only 20 samples in chain.
You can find the C code in this temporary file: /tmp/pytensor_compilation_error_gg4zy643
library inux-gnu/12.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: is not found.
library gcc: is not found.
[2023-11-27 09:55:43,422] {basic.py:1757} ERROR - Rewrite failure due to: constant_folding
[2023-11-27 09:55:43,423] {basic.py:1758} ERROR - node: Cast{float64}(0)
[2023-11-27 09:55:43,423] {basic.py:1759} ERROR - TRACEBACK:
[2023-11-27 09:55:43,425] {basic.py:1760} ERROR - Traceback (most recent call last):
  File "/opt/mamba/lib/python3.10/site-packages/pytensor/graph/rewriting/basic.py", line 1922, in process_node
    replacements = node_rewriter.transform(fgraph, node)
  File "/opt/mamba/lib/python3.10/site-packages/pytensor/graph/rewriting/basic.py", line 1082, in transform
    return self.fn(fgraph, node)
  File "/opt/mamba/lib/python3.10/site-packages/pytensor/tensor/rewriting/basic.py", line 1106, in constant_folding
    thunk = node.op.make_thunk(node, storage_map, compute_map, no_recycling=[])
  File "/opt/mamba/lib/python3.10/site-packages/pytensor/link/c/op.py", line 119, in make_thunk
    return self.make_c_thunk(node, storage_map, compute_map, no_recycling)
  File "/opt/mamba/lib/python3.10/site-packages/pytensor/link/c/op.py", line 84, in make_c_thunk
    outputs = cl.make_thunk(
  File "/opt/mamba/lib/python3.10/site-packages/pytensor/link/c/basic.py", line 1209, in make_thunk
    cthunk, module, in_storage, out_storage, error_storage = self.__compile__(
  File "/opt/mamba/lib/python3.10/site-packages/pytensor/link/c/basic.py", line 1129, in __compile__
    thunk, module = self.cthunk_factory(
  File "/opt/mamba/lib/python3.10/site-packages/pytensor/link/c/basic.py", line 1653, in cthunk_factory
    module = cache.module_from_key(key=key, lnk=self)
  File "/opt/mamba/lib/python3.10/site-packages/pytensor/link/c/cmodule.py", line 1231, in module_from_key
    module = lnk.compile_cmodule(location)
  File "/opt/mamba/lib/python3.10/site-packages/pytensor/link/c/basic.py", line 1552, in compile_cmodule
    module = c_compiler.compile_str(
  File "/opt/mamba/lib/python3.10/site-packages/pytensor/link/c/cmodule.py", line 2640, in compile_str
    raise CompileError(
pytensor.link.c.exceptions.CompileError: Compilation failed (return status=1):
/opt/mamba/bin/g++ -shared -g -O3 -fno-math-errno -Wno-unused-label -Wno-unused-variable -Wno-write-strings -Wno-c++11-narrowing -fno-exceptions -fno-unwind-tables -fno-asynchronous-unwind-tables -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -fPIC -I/opt/mamba/lib/python3.10/site-packages/numpy/core/include -I/opt/mamba/include/python3.10 -I/opt/mamba/lib/python3.10/site-packages/pytensor/link/c/c_code -L/opt/mamba/lib -fvisibility=hidden -o /usr/local/airflow/.pytensor/compiledir_Linux-5.4--generic-x86_64-with-glibc2.35-x86_64-3.10.10-64/tmpsau44d15/maf7dfc2b98019ae231490759913de9c7eae2cf26062201cb60a34079dffa7f98.so /usr/local/airflow/.pytensor/compiledir_Linux-5.4--generic-x86_64-with-glibc2.35-x86_64-3.10.10-64/tmpsau44d15/mod.cpp -lpython3.10
/opt/mamba/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: cannot find -lgcc: No such file or directory
collect2: error: ld returned 1 exit status
You can find the C code in this temporary file: /tmp/pytensor_compilation_error_osy3tkjn
library inux-gnu/12.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: is not found.
library gcc: is not found.

Yesterday I manually checked inside the image and found pytensor.config.blas__ldflags == ''.

At this point I'm out of ideas and Ricardo told me to tag @lucianopaz

What can I do to facilitate debugging?

lucianopaz commented 11 months ago

@michaelosthege, any luck debugging this on your own? I find it strange that the compilation error says that libgcc isn't being found, and to be honest, I don't know why this is happening. This seems to indicate that the problem might not be that pytensor can't find the blas flags, but that it doesn't manage to find a valid c compiler or that the c compiler that is being used doesn't understand the libgcc.so library that you have installed?

michaelosthege commented 11 months ago

@michaelosthege, any luck debugging this on your own? I find it strange that the compilation error says that libgcc isn't being found, and to be honest, I don't know why this is happening. This seems to indicate that the problem might not be that pytensor can't find the blas flags, but that it doesn't manage to find a valid c compiler or that the c compiler that is being used doesn't understand the libgcc.so library that you have installed?

not yet. Earliest time window I have for debugging is tomorrow evening.

michaelosthege commented 11 months ago

Crossposting my debugging attempts from the thread in https://github.com/pymc-devs/pytensor/pull/528

With a bit more detailed logging (https://github.com/michaelosthege/pytensor/tree/mode-cmodule-logging)

DEBUG:pytensor.link.c.cmodule:Checking MKL flags with intel threading
DEBUG:pytensor.link.c.cmodule:Compiling for 64 bit architecture
DEBUG:pytensor.link.c.cmodule:Test compilation with flags '['-lmkl_core', '-lmkl_rt', '-lmkl_intel_thread', '-liomp5', '-lpthread', '-Wl,-rpath,/opt/mamba/lib', '-L/opt/mamba/lib']' resulted in
(False, False)
DEBUG:pytensor.link.c.cmodule:Supplied flags '['-lmkl_core', '-lmkl_rt', '-lmkl_intel_thread', '-liomp5', '-lpthread', '-Wl,-rpath,/opt/mamba/lib']' failed to compile
DEBUG:pytensor.link.c.cmodule:Supplied flags ['-lmkl_core', '-lmkl_rt', '-lmkl_intel_thread', '-liomp5', '-lpthread', '-Wl,-rpath,/opt/mamba/lib'] failed to compile
DEBUG:pytensor.link.c.cmodule:Checking MKL flags with GNU OpenMP threading
DEBUG:pytensor.link.c.cmodule:Compiling for 64 bit architecture
DEBUG:pytensor.link.c.cmodule:Test compilation with flags '['-lmkl_core', '-lmkl_rt', '-lmkl_gnu_thread', '-lgomp', '-lpthread', '-Wl,-rpath,/opt/mamba/lib', '-L/opt/mamba/lib']' resulted in
(False, False)
DEBUG:pytensor.link.c.cmodule:Supplied flags '['-lmkl_core', '-lmkl_rt', '-lmkl_gnu_thread', '-lgomp', '-lpthread', '-Wl,-rpath,/opt/mamba/lib']' failed to compile
DEBUG:pytensor.link.c.cmodule:Supplied flags ['-lmkl_core', '-lmkl_rt', '-lmkl_gnu_thread', '-lgomp', '-lpthread', '-Wl,-rpath,/opt/mamba/lib'] failed to compile
DEBUG:pytensor.link.c.cmodule:Checking Lapack + blas
DEBUG:pytensor.link.c.cmodule:Compiling for 64 bit architecture
DEBUG:pytensor.link.c.cmodule:Test compilation with flags '['-llapack', '-lblas', '-lcblas', '-lm', '-Wl,-rpath,/opt/mamba/lib', '-L/opt/mamba/lib']' resulted in
(False, False)
DEBUG:pytensor.link.c.cmodule:Supplied flags '['-llapack', '-lblas', '-lcblas', '-lm', '-Wl,-rpath,/opt/mamba/lib']' failed to compile
DEBUG:pytensor.link.c.cmodule:Supplied flags ['-llapack', '-lblas', '-lcblas', '-lm', '-Wl,-rpath,/opt/mamba/lib'] failed to compile
DEBUG:pytensor.link.c.cmodule:Checking blas alone
DEBUG:pytensor.link.c.cmodule:Compiling for 64 bit architecture
DEBUG:pytensor.link.c.cmodule:Test compilation with flags '['-lblas', '-lcblas', '-Wl,-rpath,/opt/mamba/lib', '-L/opt/mamba/lib']' resulted in
(False, False)
DEBUG:pytensor.link.c.cmodule:Supplied flags '['-lblas', '-lcblas', '-Wl,-rpath,/opt/mamba/lib']' failed to compile
DEBUG:pytensor.link.c.cmodule:Supplied flags ['-lblas', '-lcblas', '-Wl,-rpath,/opt/mamba/lib'] failed to compile
DEBUG:pytensor.link.c.cmodule:Checking openblas
DEBUG:pytensor.link.c.cmodule:Compiling for 64 bit architecture
DEBUG:pytensor.link.c.cmodule:Test compilation with flags '['-lopenblas', '-lgfortran', '-lgomp', '-lm', '-fopenmp', '-Wl,-rpath,/opt/mamba/lib', '-L/opt/mamba/lib']' resulted in
(False, False)
DEBUG:pytensor.link.c.cmodule:Supplied flags '['-lopenblas', '-lgfortran', '-lgomp', '-lm', '-fopenmp', '-Wl,-rpath,/opt/mamba/lib']' failed to compile
DEBUG:pytensor.link.c.cmodule:Supplied flags ['-lopenblas', '-lgfortran', '-lgomp', '-lm', '-fopenmp', '-Wl,-rpath,/opt/mamba/lib'] failed to compile
DEBUG:pytensor.link.c.cmodule:Failed to identify blas ldflags. Will leave them empty.
''

lucianopaz commented 11 months ago

@michaelosthege, this still does not look like a blas flag issue. Your initial error when trying to run pymc.sample is that the linker, ld, can't find the library called libgcc.so. This is a more serious problem than not having blas flags, and it points to some problem with the image you are pulling or building. Could you have mistakenly deleted libgcc.so? Maybe you have a file called libgcc.so.1 and the one that ended in .so was simply a symlink that was deleted for some reason?

michaelosthege commented 11 months ago

I checked the diff of my merge request again, but even after taking out a change that removed RUN apt-get update -y && apt-get install openssh-client -y the error still persists.

The potentially relevant parts of my diff amount to:

environment.yml
- numpy>=1.24.2,<2 → numpy==1.24.4
- blas → ``
- libblas → ``
- pytensor-base>=2.11.0,<2.12 → pytensor>=2.18.1
- pip: pymc==5.9.2 → pymc==5.10.0

Essentially I switched from only installing pytensor-base, blas and libblas via conda-forge and PyMC from PyPI to installing everything from conda-forge. My thought here was that installs from conda-forge should be more reliable..

I will try to reverse these changes one by one.

maresb commented 11 months ago

Why not use micromamba-docker as your base image?

michaelosthege commented 11 months ago

Why not use micromamba-docker as your base image?

This particular one is a stack of Apache Airflow images, and I'm considering to switch to official Airflow images as the base (instead of ubuntu:22.04). I have micromamba as the base image in another project, but it's not necessarily simpler.

Also, I just fixed my build!

The trick was to NOT install pytensor or pymc through mamba, but instead do this:

channels:
- conda-forge
dependencies:
- numpy==1.24.4
- blas
- jax
- jaxlib >=0.3.7
- libblas
- mkl-service
- numba==0.57.1
- numpyro
- python-graphviz
- pytensor-base==2.18.1
- pip
- pip:
  - mcbackend==0.5.1
  - pymc==5.10.0
  - pytensor-federated==1.0.1

maresb commented 11 months ago

Hmm, do you have any insight as to why the pymc conda-forge package doesn't work for you?

michaelosthege commented 11 months ago

I don't know why, but here's a diff of the broken (-) vs. working (+) environments 👇

The difference in the environment.yml between these two was only the move of a pymc==5.10.0 into the pip section.

diff --git a/env_broken.yml b/env_working.yml
index 8942157..c38d7fd 100644
--- a/env_broken.yml
+++ b/env_working.yml
@@ -24,7 +24,7 @@ apache-airflow-providers-sqlite 3.3.1                    pypi_0    pypi     
 apache-airflow-providers-ssh 3.5.0                    pypi_0    pypi
 apispec                   3.3.2                    pypi_0    pypi
 argcomplete               3.0.5                    pypi_0    pypi
-arviz                     0.16.1             pyhd8ed1ab_1    conda-forge
+arviz                     0.16.1                   pypi_0    pypi
 asgiref                   3.6.0                    pypi_0    pypi
 asn1crypto                1.5.1                    pypi_0    pypi
 asttokens                 2.4.1              pyhd8ed1ab_0    conda-forge
@@ -38,8 +38,6 @@ bcrypt                    4.0.1                    pypi_0    pypi
 beautifulsoup4            4.12.0             pyha770c72_0    conda-forge
 betterproto               2.0.0b6                  pypi_0    pypi
 billiard                  3.6.4.0                  pypi_0    pypi
-binutils_impl_linux-64    2.40                 hf600244_0    conda-forge
-binutils_linux-64         2.40                 hbdbef99_2    conda-forge
 black                     23.11.0                  pypi_0    pypi
 blas                      2.120                  openblas    conda-forge
 blas-devel                3.9.0           20_linux64_openblas    conda-forge
@@ -53,10 +51,8 @@ brotlipy                  0.7.0           py310h5764c6d_1005    conda-forge
 bzip2                     1.0.8                h7f98852_4    conda-forge
 c-ares                    1.23.0               hd590300_0    conda-forge
 ca-certificates           2022.12.7            ha878542_0    conda-forge
-cached-property           1.5.2                hd8ed1ab_1    conda-forge
-cached_property           1.5.2              pyha770c72_1    conda-forge
 cachelib                  0.9.0                    pypi_0    pypi
-cachetools                5.3.2              pyhd8ed1ab_0    conda-forge
+cachetools                5.3.2                    pypi_0    pypi
 cairo                     1.16.0            ha61ee94_1014    conda-forge
 calibr8                   7.1.0                    pypi_0    pypi
 calibr8-contrib           0.4.0                    pypi_0    pypi
@@ -72,7 +68,7 @@ click-plugins             1.1.1                    pypi_0    pypi
 click-repl                0.2.0                    pypi_0    pypi
 clickclick                20.10.2                  pypi_0    pypi
 clickhouse-driver         0.2.6           py310h2372a71_1    conda-forge
-cloudpickle               3.0.0              pyhd8ed1ab_0    conda-forge
+cloudpickle               3.0.0                    pypi_0    pypi
 colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
 colorlog                  4.8.0                    pypi_0    pypi
 conda                     23.1.0          py310hff52083_0    conda-forge
@@ -106,7 +102,7 @@ etuples                   0.3.9              pyhd8ed1ab_0    conda-forge  
 exceptiongroup            1.1.1                    pypi_0    pypi
 executing                 2.0.1              pyhd8ed1ab_0    conda-forge
 expat                     2.5.0                hcb278e6_1    conda-forge
-fastprogress              1.0.3              pyhd8ed1ab_0    conda-forge
+fastprogress              1.0.3                    pypi_0    pypi
 ffmpeg                    5.1.2           gpl_h8dda1f0_106    conda-forge
 fftw                      3.3.10          nompi_hc118613_108    conda-forge
 filelock                  3.10.7             pyhd8ed1ab_0    conda-forge
@@ -133,9 +129,6 @@ freetype                  2.12.1               h267a509_2    conda-forge  
 fribidi                   1.0.10               h36c2ea0_0    conda-forge
 frozenlist                1.3.3           py310h5764c6d_0    conda-forge
 fsspec                    2023.12.0                pypi_0    pypi
-gcc                       12.3.0               h8d2909c_2    conda-forge
-gcc_impl_linux-64         12.3.0               he2b93b0_3    conda-forge
-gcc_linux-64              12.3.0               h76fc315_2    conda-forge
 gdk-pixbuf                2.42.10              h05c8ddd_0    conda-forge
 gettext                   0.21.1               h27087fc_0    conda-forge
 giflib                    5.2.1                h0b41bf4_3    conda-forge
@@ -155,16 +148,12 @@ gstreamer-orc             0.4.34               hd590300_0    conda-forge
 gtk2                      2.24.33              h90689f9_2    conda-forge
 gts                       0.7.6                h977cf35_4    conda-forge
 gunicorn                  20.1.0                   pypi_0    pypi
-gxx                       12.3.0               h8d2909c_2    conda-forge
-gxx_impl_linux-64         12.3.0               he2b93b0_3    conda-forge
-gxx_linux-64              12.3.0               h8a814eb_2    conda-forge
 h11                       0.14.0                   pypi_0    pypi
 h2                        4.1.0                    pypi_0    pypi
-h5netcdf                  1.3.0              pyhd8ed1ab_0    conda-forge
-h5py                      3.8.0           nompi_py310h0311031_100    conda-forge
+h5netcdf                  1.3.0                    pypi_0    pypi
+h5py                      3.10.0                   pypi_0    pypi
 hagelkorn                 1.2.3                    pypi_0    pypi
 harfbuzz                  6.0.0                h8e241bc_0    conda-forge
-hdf5                      1.12.2          nompi_h4df4325_100    conda-forge
 hpack                     4.0.0                    pypi_0    pypi
 httpcore                  0.16.3                   pypi_0    pypi
 httpx                     0.23.3                   pypi_0    pypi
@@ -192,7 +181,6 @@ joblib                    1.3.2              pyhd8ed1ab_0    conda-forge  
 jpeg                      9e                   h0b41bf4_3    conda-forge
 jsonpath-ng               1.5.3                    pypi_0    pypi
 jsonschema                4.17.3                   pypi_0    pypi
-kernel-headers_linux-64   2.6.32              he073ed8_16    conda-forge
 keyutils                  1.6.1                h166bdaf_0    conda-forge
 kiwisolver                1.4.5           py310hd41b1e2_1    conda-forge
 kombu                     5.2.4                    pypi_0    pypi
@@ -224,14 +212,13 @@ libevent                  2.1.10               h28343ad_4    conda-forge
 libexpat                  2.5.0                hcb278e6_1    conda-forge
 libffi                    3.4.2                h7f98852_5    conda-forge
 libflac                   1.4.3                h59595ed_0    conda-forge
-libgcc-devel_linux-64     12.3.0             h8bca6fd_103    conda-forge
-libgcc-ng                 13.2.0               h807b86a_3    conda-forge
+libgcc-ng                 12.2.0              h65d4601_19    conda-forge
 libgcrypt                 1.10.3               hd590300_0    conda-forge
 libgd                     2.3.3                h5aea950_4    conda-forge
-libgfortran-ng            13.2.0               h69a702a_3    conda-forge
-libgfortran5              13.2.0               ha4646dd_3    conda-forge
+libgfortran-ng            13.2.0               h69a702a_0    conda-forge
+libgfortran5              13.2.0               ha4646dd_0    conda-forge
 libglib                   2.78.1               h783c2da_1    conda-forge
-libgomp                   13.2.0               h807b86a_3    conda-forge
+libgomp                   12.2.0              h65d4601_19    conda-forge
 libgpg-error              1.47                 h71f35ed_0    conda-forge
 libgrpc                   1.58.2               he06187c_0    conda-forge
 libhwloc                  2.9.1                hd6dc26d_0    conda-forge
@@ -256,13 +243,11 @@ libpq                     15.1                 h67c24c5_1    conda-forge
 libprotobuf               4.24.3               hf27288f_1    conda-forge
 libre2-11                 2023.06.02           h7a70373_0    conda-forge
 librsvg                   2.54.4               h7abd40a_0    conda-forge
-libsanitizer              12.3.0               h0f45ef3_3    conda-forge
 libsndfile                1.2.2                hc60ed4a_1    conda-forge
 libsolv                   0.7.23               h3eb15da_0    conda-forge
 libsqlite                 3.40.0               h753d276_0    conda-forge
 libssh2                   1.10.0               hf14f497_3    conda-forge
-libstdcxx-devel_linux-64  12.3.0             h8bca6fd_103    conda-forge
-libstdcxx-ng              13.2.0               h7e041cc_3    conda-forge
+libstdcxx-ng              12.2.0              h46fd767_19    conda-forge
 libsystemd0               253                  h8c4010b_1    conda-forge
 libtasn1                  4.19.0               h166bdaf_0    conda-forge
 libtiff                   4.5.0                h6adf6a1_2    conda-forge
@@ -382,8 +367,7 @@ pycosat                   0.6.4           py310h5764c6d_1    conda-forge
 pycparser                 2.21               pyhd8ed1ab_0    conda-forge
 pygments                  2.14.0                   pypi_0    pypi
 pyjwt                     2.6.0                    pypi_0    pypi
-pymc                      5.10.0               hd8ed1ab_0    conda-forge
-pymc-base                 5.10.0             pyhd8ed1ab_0    conda-forge
+pymc                      5.10.0                   pypi_0    pypi
 pynacl                    1.5.0                    pypi_0    pypi
 pyopenssl                 23.1.1             pyhd8ed1ab_0    conda-forge
 pyparsing                 3.1.1              pyhd8ed1ab_0    conda-forge
@@ -392,7 +376,6 @@ pyqt5-sip                 12.11.0         py310heca2aa9_3    conda-forge
 pyrff                     2.0.2                    pypi_0    pypi
 pyrsistent                0.19.3                   pypi_0    pypi
 pysocks                   1.7.1              pyha2e5f31_6    conda-forge
-pytensor                  2.18.1          py310hc6cd4ac_0    conda-forge
 pytensor-base             2.18.1          py310hcc13569_0    conda-forge
 pytensor-federated        1.0.1                    pypi_0    pypi
 pytest                    7.4.3                    pypi_0    pypi
@@ -449,7 +432,6 @@ stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
 statsmodels               0.14.0          py310h1f7b6fc_2    conda-forge
 stumpy                    1.12.0                   pypi_0    pypi
 svt-av1                   1.4.1                hcb278e6_0    conda-forge
-sysroot_linux-64          2.12                he073ed8_16    conda-forge
 tabulate                  0.9.0                    pypi_0    pypi
 tbb                       2021.9.0             hf52228f_0    conda-forge
 tblib                     3.0.0                    pypi_0    pypi
@@ -484,8 +466,8 @@ wrapt                     1.15.0          py310h1fa729e_0    conda-forge
 wtforms                   3.0.1                    pypi_0    pypi
 x264                      1!164.3095           h166bdaf_2    conda-forge
 x265                      3.5                  h924138e_3    conda-forge
-xarray                    2023.11.0          pyhd8ed1ab_0    conda-forge
-xarray-einstats           0.6.0              pyhd8ed1ab_0    conda-forge
+xarray                    2023.11.0                pypi_0    pypi
+xarray-einstats           0.6.0                    pypi_0    pypi
 xcb-util                  0.4.0                h516909a_0    conda-forge
 xcb-util-image            0.4.0                h166bdaf_0    conda-forge
 xcb-util-keysyms          0.4.0                h516909a_0    conda-forge

maresb commented 11 months ago

Wow that's really weird. Any chance you could provide conda env export so I could try to reproduce it?

michaelosthege commented 11 months ago

broken.yml

```yaml name: base channels: - conda-forge dependencies: - _libgcc_mutex=0.1=conda_forge - _openmp_mutex=4.5=2_kmp_llvm - aiobotocore=1.1.2=py_0 - aiohttp=3.8.4=py310h1fa729e_0 - aioitertools=0.11.0=pyhd8ed1ab_0 - aiosignal=1.3.1=pyhd8ed1ab_0 - alsa-lib=1.2.8=h166bdaf_0 - aom=3.5.0=h27087fc_0 - arviz=0.16.1=pyhd8ed1ab_1 - asttokens=2.4.1=pyhd8ed1ab_0 - async-timeout=4.0.2=pyhd8ed1ab_0 - atk-1.0=2.38.0=hd4edc92_1 - attr=2.5.1=h166bdaf_1 - attrs=22.2.0=pyh71513ae_0 - beautifulsoup4=4.12.0=pyha770c72_0 - binutils_impl_linux-64=2.40=hf600244_0 - binutils_linux-64=2.40=hbdbef99_2 - blas=2.120=openblas - blas-devel=3.9.0=20_linux64_openblas - brotli=1.1.0=hd590300_1 - brotli-bin=1.1.0=hd590300_1 - brotlipy=0.7.0=py310h5764c6d_1005 - bzip2=1.0.8=h7f98852_4 - c-ares=1.23.0=hd590300_0 - ca-certificates=2022.12.7=ha878542_0 - cached-property=1.5.2=hd8ed1ab_1 - cached_property=1.5.2=pyha770c72_1 - cachetools=5.3.2=pyhd8ed1ab_0 - cairo=1.16.0=ha61ee94_1014 - cffi=1.15.1=py310h255011f_3 - chardet=5.1.0=py310hff52083_0 - clickhouse-driver=0.2.6=py310h2372a71_1 - cloudpickle=3.0.0=pyhd8ed1ab_0 - colorama=0.4.6=pyhd8ed1ab_0 - conda=23.1.0=py310hff52083_0 - conda-build=3.24.0=py310hff52083_0 - conda-package-handling=2.0.2=pyh38be061_0 - conda-package-streaming=0.7.0=pyhd8ed1ab_1 - cons=0.4.6=pyhd8ed1ab_0 - contourpy=1.2.0=py310hd41b1e2_0 - cryptography=40.0.1=py310h34c0648_0 - cycler=0.12.1=pyhd8ed1ab_0 - cyrus-sasl=2.1.27=h7604b24_5 - dbus=1.13.6=h5008d03_3 - decorator=5.1.1=pyhd8ed1ab_0 - docutils=0.15.2=py310hff52083_6 - et_xmlfile=1.1.0=pyhd8ed1ab_0 - etuples=0.3.9=pyhd8ed1ab_0 - executing=2.0.1=pyhd8ed1ab_0 - expat=2.5.0=hcb278e6_1 - fastprogress=1.0.3=pyhd8ed1ab_0 - ffmpeg=5.1.2=gpl_h8dda1f0_106 - fftw=3.3.10=nompi_hc118613_108 - filelock=3.10.7=pyhd8ed1ab_0 - fmt=9.1.0=h924138e_0 - font-ttf-dejavu-sans-mono=2.37=hab24e00_0 - font-ttf-inconsolata=3.000=h77eed37_0 - font-ttf-source-code-pro=2.038=h77eed37_0 - font-ttf-ubuntu=0.83=h77eed37_1 - fontconfig=2.14.2=h14ed4e7_0 - fonts-conda-ecosystem=1=0 - fonts-conda-forge=1=0 - fonttools=4.46.0=py310h2372a71_0 - freetype=2.12.1=h267a509_2 - fribidi=1.0.10=h36c2ea0_0 - frozenlist=1.3.3=py310h5764c6d_0 - gcc=12.3.0=h8d2909c_2 - gcc_impl_linux-64=12.3.0=he2b93b0_3 - gcc_linux-64=12.3.0=h76fc315_2 - gdk-pixbuf=2.42.10=h05c8ddd_0 - gettext=0.21.1=h27087fc_0 - giflib=5.2.1=h0b41bf4_3 - glib=2.78.2=hfc55251_0 - glib-tools=2.78.2=hfc55251_0 - glob2=0.7=py_0 - gmp=6.3.0=h59595ed_0 - gnutls=3.7.9=hb077bed_0 - graphite2=1.3.13=h58526e2_1001 - graphviz=7.1.0=h2e5815a_0 - gst-plugins-base=1.21.3=h4243ec0_1 - gstreamer=1.21.3=h25f0c4b_1 - gstreamer-orc=0.4.34=hd590300_0 - gtk2=2.24.33=h90689f9_2 - gts=0.7.6=h977cf35_4 - gxx=12.3.0=h8d2909c_2 - gxx_impl_linux-64=12.3.0=he2b93b0_3 - gxx_linux-64=12.3.0=h8a814eb_2 - h5netcdf=1.3.0=pyhd8ed1ab_0 - h5py=3.8.0=nompi_py310h0311031_100 - harfbuzz=6.0.0=h8e241bc_0 - hdf5=1.12.2=nompi_h4df4325_100 - icu=70.1=h27087fc_0 - imageio=2.31.5=pyh8c1a49c_0 - imageio-ffmpeg=0.4.9=pyhd8ed1ab_0 - importlib-metadata=7.0.0=pyha770c72_0 - importlib_metadata=7.0.0=hd8ed1ab_0 - ipython=8.18.1=pyh707e725_3 - iso3166=2.1.1=pyhd8ed1ab_0 - jack=1.9.22=h11f4161_0 - jax=0.4.21=pyhd8ed1ab_0 - jaxlib=0.4.19=cpu_py310h9ed8a0c_0 - jedi=0.19.1=pyhd8ed1ab_0 - jinja2=3.1.2=pyhd8ed1ab_1 - jmespath=0.10.0=pyh9f0ad1d_0 - joblib=1.3.2=pyhd8ed1ab_0 - jpeg=9e=h0b41bf4_3 - kernel-headers_linux-64=2.6.32=he073ed8_16 - keyutils=1.6.1=h166bdaf_0 - kiwisolver=1.4.5=py310hd41b1e2_1 - krb5=1.19.3=h08a2579_0 - lame=3.100=h166bdaf_1003 - lcms2=2.15=hfd0df8a_0 - ld_impl_linux-64=2.40=h41732ed_0 - lerc=4.0.0=h27087fc_0 - libabseil=20230802.1=cxx17_h59595ed_0 - libarchive=3.6.2=h3d51595_0 - libblas=3.9.0=20_linux64_openblas - libbrotlicommon=1.1.0=hd590300_1 - libbrotlidec=1.1.0=hd590300_1 - libbrotlienc=1.1.0=hd590300_1 - libcap=2.67=he9d0100_0 - libcblas=3.9.0=20_linux64_openblas - libclang=15.0.7=default_hb11cfb5_4 - libclang13=15.0.7=default_ha2b6cf4_4 - libcups=2.3.3=h3e49a29_2 - libcurl=7.86.0=h2283fc2_1 - libdb=6.2.32=h9c3ff4c_0 - libdeflate=1.17=h0b41bf4_0 - libdrm=2.4.114=h166bdaf_0 - libedit=3.1.20191231=he28a2e2_2 - libev=4.33=h516909a_1 - libevent=2.1.10=h28343ad_4 - libexpat=2.5.0=hcb278e6_1 - libffi=3.4.2=h7f98852_5 - libflac=1.4.3=h59595ed_0 - libgcc-devel_linux-64=12.3.0=h8bca6fd_103 - libgcc-ng=13.2.0=h807b86a_3 - libgcrypt=1.10.3=hd590300_0 - libgd=2.3.3=h5aea950_4 - libgfortran-ng=13.2.0=h69a702a_3 - libgfortran5=13.2.0=ha4646dd_3 - libglib=2.78.2=h783c2da_0 - libgomp=13.2.0=h807b86a_3 - libgpg-error=1.47=h71f35ed_0 - libgrpc=1.58.2=he06187c_0 - libhwloc=2.9.1=hd6dc26d_0 - libiconv=1.17=h166bdaf_0 - libidn2=2.3.4=h166bdaf_0 - liblapack=3.9.0=20_linux64_openblas - liblapacke=3.9.0=20_linux64_openblas - liblief=0.12.3=h27087fc_0 - libllvm14=14.0.6=hcd5def8_4 - libllvm15=15.0.7=hadd5161_1 - libmamba=1.1.0=hde2b089_3 - libmambapy=1.1.0=py310h1428755_3 - libnghttp2=1.52.0=h61bc06f_0 - libnsl=2.0.0=h7f98852_0 - libntlm=1.4=h7f98852_1002 - libogg=1.3.4=h7f98852_1 - libopenblas=0.3.25=pthreads_h413a1c8_0 - libopus=1.3.1=h7f98852_1 - libpciaccess=0.17=h166bdaf_0 - libpng=1.6.39=h753d276_0 - libpq=15.1=h67c24c5_1 - libprotobuf=4.24.3=hf27288f_1 - libre2-11=2023.06.02=h7a70373_0 - librsvg=2.54.4=h7abd40a_0 - libsanitizer=12.3.0=h0f45ef3_3 - libsndfile=1.2.2=hc60ed4a_1 - libsolv=0.7.23=h3eb15da_0 - libsqlite=3.40.0=h753d276_0 - libssh2=1.10.0=hf14f497_3 - libstdcxx-devel_linux-64=12.3.0=h8bca6fd_103 - libstdcxx-ng=13.2.0=h7e041cc_3 - libsystemd0=253=h8c4010b_1 - libtasn1=4.19.0=h166bdaf_0 - libtiff=4.5.0=h6adf6a1_2 - libtool=2.4.7=h27087fc_0 - libudev1=253=h0b41bf4_1 - libunistring=0.9.10=h7f98852_0 - libuuid=2.32.1=h7f98852_1000 - libva=2.18.0=h0b41bf4_0 - libvorbis=1.3.7=h9c3ff4c_0 - libvpx=1.11.0=h9c3ff4c_3 - libwebp=1.2.4=h1daa5a0_1 - libwebp-base=1.2.4=h166bdaf_0 - libxcb=1.13=h7f98852_1004 - libxkbcommon=1.5.0=h79f4944_1 - libxml2=2.10.3=hca2bb57_4 - libzlib=1.2.13=h166bdaf_4 - llvm-openmp=17.0.6=h4dfa4b3_0 - llvmlite=0.40.1=py310h1b8f574_0 - logical-unification=0.4.6=pyhd8ed1ab_0 - lz4-c=1.9.4=hcb278e6_0 - lzo=2.10=h516909a_1000 - mamba=1.1.0=py310h51d5547_3 - markupsafe=2.1.2=py310h1fa729e_0 - matplotlib=3.8.2=py310hff52083_0 - matplotlib-base=3.8.2=py310h62c0568_0 - matplotlib-inline=0.1.6=pyhd8ed1ab_0 - minikanren=1.0.3=pyhd8ed1ab_0 - mkl=2022.2.1=h84fe81f_16997 - mkl-service=2.4.0=py310h9263142_0 - ml_dtypes=0.3.1=py310hcc13569_2 - mpg123=1.32.3=h59595ed_0 - multidict=6.0.4=py310h1fa729e_0 - multipledispatch=0.6.0=py_0 - munkres=1.1.4=pyh9f0ad1d_0 - mysql-common=8.0.33=hf1915f5_6 - mysql-libs=8.0.33=hca2cd23_6 - ncurses=6.3=h27087fc_1 - ndg-httpsclient=0.5.1=py_1 - nettle=3.9.1=h7ab15ed_0 - nspr=4.35=h27087fc_0 - nss=3.89=he45b914_0 - numba=0.57.1=py310h0f6aa51_0 - numpy=1.24.4=py310ha4c1d20_0 - numpyro=0.13.2=pyhd8ed1ab_1 - openblas=0.3.25=pthreads_h7a3da1a_0 - openh264=2.3.1=hcb278e6_2 - openjpeg=2.5.0=hfec8fc6_2 - openldap=2.4.59=hc311a53_0 - openpyxl=3.1.2=py310h2372a71_1 - openssl=3.1.4=hd590300_0 - opt-einsum=3.3.0=hd8ed1ab_2 - opt_einsum=3.3.0=pyhc1e730c_2 - p11-kit=0.24.1=hc5aa10d_0 - packaging=23.0=pyhd8ed1ab_0 - pandas=1.5.3=py310h9b08913_1 - pango=1.50.14=hd33c08f_0 - parso=0.8.3=pyhd8ed1ab_0 - patch=2.7.6=h7f98852_1002 - patchelf=0.17.2=h58526e2_0 - patsy=0.5.4=pyhd8ed1ab_0 - pcre2=10.42=hcad00b1_0 - pexpect=4.8.0=pyh1a96a4e_2 - pickleshare=0.7.5=py_1003 - pillow=9.4.0=py310h023d228_1 - pip=23.3.1=pyhd8ed1ab_0 - pixman=0.42.2=h59595ed_0 - pkginfo=1.9.6=pyhd8ed1ab_0 - pluggy=1.0.0=pyhd8ed1ab_5 - psutil=5.9.4=py310h5764c6d_0 - pthread-stubs=0.4=h36c2ea0_1001 - ptyprocess=0.7.0=pyhd3deb0d_0 - pulseaudio=16.1=hcb278e6_3 - pulseaudio-client=16.1=h5195f5e_3 - pulseaudio-daemon=16.1=ha8d29e2_3 - pure_eval=0.2.2=pyhd8ed1ab_0 - py-lief=0.12.3=py310hd8f1fbe_0 - pyasn1=0.4.8=py_0 - pyasn1-modules=0.2.7=py_0 - pybind11-abi=4=hd8ed1ab_3 - pycosat=0.6.4=py310h5764c6d_1 - pycparser=2.21=pyhd8ed1ab_0 - pymc=5.10.0=hd8ed1ab_0 - pymc-base=5.10.0=pyhd8ed1ab_0 - pyopenssl=23.1.1=pyhd8ed1ab_0 - pyparsing=3.1.1=pyhd8ed1ab_0 - pyqt=5.15.7=py310hab646b1_3 - pyqt5-sip=12.11.0=py310heca2aa9_3 - pysocks=1.7.1=pyha2e5f31_6 - pytensor=2.18.1=py310hc6cd4ac_0 - pytensor-base=2.18.1=py310hcc13569_0 - python=3.10.10=he550d4f_0_cpython - python-dateutil=2.8.2=pyhd8ed1ab_0 - python-graphviz=0.20.1=pyh22cad53_0 - python-ldap=3.4.0=py310h1fa729e_3 - python-libarchive-c=4.0=py310hff52083_2 - python_abi=3.10=3_cp310 - pyyaml=6.0=py310h5764c6d_5 - qt-main=5.15.6=hafeba50_4 - re2=2023.06.02=h2873b5e_0 - readline=8.2=h8228510_1 - reproc=14.2.4=h0b41bf4_0 - reproc-cpp=14.2.4=hcb278e6_0 - ripgrep=13.0.0=h2f28480_2 - ruamel.yaml=0.17.21=py310h1fa729e_3 - ruamel.yaml.clib=0.2.7=py310h1fa729e_1 - s3fs=2021.7.0=pyhd8ed1ab_0 - scikit-learn=1.3.2=py310h1fdf081_1 - scipy=1.11.4=py310hb13e2d6_0 - setuptools=65.6.3=pyhd8ed1ab_0 - sip=6.7.12=py310hc6cd4ac_0 - six=1.16.0=pyh6c4a22f_0 - soupsieve=2.3.2.post1=pyhd8ed1ab_0 - stack_data=0.6.2=pyhd8ed1ab_0 - statsmodels=0.14.0=py310h1f7b6fc_2 - svt-av1=1.4.1=hcb278e6_0 - sysroot_linux-64=2.12=he073ed8_16 - tbb=2021.9.0=hf52228f_0 - threadpoolctl=3.2.0=pyha21a80b_0 - tk=8.6.13=noxft_h4845f30_101 - toml=0.10.2=pyhd8ed1ab_0 - tomli=2.0.1=pyhd8ed1ab_0 - toolz=0.12.0=pyhd8ed1ab_0 - tqdm=4.65.0=pyhd8ed1ab_1 - traitlets=5.14.0=pyhd8ed1ab_0 - tzdata=2023b=h71feb2d_0 - tzlocal=5.2=py310hff52083_0 - unicodedata2=15.1.0=py310h2372a71_0 - watermark=2.4.3=pyhd8ed1ab_0 - wheel=0.40.0=pyhd8ed1ab_0 - wrapt=1.15.0=py310h1fa729e_0 - x264=1!164.3095=h166bdaf_2 - x265=3.5=h924138e_3 - xarray=2023.11.0=pyhd8ed1ab_0 - xarray-einstats=0.6.0=pyhd8ed1ab_0 - xcb-util=0.4.0=h516909a_0 - xcb-util-image=0.4.0=h166bdaf_0 - xcb-util-keysyms=0.4.0=h516909a_0 - xcb-util-renderutil=0.3.9=h166bdaf_0 - xcb-util-wm=0.4.1=h516909a_0 - xkeyboard-config=2.38=h0b41bf4_0 - xorg-fixesproto=5.0=h7f98852_1002 - xorg-kbproto=1.0.7=h7f98852_1002 - xorg-libice=1.0.10=h7f98852_0 - xorg-libsm=1.2.3=hd9c2040_1000 - xorg-libx11=1.8.4=h0b41bf4_0 - xorg-libxau=1.0.11=hd590300_0 - xorg-libxdmcp=1.1.3=h7f98852_0 - xorg-libxext=1.3.4=h0b41bf4_2 - xorg-libxfixes=5.0.3=h7f98852_1004 - xorg-libxrender=0.9.10=h7f98852_1003 - xorg-renderproto=0.11.1=h7f98852_1002 - xorg-xextproto=7.3.0=h0b41bf4_1003 - xorg-xproto=7.0.31=h7f98852_1007 - xz=5.2.6=h166bdaf_0 - yaml=0.2.5=h7f98852_2 - yaml-cpp=0.7.0=h27087fc_2 - yarl=1.8.2=py310h5764c6d_0 - zipp=3.17.0=pyhd8ed1ab_0 - zlib=1.2.13=h166bdaf_4 - zstandard=0.19.0=py310hdeb6495_1 - zstd=1.5.5=hfc55251_0 - pip: - alembic==1.10.2 - amqp==5.1.1 - anyio==3.6.2 - apache-airflow==2.5.2 - apache-airflow-flowitems==1.0.2 - apache-airflow-providers-amazon==7.3.0 - apache-airflow-providers-celery==3.1.0 - apache-airflow-providers-common-sql==1.3.4 - apache-airflow-providers-docker==3.5.1 - apache-airflow-providers-ftp==3.3.1 - apache-airflow-providers-http==4.2.0 - apache-airflow-providers-imap==3.1.1 - apache-airflow-providers-mysql==4.0.2 - apache-airflow-providers-slack==7.2.0 - apache-airflow-providers-sqlite==3.3.1 - apache-airflow-providers-ssh==3.5.0 - apispec==3.3.2 - argcomplete==3.0.5 - asgiref==3.6.0 - asn1crypto==1.5.1 - avro-python3==1.10.2 - babel==2.12.1 - bcrypt==4.0.1 - betterproto==2.0.0b6 - billiard==3.6.4.0 - black==23.11.0 - bletl==1.4.1 - blinker==1.5 - boto3==1.26.104 - botocore==1.29.104 - cachelib==0.9.0 - calibr8==7.1.0 - cattrs==22.2.0 - celery==5.2.7 - certifi==2023.7.22 - charset-normalizer==2.0.12 - click==8.1.3 - click-didyoumean==0.3.0 - click-plugins==1.1.1 - click-repl==0.2.0 - clickclick==20.10.2 - colorlog==4.8.0 - configupdater==3.1.1 - connexion==2.14.2 - cron-descriptor==1.2.35 - croniter==1.3.8 - csaps==1.1.0 - dask==2023.12.0 - deprecated==1.2.13 - dill==0.3.6 - distributed==2023.12.0 - dnspython==2.3.0 - docker==6.0.1 - email-validator==1.3.1 - exceptiongroup==1.1.1 - flask==2.2.3 - flask-appbuilder==4.1.4 - flask-babel==2.0.0 - flask-caching==2.0.2 - flask-jwt-extended==4.4.4 - flask-login==0.6.2 - flask-session==0.4.0 - flask-sqlalchemy==2.5.1 - flask-wtf==1.1.1 - flower==1.2.0 - fsspec==2023.12.1 - gittrail==0.1.1 - greenlet==2.0.2 - grpclib==0.4.6 - gunicorn==20.1.0 - h11==0.14.0 - h2==4.1.0 - hagelkorn==1.2.3 - hpack==4.0.0 - httpcore==0.16.3 - httpx==0.23.3 - humanize==4.6.0 - hyperframe==6.0.1 - idna==3.3 - inflection==0.5.1 - iniconfig==2.0.0 - isort==5.12.0 - itsdangerous==2.1.2 - jsonpath-ng==1.5.3 - jsonschema==4.17.3 - kombu==5.2.4 - lazy-object-proxy==1.9.0 - ldap3==2.9.1 - linkify-it-py==2.0.0 - locket==1.0.0 - lockfile==0.12.2 - lxml==4.9.3 - mako==1.2.4 - markdown==3.4.3 - markdown-it-py==2.2.0 - marshmallow==3.19.0 - marshmallow-enum==1.5.1 - marshmallow-oneofschema==3.0.1 - marshmallow-sqlalchemy==0.26.1 - mcbackend==0.5.1 - mdit-py-plugins==0.3.5 - mdurl==0.1.2 - msgpack==1.0.7 - murefi==5.3.0 - mypy-boto3-appflow==1.26.78 - mypy-boto3-rds==1.26.102 - mypy-boto3-redshift-data==1.26.88 - mypy-extensions==1.0.0 - mysql-connector-python==8.0.32 - mysqlclient==2.1.1 - nest-asyncio==1.5.8 - paramiko==3.1.0 - partd==1.4.1 - pathspec==0.9.0 - peak-performance==0.6.4 - pendulum==2.1.2 - platformdirs==4.1.0 - ply==3.11 - prison==0.2.1 - prometheus-client==0.16.0 - prompt-toolkit==3.0.38 - protobuf==3.20.3 - psycopg2-binary==2.9.9 - pygments==2.14.0 - pyjwt==2.6.0 - pynacl==1.5.0 - pyrff==2.0.2 - pyrsistent==0.19.3 - pytensor-federated==1.0.1 - pytest==7.4.3 - python-daemon==3.0.1 - python-dotenv==1.0.0 - python-nvd3==0.15.0 - python-slugify==8.0.1 - pytz==2022.1 - pytzdata==2020.1 - redis==3.5.3 - redshift-connector==2.0.910 - requests==2.31.0 - requests-toolbelt==0.10.1 - rfc3339-validator==0.1.4 - rfc3986==1.5.0 - rich==13.3.3 - robotools==1.8.0 - s3transfer==0.6.0 - scramp==1.4.4 - setproctitle==1.3.2 - slack-sdk==3.20.2 - sniffio==1.3.0 - sortedcontainers==2.4.0 - sqlalchemy==1.4.46 - sqlalchemy-jsonfield==1.0.1.post0 - sqlalchemy-redshift==0.8.13 - sqlalchemy-utils==0.40.0 - sqlparse==0.4.3 - sshtunnel==0.4.0 - stumpy==1.12.0 - tabulate==0.9.0 - tblib==3.0.0 - tenacity==8.2.2 - termcolor==2.2.0 - text-unidecode==1.3 - tornado==6.2 - tsfresh==0.20.1 - typing==3.7.4.3 - typing-extensions==4.1.0 - uc-micro-py==1.0.1 - unicodecsv==0.14.1 - urllib3==1.26.8 - vine==5.0.0 - watchtower==2.0.1 - wcwidth==0.2.6 - websocket-client==1.5.1 - werkzeug==2.2.3 - wtforms==3.0.1 - zict==3.0.0 ```

working.yml

```yml name: base channels: - conda-forge dependencies: - _libgcc_mutex=0.1=conda_forge - _openmp_mutex=4.5=2_kmp_llvm - aiobotocore=1.1.2=py_0 - aiohttp=3.8.4=py310h1fa729e_0 - aioitertools=0.11.0=pyhd8ed1ab_0 - aiosignal=1.3.1=pyhd8ed1ab_0 - alsa-lib=1.2.8=h166bdaf_0 - aom=3.5.0=h27087fc_0 - asttokens=2.4.1=pyhd8ed1ab_0 - async-timeout=4.0.2=pyhd8ed1ab_0 - atk-1.0=2.38.0=hd4edc92_1 - attr=2.5.1=h166bdaf_1 - attrs=22.2.0=pyh71513ae_0 - beautifulsoup4=4.12.0=pyha770c72_0 - blas=2.120=openblas - blas-devel=3.9.0=20_linux64_openblas - brotli=1.1.0=hd590300_1 - brotli-bin=1.1.0=hd590300_1 - brotlipy=0.7.0=py310h5764c6d_1005 - bzip2=1.0.8=h7f98852_4 - c-ares=1.23.0=hd590300_0 - ca-certificates=2022.12.7=ha878542_0 - cairo=1.16.0=ha61ee94_1014 - cffi=1.15.1=py310h255011f_3 - chardet=5.1.0=py310hff52083_0 - clickhouse-driver=0.2.6=py310h2372a71_1 - colorama=0.4.6=pyhd8ed1ab_0 - conda=23.1.0=py310hff52083_0 - conda-build=3.24.0=py310hff52083_0 - conda-package-handling=2.0.2=pyh38be061_0 - conda-package-streaming=0.7.0=pyhd8ed1ab_1 - cons=0.4.6=pyhd8ed1ab_0 - contourpy=1.2.0=py310hd41b1e2_0 - cryptography=40.0.1=py310h34c0648_0 - cycler=0.12.1=pyhd8ed1ab_0 - cyrus-sasl=2.1.27=h7604b24_5 - dbus=1.13.6=h5008d03_3 - decorator=5.1.1=pyhd8ed1ab_0 - docutils=0.15.2=py310hff52083_6 - et_xmlfile=1.1.0=pyhd8ed1ab_0 - etuples=0.3.9=pyhd8ed1ab_0 - executing=2.0.1=pyhd8ed1ab_0 - expat=2.5.0=hcb278e6_1 - ffmpeg=5.1.2=gpl_h8dda1f0_106 - fftw=3.3.10=nompi_hc118613_108 - filelock=3.10.7=pyhd8ed1ab_0 - fmt=9.1.0=h924138e_0 - font-ttf-dejavu-sans-mono=2.37=hab24e00_0 - font-ttf-inconsolata=3.000=h77eed37_0 - font-ttf-source-code-pro=2.038=h77eed37_0 - font-ttf-ubuntu=0.83=h77eed37_1 - fontconfig=2.14.2=h14ed4e7_0 - fonts-conda-ecosystem=1=0 - fonts-conda-forge=1=0 - fonttools=4.46.0=py310h2372a71_0 - freetype=2.12.1=h267a509_2 - fribidi=1.0.10=h36c2ea0_0 - frozenlist=1.3.3=py310h5764c6d_0 - gdk-pixbuf=2.42.10=h05c8ddd_0 - gettext=0.21.1=h27087fc_0 - giflib=5.2.1=h0b41bf4_3 - glib=2.78.2=hfc55251_0 - glib-tools=2.78.2=hfc55251_0 - glob2=0.7=py_0 - gmp=6.3.0=h59595ed_0 - gnutls=3.7.9=hb077bed_0 - graphite2=1.3.13=h58526e2_1001 - graphviz=7.1.0=h2e5815a_0 - gst-plugins-base=1.21.3=h4243ec0_1 - gstreamer=1.21.3=h25f0c4b_1 - gstreamer-orc=0.4.34=hd590300_0 - gtk2=2.24.33=h90689f9_2 - gts=0.7.6=h977cf35_4 - harfbuzz=6.0.0=h8e241bc_0 - icu=70.1=h27087fc_0 - imageio=2.31.5=pyh8c1a49c_0 - imageio-ffmpeg=0.4.9=pyhd8ed1ab_0 - importlib-metadata=7.0.0=pyha770c72_0 - importlib_metadata=7.0.0=hd8ed1ab_0 - ipython=8.18.1=pyh707e725_3 - iso3166=2.1.1=pyhd8ed1ab_0 - jack=1.9.22=h11f4161_0 - jax=0.4.21=pyhd8ed1ab_0 - jaxlib=0.4.19=cpu_py310h9ed8a0c_0 - jedi=0.19.1=pyhd8ed1ab_0 - jinja2=3.1.2=pyhd8ed1ab_1 - jmespath=0.10.0=pyh9f0ad1d_0 - joblib=1.3.2=pyhd8ed1ab_0 - jpeg=9e=h0b41bf4_3 - keyutils=1.6.1=h166bdaf_0 - kiwisolver=1.4.5=py310hd41b1e2_1 - krb5=1.19.3=h08a2579_0 - lame=3.100=h166bdaf_1003 - lcms2=2.15=hfd0df8a_0 - ld_impl_linux-64=2.40=h41732ed_0 - lerc=4.0.0=h27087fc_0 - libabseil=20230802.1=cxx17_h59595ed_0 - libarchive=3.6.2=h3d51595_0 - libblas=3.9.0=20_linux64_openblas - libbrotlicommon=1.1.0=hd590300_1 - libbrotlidec=1.1.0=hd590300_1 - libbrotlienc=1.1.0=hd590300_1 - libcap=2.67=he9d0100_0 - libcblas=3.9.0=20_linux64_openblas - libclang=15.0.7=default_hb11cfb5_4 - libclang13=15.0.7=default_ha2b6cf4_4 - libcups=2.3.3=h3e49a29_2 - libcurl=7.86.0=h2283fc2_1 - libdb=6.2.32=h9c3ff4c_0 - libdeflate=1.17=h0b41bf4_0 - libdrm=2.4.114=h166bdaf_0 - libedit=3.1.20191231=he28a2e2_2 - libev=4.33=h516909a_1 - libevent=2.1.10=h28343ad_4 - libexpat=2.5.0=hcb278e6_1 - libffi=3.4.2=h7f98852_5 - libflac=1.4.3=h59595ed_0 - libgcc-ng=12.2.0=h65d4601_19 - libgcrypt=1.10.3=hd590300_0 - libgd=2.3.3=h5aea950_4 - libgfortran-ng=13.2.0=h69a702a_0 - libgfortran5=13.2.0=ha4646dd_0 - libglib=2.78.2=h783c2da_0 - libgomp=12.2.0=h65d4601_19 - libgpg-error=1.47=h71f35ed_0 - libgrpc=1.58.2=he06187c_0 - libhwloc=2.9.1=hd6dc26d_0 - libiconv=1.17=h166bdaf_0 - libidn2=2.3.4=h166bdaf_0 - liblapack=3.9.0=20_linux64_openblas - liblapacke=3.9.0=20_linux64_openblas - liblief=0.12.3=h27087fc_0 - libllvm14=14.0.6=hcd5def8_4 - libllvm15=15.0.7=hadd5161_1 - libmamba=1.1.0=hde2b089_3 - libmambapy=1.1.0=py310h1428755_3 - libnghttp2=1.52.0=h61bc06f_0 - libnsl=2.0.0=h7f98852_0 - libntlm=1.4=h7f98852_1002 - libogg=1.3.4=h7f98852_1 - libopenblas=0.3.25=pthreads_h413a1c8_0 - libopus=1.3.1=h7f98852_1 - libpciaccess=0.17=h166bdaf_0 - libpng=1.6.39=h753d276_0 - libpq=15.1=h67c24c5_1 - libprotobuf=4.24.3=hf27288f_1 - libre2-11=2023.06.02=h7a70373_0 - librsvg=2.54.4=h7abd40a_0 - libsndfile=1.2.2=hc60ed4a_1 - libsolv=0.7.23=h3eb15da_0 - libsqlite=3.40.0=h753d276_0 - libssh2=1.10.0=hf14f497_3 - libstdcxx-ng=12.2.0=h46fd767_19 - libsystemd0=253=h8c4010b_1 - libtasn1=4.19.0=h166bdaf_0 - libtiff=4.5.0=h6adf6a1_2 - libtool=2.4.7=h27087fc_0 - libudev1=253=h0b41bf4_1 - libunistring=0.9.10=h7f98852_0 - libuuid=2.32.1=h7f98852_1000 - libva=2.18.0=h0b41bf4_0 - libvorbis=1.3.7=h9c3ff4c_0 - libvpx=1.11.0=h9c3ff4c_3 - libwebp=1.2.4=h1daa5a0_1 - libwebp-base=1.2.4=h166bdaf_0 - libxcb=1.13=h7f98852_1004 - libxkbcommon=1.5.0=h79f4944_1 - libxml2=2.10.3=hca2bb57_4 - libzlib=1.2.13=h166bdaf_4 - llvm-openmp=17.0.6=h4dfa4b3_0 - llvmlite=0.40.1=py310h1b8f574_0 - logical-unification=0.4.6=pyhd8ed1ab_0 - lz4-c=1.9.4=hcb278e6_0 - lzo=2.10=h516909a_1000 - mamba=1.1.0=py310h51d5547_3 - markupsafe=2.1.2=py310h1fa729e_0 - matplotlib=3.8.2=py310hff52083_0 - matplotlib-base=3.8.2=py310h62c0568_0 - matplotlib-inline=0.1.6=pyhd8ed1ab_0 - minikanren=1.0.3=pyhd8ed1ab_0 - mkl=2022.2.1=h84fe81f_16997 - mkl-service=2.4.0=py310h9263142_0 - ml_dtypes=0.3.1=py310hcc13569_2 - mpg123=1.32.3=h59595ed_0 - multidict=6.0.4=py310h1fa729e_0 - multipledispatch=0.6.0=py_0 - munkres=1.1.4=pyh9f0ad1d_0 - mysql-common=8.0.33=hf1915f5_6 - mysql-libs=8.0.33=hca2cd23_6 - ncurses=6.3=h27087fc_1 - ndg-httpsclient=0.5.1=py_1 - nettle=3.9.1=h7ab15ed_0 - nspr=4.35=h27087fc_0 - nss=3.89=he45b914_0 - numba=0.57.1=py310h0f6aa51_0 - numpy=1.24.4=py310ha4c1d20_0 - numpyro=0.13.2=pyhd8ed1ab_1 - openblas=0.3.25=pthreads_h7a3da1a_0 - openh264=2.3.1=hcb278e6_2 - openjpeg=2.5.0=hfec8fc6_2 - openldap=2.4.59=hc311a53_0 - openpyxl=3.1.2=py310h2372a71_1 - openssl=3.1.4=hd590300_0 - opt-einsum=3.3.0=hd8ed1ab_2 - opt_einsum=3.3.0=pyhc1e730c_2 - p11-kit=0.24.1=hc5aa10d_0 - packaging=23.0=pyhd8ed1ab_0 - pandas=1.5.3=py310h9b08913_1 - pango=1.50.14=hd33c08f_0 - parso=0.8.3=pyhd8ed1ab_0 - patch=2.7.6=h7f98852_1002 - patchelf=0.17.2=h58526e2_0 - patsy=0.5.4=pyhd8ed1ab_0 - pcre2=10.42=hcad00b1_0 - pexpect=4.8.0=pyh1a96a4e_2 - pickleshare=0.7.5=py_1003 - pillow=9.4.0=py310h023d228_1 - pip=23.3.1=pyhd8ed1ab_0 - pixman=0.42.2=h59595ed_0 - pkginfo=1.9.6=pyhd8ed1ab_0 - pluggy=1.0.0=pyhd8ed1ab_5 - psutil=5.9.4=py310h5764c6d_0 - pthread-stubs=0.4=h36c2ea0_1001 - ptyprocess=0.7.0=pyhd3deb0d_0 - pulseaudio=16.1=hcb278e6_3 - pulseaudio-client=16.1=h5195f5e_3 - pulseaudio-daemon=16.1=ha8d29e2_3 - pure_eval=0.2.2=pyhd8ed1ab_0 - py-lief=0.12.3=py310hd8f1fbe_0 - pyasn1=0.4.8=py_0 - pyasn1-modules=0.2.7=py_0 - pybind11-abi=4=hd8ed1ab_3 - pycosat=0.6.4=py310h5764c6d_1 - pycparser=2.21=pyhd8ed1ab_0 - pyopenssl=23.1.1=pyhd8ed1ab_0 - pyparsing=3.1.1=pyhd8ed1ab_0 - pyqt=5.15.7=py310hab646b1_3 - pyqt5-sip=12.11.0=py310heca2aa9_3 - pysocks=1.7.1=pyha2e5f31_6 - pytensor-base=2.18.1=py310hcc13569_0 - python=3.10.10=he550d4f_0_cpython - python-dateutil=2.8.2=pyhd8ed1ab_0 - python-graphviz=0.20.1=pyh22cad53_0 - python-ldap=3.4.0=py310h1fa729e_3 - python-libarchive-c=4.0=py310hff52083_2 - python_abi=3.10=3_cp310 - pyyaml=6.0=py310h5764c6d_5 - qt-main=5.15.6=hafeba50_4 - re2=2023.06.02=h2873b5e_0 - readline=8.2=h8228510_1 - reproc=14.2.4=h0b41bf4_0 - reproc-cpp=14.2.4=hcb278e6_0 - ripgrep=13.0.0=h2f28480_2 - ruamel.yaml=0.17.21=py310h1fa729e_3 - ruamel.yaml.clib=0.2.7=py310h1fa729e_1 - s3fs=2021.7.0=pyhd8ed1ab_0 - scikit-learn=1.3.2=py310h1fdf081_1 - scipy=1.11.4=py310hb13e2d6_0 - setuptools=65.6.3=pyhd8ed1ab_0 - sip=6.7.12=py310hc6cd4ac_0 - six=1.16.0=pyh6c4a22f_0 - soupsieve=2.3.2.post1=pyhd8ed1ab_0 - stack_data=0.6.2=pyhd8ed1ab_0 - statsmodels=0.14.0=py310h1f7b6fc_2 - svt-av1=1.4.1=hcb278e6_0 - tbb=2021.9.0=hf52228f_0 - threadpoolctl=3.2.0=pyha21a80b_0 - tk=8.6.13=noxft_h4845f30_101 - toml=0.10.2=pyhd8ed1ab_0 - tomli=2.0.1=pyhd8ed1ab_0 - toolz=0.12.0=pyhd8ed1ab_0 - tqdm=4.65.0=pyhd8ed1ab_1 - traitlets=5.14.0=pyhd8ed1ab_0 - tzdata=2023b=h71feb2d_0 - tzlocal=5.2=py310hff52083_0 - unicodedata2=15.1.0=py310h2372a71_0 - watermark=2.4.3=pyhd8ed1ab_0 - wheel=0.40.0=pyhd8ed1ab_0 - wrapt=1.15.0=py310h1fa729e_0 - x264=1!164.3095=h166bdaf_2 - x265=3.5=h924138e_3 - xcb-util=0.4.0=h516909a_0 - xcb-util-image=0.4.0=h166bdaf_0 - xcb-util-keysyms=0.4.0=h516909a_0 - xcb-util-renderutil=0.3.9=h166bdaf_0 - xcb-util-wm=0.4.1=h516909a_0 - xkeyboard-config=2.38=h0b41bf4_0 - xorg-fixesproto=5.0=h7f98852_1002 - xorg-kbproto=1.0.7=h7f98852_1002 - xorg-libice=1.0.10=h7f98852_0 - xorg-libsm=1.2.3=hd9c2040_1000 - xorg-libx11=1.8.4=h0b41bf4_0 - xorg-libxau=1.0.11=hd590300_0 - xorg-libxdmcp=1.1.3=h7f98852_0 - xorg-libxext=1.3.4=h0b41bf4_2 - xorg-libxfixes=5.0.3=h7f98852_1004 - xorg-libxrender=0.9.10=h7f98852_1003 - xorg-renderproto=0.11.1=h7f98852_1002 - xorg-xextproto=7.3.0=h0b41bf4_1003 - xorg-xproto=7.0.31=h7f98852_1007 - xz=5.2.6=h166bdaf_0 - yaml=0.2.5=h7f98852_2 - yaml-cpp=0.7.0=h27087fc_2 - yarl=1.8.2=py310h5764c6d_0 - zipp=3.17.0=pyhd8ed1ab_0 - zlib=1.2.13=h166bdaf_4 - zstandard=0.19.0=py310hdeb6495_1 - zstd=1.5.5=hfc55251_0 - pip: - alembic==1.10.2 - amqp==5.1.1 - anyio==3.6.2 - apache-airflow==2.5.2 - apache-airflow-flowitems==1.0.2 - apache-airflow-providers-amazon==7.3.0 - apache-airflow-providers-celery==3.1.0 - apache-airflow-providers-common-sql==1.3.4 - apache-airflow-providers-docker==3.5.1 - apache-airflow-providers-ftp==3.3.1 - apache-airflow-providers-http==4.2.0 - apache-airflow-providers-imap==3.1.1 - apache-airflow-providers-mysql==4.0.2 - apache-airflow-providers-slack==7.2.0 - apache-airflow-providers-sqlite==3.3.1 - apache-airflow-providers-ssh==3.5.0 - apispec==3.3.2 - argcomplete==3.0.5 - arviz==0.16.1 - asgiref==3.6.0 - asn1crypto==1.5.1 - avro-python3==1.10.2 - babel==2.12.1 - bcrypt==4.0.1 - betterproto==2.0.0b6 - billiard==3.6.4.0 - black==23.11.0 - bletl==1.4.1 - blinker==1.5 - boto3==1.26.104 - botocore==1.29.104 - cachelib==0.9.0 - cachetools==5.3.2 - calibr8==7.1.0 - cattrs==22.2.0 - celery==5.2.7 - certifi==2023.7.22 - charset-normalizer==2.0.12 - click==8.1.3 - click-didyoumean==0.3.0 - click-plugins==1.1.1 - click-repl==0.2.0 - clickclick==20.10.2 - cloudpickle==3.0.0 - colorlog==4.8.0 - configupdater==3.1.1 - connexion==2.14.2 - cron-descriptor==1.2.35 - croniter==1.3.8 - csaps==1.1.0 - dask==2023.12.0 - deprecated==1.2.13 - dill==0.3.6 - distributed==2023.12.0 - dnspython==2.3.0 - docker==6.0.1 - email-validator==1.3.1 - exceptiongroup==1.1.1 - fastprogress==1.0.3 - flask==2.2.3 - flask-appbuilder==4.1.4 - flask-babel==2.0.0 - flask-caching==2.0.2 - flask-jwt-extended==4.4.4 - flask-login==0.6.2 - flask-session==0.4.0 - flask-sqlalchemy==2.5.1 - flask-wtf==1.1.1 - flower==1.2.0 - fsspec==2023.12.1 - gittrail==0.1.1 - greenlet==2.0.2 - grpclib==0.4.6 - gunicorn==20.1.0 - h11==0.14.0 - h2==4.1.0 - h5netcdf==1.3.0 - h5py==3.10.0 - hagelkorn==1.2.3 - hpack==4.0.0 - httpcore==0.16.3 - httpx==0.23.3 - humanize==4.6.0 - hyperframe==6.0.1 - idna==3.3 - inflection==0.5.1 - iniconfig==2.0.0 - isort==5.12.0 - itsdangerous==2.1.2 - jsonpath-ng==1.5.3 - jsonschema==4.17.3 - kombu==5.2.4 - lazy-object-proxy==1.9.0 - ldap3==2.9.1 - linkify-it-py==2.0.0 - locket==1.0.0 - lockfile==0.12.2 - lxml==4.9.3 - mako==1.2.4 - markdown==3.4.3 - markdown-it-py==2.2.0 - marshmallow==3.19.0 - marshmallow-enum==1.5.1 - marshmallow-oneofschema==3.0.1 - marshmallow-sqlalchemy==0.26.1 - mcbackend==0.5.1 - mdit-py-plugins==0.3.5 - mdurl==0.1.2 - msgpack==1.0.7 - murefi==5.3.0 - mypy-boto3-appflow==1.26.78 - mypy-boto3-rds==1.26.102 - mypy-boto3-redshift-data==1.26.88 - mypy-extensions==1.0.0 - mysql-connector-python==8.0.32 - mysqlclient==2.1.1 - nest-asyncio==1.5.8 - paramiko==3.1.0 - partd==1.4.1 - pathspec==0.9.0 - peak-performance==0.6.4 - pendulum==2.1.2 - platformdirs==4.1.0 - ply==3.11 - prison==0.2.1 - prometheus-client==0.16.0 - prompt-toolkit==3.0.38 - protobuf==3.20.3 - psycopg2-binary==2.9.9 - pygments==2.14.0 - pyjwt==2.6.0 - pymc==5.10.0 - pynacl==1.5.0 - pyrff==2.0.2 - pyrsistent==0.19.3 - pytensor-federated==1.0.1 - pytest==7.4.3 - python-daemon==3.0.1 - python-dotenv==1.0.0 - python-nvd3==0.15.0 - python-slugify==8.0.1 - pytz==2022.1 - pytzdata==2020.1 - redis==3.5.3 - redshift-connector==2.0.910 - requests==2.31.0 - requests-toolbelt==0.10.1 - rfc3339-validator==0.1.4 - rfc3986==1.5.0 - rich==13.3.3 - robotools==1.8.0 - s3transfer==0.6.0 - scramp==1.4.4 - setproctitle==1.3.2 - slack-sdk==3.20.2 - sniffio==1.3.0 - sortedcontainers==2.4.0 - sqlalchemy==1.4.46 - sqlalchemy-jsonfield==1.0.1.post0 - sqlalchemy-redshift==0.8.13 - sqlalchemy-utils==0.40.0 - sqlparse==0.4.3 - sshtunnel==0.4.0 - stumpy==1.12.0 - tabulate==0.9.0 - tblib==3.0.0 - tenacity==8.2.2 - termcolor==2.2.0 - text-unidecode==1.3 - tornado==6.2 - tsfresh==0.20.1 - typing==3.7.4.3 - typing-extensions==4.1.0 - uc-micro-py==1.0.1 - unicodecsv==0.14.1 - urllib3==1.26.8 - vine==5.0.0 - watchtower==2.0.1 - wcwidth==0.2.6 - websocket-client==1.5.1 - werkzeug==2.2.3 - wtforms==3.0.1 - xarray==2023.11.0 - xarray-einstats==0.6.0 - zict==3.0.0 ```

michaelosthege commented 10 months ago

@lucianopaz does this make any sense to you?

python -c "import pytensor.link.c.cmodule as m;print(m.default_blas_ldflags())"

not interesting

``` DEBUG (pytensor.link.c.cmodule): Will search for BLAS libraries in the following directories: /opt/mamba/lib/gcc/x86_64-conda-linux-gnu/12.3.0 /opt/mamba/lib/gcc /opt/mamba/x86_64-conda-linux-gnu/lib /opt/mamba/lib /opt/mamba/x86_64-conda-linux-gnu/sysroot/lib64 /opt/mamba/x86_64-conda-linux-gnu/sysroot/usr/lib64 /opt/mamba/x86_64-conda-linux-gnu/lib /opt/mamba/lib /opt/mamba/x86_64-conda-linux-gnu/sysroot/lib64 /opt/mamba/x86_64-conda-linux-gnu/sysroot/usr/lib64 /opt/mamba/lib DEBUG (pytensor.link.c.cmodule): Checking MKL flags with intel threading INFO (pytensor.link.c.cmodule): g++ -march=native selected lines: ['/opt/mamba/bin/../libexec/gcc/x86_64-conda-linux-gnu/12.3.0/cc1 -E -quiet -v -iprefix /opt/mamba/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0/ -isysroot /opt/mamba/bin/../x86_64-conda-linux-gnu/sysroot - -march=bdver4 -mmmx -mpopcnt -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2 -mavx -mavx2 -msse4a -mno-fma4 -mno-xop -mfma -mno-avx512f -mbmi -mbmi2 -maes -mpclmul -mno-avx512vl -mno-avx512bw -mno-avx512dq -mno-avx512cd -mno-avx512er -mno-avx512pf -mno-avx512vbmi -mno-avx512ifma -mno-avx5124vnniw -mno-avx5124fmaps -mno-avx512vpopcntdq -mno-avx512vbmi2 -mno-gfni -mno-vpclmulqdq -mno-avx512vnni -mno-avx512bitalg -mno-avx512bf16 -mno-avx512vp2intersect -mno-3dnow -madx -mabm -mno-cldemote -mclflushopt -mno-clwb -mno-clzero -mcx16 -mno-enqcmd -mf16c -mfsgsbase -mfxsr -mno-hle -msahf -mno-lwp -mlzcnt -mmovbe -mno-movdir64b -mno-movdiri -mno-mwaitx -mno-pconfig -mno-pku -mno-prefetchwt1 -mprfchw -mno-ptwrite -mno-rdpid -mrdrnd -mrdseed -mno-rtm -mno-serialize -mno-sgx -msha -mno-shstk -mno-tbm -mno-tsxldtrk -mno-vaes -mno-waitpkg -mno-wbnoinvd -mxsave -mxsavec -mxsaveopt -mxsaves -mno-amx-tile -mno-amx-int8 -mno-amx-bf16 -mno-uintr -mno-hreset -mno-kl -mno-widekl -mno-avxvnni -mno-avx512fp16 --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=512 -mtune=bdver4 -dumpbase -'] INFO (pytensor.link.c.cmodule): g++ default lines: ['/opt/mamba/bin/../libexec/gcc/x86_64-conda-linux-gnu/12.3.0/cc1 -E -quiet -v -iprefix /opt/mamba/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0/ -isysroot /opt/mamba/bin/../x86_64-conda-linux-gnu/sysroot - -mtune=generic -march=x86-64 -dumpbase -'] INFO (pytensor.link.c.cmodule): g++ -march=native equivalent flags: ['-march=bdver4', '-mmmx', '-mpopcnt', '-msse', '-msse2', '-msse3', '-mssse3', '-msse4.1', '-msse4.2', '-mavx', '-mavx2', '-msse4a', '-mno-fma4', '-mno-xop', '-mfma', '-mno-avx512f', '-mbmi', '-mbmi2', '-maes', '-mpclmul', '-mno-avx512vl', '-mno-avx512bw', '-mno-avx512dq', '-mno-avx512cd', '-mno-avx512er', '-mno-avx512pf', '-mno-avx512vbmi', '-mno-avx512ifma', '-mno-avx5124vnniw', '-mno-avx5124fmaps', '-mno-avx512vpopcntdq', '-mno-avx512vbmi2', '-mno-gfni', '-mno-vpclmulqdq', '-mno-avx512vnni', '-mno-avx512bitalg', '-mno-avx512bf16', '-mno-avx512vp2intersect', '-mno-3dnow', '-madx', '-mabm', '-mno-cldemote', '-mclflushopt', '-mno-clwb', '-mno-clzero', '-mcx16', '-mno-enqcmd', '-mf16c', '-mfsgsbase', '-mfxsr', '-mno-hle', '-msahf', '-mno-lwp', '-mlzcnt', '-mmovbe', '-mno-movdir64b', '-mno-movdiri', '-mno-mwaitx', '-mno-pconfig', '-mno-pku', '-mno-prefetchwt1', '-mprfchw', '-mno-ptwrite', '-mno-rdpid', '-mrdrnd', '-mrdseed', '-mno-rtm', '-mno-serialize', '-mno-sgx', '-msha', '-mno-shstk', '-mno-tbm', '-mno-tsxldtrk', '-mno-vaes', '-mno-waitpkg', '-mno-wbnoinvd', '-mxsave', '-mxsavec', '-mxsaveopt', '-mxsaves', '-mno-amx-tile', '-mno-amx-int8', '-mno-amx-bf16', '-mno-uintr', '-mno-hreset', '-mno-kl', '-mno-widekl', '-mno-avxvnni', '-mno-avx512fp16', '--param', 'l1-cache-size=32', '--param', 'l1-cache-line-size=64', '--param', 'l2-cache-size=512', '-mtune=bdver4'] DEBUG (pytensor.link.c.cmodule): Will search for BLAS libraries in the following directories: /opt/mamba/lib/gcc/x86_64-conda-linux-gnu/12.3.0 /opt/mamba/lib/gcc /opt/mamba/x86_64-conda-linux-gnu/lib /opt/mamba/lib /opt/mamba/x86_64-conda-linux-gnu/sysroot/lib64 /opt/mamba/x86_64-conda-linux-gnu/sysroot/usr/lib64 /opt/mamba/x86_64-conda-linux-gnu/lib /opt/mamba/lib /opt/mamba/x86_64-conda-linux-gnu/sysroot/lib64 /opt/mamba/x86_64-conda-linux-gnu/sysroot/usr/lib64 /opt/mamba/lib DEBUG (pytensor.link.c.cmodule): Checking MKL flags with intel threading INFO (pytensor.link.c.cmodule): g++ -march=native selected lines: ['/opt/mamba/bin/../libexec/gcc/x86_64-conda-linux-gnu/12.3.0/cc1 -E -quiet -v -iprefix /opt/mamba/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0/ -isysroot /opt/mamba/bin/../x86_64-conda-linux-gnu/sysroot - -march=bdver4 -mmmx -mpopcnt -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2 -mavx -mavx2 -msse4a -mno-fma4 -mno-xop -mfma -mno-avx512f -mbmi -mbmi2 -maes -mpclmul -mno-avx512vl -mno-avx512bw -mno-avx512dq -mno-avx512cd -mno-avx512er -mno-avx512pf -mno-avx512vbmi -mno-avx512ifma -mno-avx5124vnniw -mno-avx5124fmaps -mno-avx512vpopcntdq -mno-avx512vbmi2 -mno-gfni -mno-vpclmulqdq -mno-avx512vnni -mno-avx512bitalg -mno-avx512bf16 -mno-avx512vp2intersect -mno-3dnow -madx -mabm -mno-cldemote -mclflushopt -mno-clwb -mno-clzero -mcx16 -mno-enqcmd -mf16c -mfsgsbase -mfxsr -mno-hle -msahf -mno-lwp -mlzcnt -mmovbe -mno-movdir64b -mno-movdiri -mno-mwaitx -mno-pconfig -mno-pku -mno-prefetchwt1 -mprfchw -mno-ptwrite -mno-rdpid -mrdrnd -mrdseed -mno-rtm -mno-serialize -mno-sgx -msha -mno-shstk -mno-tbm -mno-tsxldtrk -mno-vaes -mno-waitpkg -mno-wbnoinvd -mxsave -mxsavec -mxsaveopt -mxsaves -mno-amx-tile -mno-amx-int8 -mno-amx-bf16 -mno-uintr -mno-hreset -mno-kl -mno-widekl -mno-avxvnni -mno-avx512fp16 --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=512 -mtune=bdver4 -dumpbase -'] INFO (pytensor.link.c.cmodule): g++ default lines: ['/opt/mamba/bin/../libexec/gcc/x86_64-conda-linux-gnu/12.3.0/cc1 -E -quiet -v -iprefix /opt/mamba/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0/ -isysroot /opt/mamba/bin/../x86_64-conda-linux-gnu/sysroot - -mtune=generic -march=x86-64 -dumpbase -'] INFO (pytensor.link.c.cmodule): g++ -march=native equivalent flags: ['-march=bdver4', '-mmmx', '-mpopcnt', '-msse', '-msse2', '-msse3', '-mssse3', '-msse4.1', '-msse4.2', '-mavx', '-mavx2', '-msse4a', '-mno-fma4', '-mno-xop', '-mfma', '-mno-avx512f', '-mbmi', '-mbmi2', '-maes', '-mpclmul', '-mno-avx512vl', '-mno-avx512bw', '-mno-avx512dq', '-mno-avx512cd', '-mno-avx512er', '-mno-avx512pf', '-mno-avx512vbmi', '-mno-avx512ifma', '-mno-avx5124vnniw', '-mno-avx5124fmaps', '-mno-avx512vpopcntdq', '-mno-avx512vbmi2', '-mno-gfni', '-mno-vpclmulqdq', '-mno-avx512vnni', '-mno-avx512bitalg', '-mno-avx512bf16', '-mno-avx512vp2intersect', '-mno-3dnow', '-madx', '-mabm', '-mno-cldemote', '-mclflushopt', '-mno-clwb', '-mno-clzero', '-mcx16', '-mno-enqcmd', '-mf16c', '-mfsgsbase', '-mfxsr', '-mno-hle', '-msahf', '-mno-lwp', '-mlzcnt', '-mmovbe', '-mno-movdir64b', '-mno-movdiri', '-mno-mwaitx', '-mno-pconfig', '-mno-pku', '-mno-prefetchwt1', '-mprfchw', '-mno-ptwrite', '-mno-rdpid', '-mrdrnd', '-mrdseed', '-mno-rtm', '-mno-serialize', '-mno-sgx', '-msha', '-mno-shstk', '-mno-tbm', '-mno-tsxldtrk', '-mno-vaes', '-mno-waitpkg', '-mno-wbnoinvd', '-mxsave', '-mxsavec', '-mxsaveopt', '-mxsaves', '-mno-amx-tile', '-mno-amx-int8', '-mno-amx-bf16', '-mno-uintr', '-mno-hreset', '-mno-kl', '-mno-widekl', '-mno-avxvnni', '-mno-avx512fp16', '--param', 'l1-cache-size=32', '--param', 'l1-cache-line-size=64', '--param', 'l2-cache-size=512', '-mtune=bdver4'] DEBUG (pytensor.link.c.cmodule): Compiling for 64 bit architecture DEBUG (pytensor.link.c.cmodule): Compiling for 64 bit architecture DEBUG (pytensor.link.c.cmodule): Compiling for 64 bit architecture DEBUG (pytensor.link.c.cmodule): Compiling for 64 bit architecture DEBUG (pytensor.link.c.cmodule): Compiling for 64 bit architecture DEBUG (pytensor.link.c.cmodule): Compiling for 64 bit architecture DEBUG (pytensor.link.c.cmodule): Compiling for 64 bit architecture DEBUG (pytensor.link.c.cmodule): Compiling for 64 bit architecture DEBUG (pytensor.link.c.cmodule): Compiling for 64 bit architecture DEBUG (pytensor.link.c.cmodule): Supplied flags '['-lmkl_core', '-lmkl_rt', '-lmkl_intel_thread', '-liomp5', '-lpthread', '-Wl,-rpath,/opt/mamba/lib']' failed to compile DEBUG (pytensor.link.c.cmodule): Supplied flags ['-lmkl_core', '-lmkl_rt', '-lmkl_intel_thread', '-liomp5', '-lpthread', '-Wl,-rpath,/opt/mamba/lib'] failed to compile DEBUG (pytensor.link.c.cmodule): Checking MKL flags with GNU OpenMP threading DEBUG (pytensor.link.c.cmodule): Compiling for 64 bit architecture DEBUG (pytensor.link.c.cmodule): Supplied flags '['-lmkl_core', '-lmkl_rt', '-lmkl_gnu_thread', '-lgomp', '-lpthread', '-Wl,-rpath,/opt/mamba/lib']' failed to compile DEBUG (pytensor.link.c.cmodule): Supplied flags ['-lmkl_core', '-lmkl_rt', '-lmkl_gnu_thread', '-lgomp', '-lpthread', '-Wl,-rpath,/opt/mamba/lib'] failed to compile DEBUG (pytensor.link.c.cmodule): Checking Lapack + blas DEBUG (pytensor.link.c.cmodule): Compiling for 64 bit architecture DEBUG (pytensor.link.c.cmodule): Supplied flags '['-llapack', '-lblas', '-lcblas', '-lm', '-Wl,-rpath,/opt/mamba/lib']' failed to compile DEBUG (pytensor.link.c.cmodule): Supplied flags ['-llapack', '-lblas', '-lcblas', '-lm', '-Wl,-rpath,/opt/mamba/lib'] failed to compile DEBUG (pytensor.link.c.cmodule): Checking blas alone DEBUG (pytensor.link.c.cmodule): Compiling for 64 bit architecture DEBUG (pytensor.link.c.cmodule): Supplied flags '['-lblas', '-lcblas', '-Wl,-rpath,/opt/mamba/lib']' failed to compile DEBUG (pytensor.link.c.cmodule): Supplied flags ['-lblas', '-lcblas', '-Wl,-rpath,/opt/mamba/lib'] failed to compile DEBUG (pytensor.link.c.cmodule): Checking openblas DEBUG (pytensor.link.c.cmodule): Compiling for 64 bit architecture DEBUG (pytensor.link.c.cmodule): Supplied flags '['-lopenblas', '-lgfortran', '-lgomp', '-lm', '-fopenmp', '-Wl,-rpath,/opt/mamba/lib']' failed to compile DEBUG (pytensor.link.c.cmodule): Supplied flags ['-lopenblas', '-lgfortran', '-lgomp', '-lm', '-fopenmp', '-Wl,-rpath,/opt/mamba/lib'] failed to compile DEBUG (pytensor.link.c.cmodule): Failed to identify blas ldflags. Will leave them empty. WARNING (pytensor.tensor.blas): Using NumPy C-API based implementation for BLAS functions. DEBUG (pytensor.link.c.cmodule): Will search for BLAS libraries in the following directories: /opt/mamba/lib/gcc/x86_64-conda-linux-gnu/12.3.0 /opt/mamba/lib/gcc /opt/mamba/x86_64-conda-linux-gnu/lib /opt/mamba/lib /opt/mamba/x86_64-conda-linux-gnu/sysroot/lib64 /opt/mamba/x86_64-conda-linux-gnu/sysroot/usr/lib64 /opt/mamba/x86_64-conda-linux-gnu/lib /opt/mamba/lib /opt/mamba/x86_64-conda-linux-gnu/sysroot/lib64 /opt/mamba/x86_64-conda-linux-gnu/sysroot/usr/lib64 /opt/mamba/lib DEBUG (pytensor.link.c.cmodule): Checking MKL flags with intel threading DEBUG (pytensor.link.c.cmodule): Compiling for 64 bit architecture DEBUG (pytensor.link.c.cmodule): Supplied flags '['-lmkl_core', '-lmkl_rt', '-lmkl_intel_thread', '-liomp5', '-lpthread', '-Wl,-rpath,/opt/mamba/lib']' failed to compile DEBUG (pytensor.link.c.cmodule): Supplied flags ['-lmkl_core', '-lmkl_rt', '-lmkl_intel_thread', '-liomp5', '-lpthread', '-Wl,-rpath,/opt/mamba/lib'] failed to compile DEBUG (pytensor.link.c.cmodule): Checking MKL flags with GNU OpenMP threading DEBUG (pytensor.link.c.cmodule): Compiling for 64 bit architecture DEBUG (pytensor.link.c.cmodule): Supplied flags '['-lmkl_core', '-lmkl_rt', '-lmkl_gnu_thread', '-lgomp', '-lpthread', '-Wl,-rpath,/opt/mamba/lib']' failed to compile DEBUG (pytensor.link.c.cmodule): Supplied flags ['-lmkl_core', '-lmkl_rt', '-lmkl_gnu_thread', '-lgomp', '-lpthread', '-Wl,-rpath,/opt/mamba/lib'] failed to compile DEBUG (pytensor.link.c.cmodule): Checking Lapack + blas DEBUG (pytensor.link.c.cmodule): Compiling for 64 bit architecture DEBUG (pytensor.link.c.cmodule): Supplied flags '['-llapack', '-lblas', '-lcblas', '-lm', '-Wl,-rpath,/opt/mamba/lib']' failed to compile DEBUG (pytensor.link.c.cmodule): Supplied flags ['-llapack', '-lblas', '-lcblas', '-lm', '-Wl,-rpath,/opt/mamba/lib'] failed to compile DEBUG (pytensor.link.c.cmodule): Checking blas alone DEBUG (pytensor.link.c.cmodule): Compiling for 64 bit architecture DEBUG (pytensor.link.c.cmodule): Supplied flags '['-lblas', '-lcblas', '-Wl,-rpath,/opt/mamba/lib']' failed to compile DEBUG (pytensor.link.c.cmodule): Supplied flags ['-lblas', '-lcblas', '-Wl,-rpath,/opt/mamba/lib'] failed to compile DEBUG (pytensor.link.c.cmodule): Checking openblas DEBUG (pytensor.link.c.cmodule): Compiling for 64 bit architecture DEBUG (pytensor.link.c.cmodule): Supplied flags '['-lopenblas', '-lgfortran', '-lgomp', '-lm', '-fopenmp', '-Wl,-rpath,/opt/mamba/lib']' failed to compile DEBUG (pytensor.link.c.cmodule): Supplied flags ['-lopenblas', '-lgfortran', '-lgomp', '-lm', '-fopenmp', '-Wl,-rpath,/opt/mamba/lib'] failed to compile DEBUG (pytensor.link.c.cmodule): Failed to identify blas ldflags. Will leave them empty. ```

I added even more logging:

DEBUG (pytensor.link.c.cmodule): Checking MKL flags with intel threading
DEBUG (pytensor.link.c.cmodule): Compiling for 64 bit architecture
DEBUG (pytensor.link.c.cmodule): Flags ['-lmkl_core', '-lmkl_rt', '-lmkl_intel_thread', '-liomp5', '-lpthread', '-Wl,-rpath,/opt/mamba/lib', '-L/opt/mamba/lib'] compilation exited 1 with
stdout=b''
stderr=b'/opt/mamba/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: cannot find /usr/lib64/libpthread_nonshared.a inside /opt/mamba/bin/../x86_64-conda-linux-gnu/sysroot\ncollect2: error: ld returned 1 exit status\n'
DEBUG (pytensor.link.c.cmodule): Flags ['-lmkl_core', '-lmkl_rt', '-lmkl_intel_thread', '-liomp5', '-lpthread', '-Wl,-rpath,/opt/mamba/lib', '-L/opt/mamba/lib'] → b''
DEBUG (pytensor.link.c.cmodule): Supplied flags '['-lmkl_core', '-lmkl_rt', '-lmkl_intel_thread', '-liomp5', '-lpthread', '-Wl,-rpath,/opt/mamba/lib']' failed to compile
DEBUG (pytensor.link.c.cmodule): Supplied flags ['-lmkl_core', '-lmkl_rt', '-lmkl_intel_thread', '-liomp5', '-lpthread', '-Wl,-rpath,/opt/mamba/lib'] failed to compile
DEBUG (pytensor.link.c.cmodule): Checking MKL flags with GNU OpenMP threading
DEBUG (pytensor.link.c.cmodule): Compiling for 64 bit architecture
DEBUG (pytensor.link.c.cmodule): Flags ['-lmkl_core', '-lmkl_rt', '-lmkl_gnu_thread', '-lgomp', '-lpthread', '-Wl,-rpath,/opt/mamba/lib', '-L/opt/mamba/lib'] compilation exited 1 with
stdout=b''
stderr=b'/opt/mamba/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: cannot find /usr/lib64/libpthread_nonshared.a inside /opt/mamba/bin/../x86_64-conda-linux-gnu/sysroot\ncollect2: error: ld returned 1 exit status\n'
DEBUG (pytensor.link.c.cmodule): Flags ['-lmkl_core', '-lmkl_rt', '-lmkl_gnu_thread', '-lgomp', '-lpthread', '-Wl,-rpath,/opt/mamba/lib', '-L/opt/mamba/lib'] → b''
DEBUG (pytensor.link.c.cmodule): Supplied flags '['-lmkl_core', '-lmkl_rt', '-lmkl_gnu_thread', '-lgomp', '-lpthread', '-Wl,-rpath,/opt/mamba/lib']' failed to compile
DEBUG (pytensor.link.c.cmodule): Supplied flags ['-lmkl_core', '-lmkl_rt', '-lmkl_gnu_thread', '-lgomp', '-lpthread', '-Wl,-rpath,/opt/mamba/lib'] failed to compile
DEBUG (pytensor.link.c.cmodule): Checking Lapack + blas
DEBUG (pytensor.link.c.cmodule): Compiling for 64 bit architecture
DEBUG (pytensor.link.c.cmodule): Flags ['-llapack', '-lblas', '-lcblas', '-lm', '-Wl,-rpath,/opt/mamba/lib', '-L/opt/mamba/lib'] compilation exited 1 with
stdout=b''
stderr=b'/opt/mamba/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: cannot find -lgcc: No such file or directory\ncollect2: error: ld returned 1 exit status\n'
DEBUG (pytensor.link.c.cmodule): Flags ['-llapack', '-lblas', '-lcblas', '-lm', '-Wl,-rpath,/opt/mamba/lib', '-L/opt/mamba/lib'] → b''
DEBUG (pytensor.link.c.cmodule): Supplied flags '['-llapack', '-lblas', '-lcblas', '-lm', '-Wl,-rpath,/opt/mamba/lib']' failed to compile
DEBUG (pytensor.link.c.cmodule): Supplied flags ['-llapack', '-lblas', '-lcblas', '-lm', '-Wl,-rpath,/opt/mamba/lib'] failed to compile
DEBUG (pytensor.link.c.cmodule): Checking blas alone
DEBUG (pytensor.link.c.cmodule): Compiling for 64 bit architecture
DEBUG (pytensor.link.c.cmodule): Flags ['-lblas', '-lcblas', '-Wl,-rpath,/opt/mamba/lib', '-L/opt/mamba/lib'] compilation exited 1 with
stdout=b''
stderr=b'/opt/mamba/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: cannot find -lgcc: No such file or directory\ncollect2: error: ld returned 1 exit status\n'
DEBUG (pytensor.link.c.cmodule): Flags ['-lblas', '-lcblas', '-Wl,-rpath,/opt/mamba/lib', '-L/opt/mamba/lib'] → b''
DEBUG (pytensor.link.c.cmodule): Supplied flags '['-lblas', '-lcblas', '-Wl,-rpath,/opt/mamba/lib']' failed to compile
DEBUG (pytensor.link.c.cmodule): Supplied flags ['-lblas', '-lcblas', '-Wl,-rpath,/opt/mamba/lib'] failed to compile
DEBUG (pytensor.link.c.cmodule): Checking openblas
DEBUG (pytensor.link.c.cmodule): Compiling for 64 bit architecture
DEBUG (pytensor.link.c.cmodule): Flags ['-lopenblas', '-lgfortran', '-lgomp', '-lm', '-fopenmp', '-Wl,-rpath,/opt/mamba/lib', '-L/opt/mamba/lib'] compilation exited 1 with
stdout=b''
stderr=b'/opt/mamba/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: cannot find -lgcc: No such file or directory\ncollect2: error: ld returned 1 exit status\n'
DEBUG (pytensor.link.c.cmodule): Flags ['-lopenblas', '-lgfortran', '-lgomp', '-lm', '-fopenmp', '-Wl,-rpath,/opt/mamba/lib', '-L/opt/mamba/lib'] → b''
DEBUG (pytensor.link.c.cmodule): Supplied flags '['-lopenblas', '-lgfortran', '-lgomp', '-lm', '-fopenmp', '-Wl,-rpath,/opt/mamba/lib']' failed to compile
DEBUG (pytensor.link.c.cmodule): Supplied flags ['-lopenblas', '-lgfortran', '-lgomp', '-lm', '-fopenmp', '-Wl,-rpath,/opt/mamba/lib'] failed to compile

A lot of compilation attempts fail with this error:

/opt/mamba/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0/../../../../x86_64-conda-linux-gnu/bin/ld:
cannot find -lgcc: No such file or directory
collect2: error: ld returned 1 exit status

lucianopaz commented 10 months ago

@michaelosthege, can you manage to enter the container and look for the file called libgcc.so? It should be in there somewhere but for some weird reason the compiler doesn't know where to look for it. While you're at it, you could also try to look for /usr/lib64/libpthread_nonshared.a. It seems very strange to me that the linker cannot find either of these libraries. It looks slightly related to this old issue from pandas. Maybe you can apply the patches that the people suggested at the end?

michaelosthege commented 10 months ago

First I simplified my Dockerfile to make a MRE:

FROM ubuntu:22.04

RUN apt-get update -yqq \
    && apt-get upgrade -yqq \
    && apt-get install -yqq --no-install-recommends \
        ca-certificates \
        git \
        wget

# Install mamba (base environment will be used for everything)
ENV PATH="/opt/mamba/bin:${PATH}"
ARG PATH="/opt/mamba/bin:${PATH}"
RUN wget --quiet https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Linux-x86_64.sh -O ~/mambaforge.sh \
    && /bin/bash ~/mambaforge.sh -b -p /opt/mamba \
    && rm ~/mambaforge.sh \
    && echo ". /opt/mamba/etc/profile.d/mamba.sh" >> ~/.bashrc \
    && echo "mamba activate base" >> ~/.bashrc

RUN mkdir /dependencies
COPY environment.yml /dependencies/environment.yml
RUN mamba env update --file /dependencies/environment.yml --name base \
    # Clean up to reduce the size of the resulting Docker image layer
    && mamba clean --all \
    && find /opt/mamba/ -type f,l -name '*.a' -delete \
    && find /opt/mamba/ -type f,l -name '*.pyc' -delete \
    && find /opt/mamba/ -type f,l -name '*.js.map' -delete \
    && rm -rf /opt/mamba/pkgs

channels:
- conda-forge
dependencies:
- numpy==1.24.4
- pandas>=1.5.3,<2
- python-graphviz
- pip
- python>=3.10,<3.11
#- pymc==5.10.3                 # enabling this line breaks it
- pip:
  - pymc==5.10.3

For testing:

# check availability of relevant locales
import logging
from pathlib import Path

import arviz
import graphviz
import numpy
import pandas
import pymc
import pytensor
import xarray

logging.basicConfig(
    format='%(levelname).1s [%(asctime)s] [%(name)s:%(lineno)d]: %(message)s', datefmt='%Y-%m-%d %H:%M:%S',
    level=logging.DEBUG,
)

def test_pymc_sampling():
    """Checking that standard NUTS sampling works."""
    with pymc.Model():
        pymc.Normal("n")
        pymc.sample(tune=10, draws=20, compute_convergence_checks=False)
    print("✔ Can sample with NUTS.")
    return

def test_pymc_graphviz():
    """Test the graphviz installation by plotting a PyMC model"""
    with pymc.Model() as pmodel:
        pymc.Normal("n")
    graph = pymc.model_to_graphviz(pmodel)
    graph.render(format="png", filename="modelgraph")
    assert Path("modelgraph.png").exists()
    graph.render(format="pdf", filename="modelgraph")
    assert Path("modelgraph.pdf").exists()
    # Clean up
    Path("modelgraph").unlink()
    Path("modelgraph.png").unlink()
    Path("modelgraph.pdf").unlink()
    print("🥳 Can print PyMC model graphs!")
    return

def test_pymc_jax():
    """Testing if also JAX/numpyro can be used."""
    import pymc.sampling_jax

    with pymc.Model():
        pymc.Normal("n")
        pymc.sampling_jax.sample_numpyro_nuts(tune=10, draws=20)
    print("🥳 Can run MCMC with JAX backend!")
    return

if __name__ == "__main__":
    test_pymc_sampling()
    test_pymc_graphviz()
    # Disabled because of dependency conflicts (as of 2023-12-15)
    # test_pymc_jax()

And a convenience script to run all of them:

cls
docker image rm mre:local
docker build -t mre:local .
docker run --rm -v %cd%:/wd mre:local python /wd/test.py

pymc-devs / pytensor

Empty BLAS flags in mamba-installed v2.18.1 environment under Docker #523

Description