search

fermi-lat / Fermitools-conda

Conda recipe files for the Fermi Sciencetools software analysis package: Fermitools
https://fermi.gsfc.nasa.gov/ssc/data/analysis/
BSD 3-Clause "New" or "Revised" License
34 stars 17 forks source link

Fermitools installation taking far too long #111

Closed BrunoArsioli closed 2 years ago

BrunoArsioli commented 3 years ago

Hi, I've tried to install fermitools in a MacOS 11.5.2, following instructions from fermi-wiki. I started by installing Miniconda (Miniconda3 MacOSX 64-bit pkg) and then, with zsh shell:

conda create -n fermi -c conda-forge -c fermi fermitools python=3 clhep=2.4.4.1

A series of packages were downloaded, and then I got the following messages:

"Preparing transaction: done Verifying transaction: done Executing transaction: / "

The latest one "Executing transaction: " has been running for many hours . (> 10 h)

I let it run that long, because I found many comments about users having similar problems with other packages (not fermitools) when trying to use "conda create -c conda-forge" .

There are some posts that talk about using "conda-metachannel" instead of conda-forge , and also here. Is it safe to use conda-metachannel, trying to optimize the installation?

(p.s. my uname -a :

Darwin MBP-de-xxx 20.6.0 Darwin Kernel Version 20.6.0: Wed Jun 23 00:26:31 PDT 2021; root:xnu-7195.141.2~5/RELEASE_X86_64 x86_64 )

donhorner commented 3 years ago

There certainly seems to be something wrong if it's taking that long. You can look at the troubleshooting suggestions, but I think the one that I would suggest is installing mamba, which is reimplementation of the conda package manager in C++ and is generally faster at resolving the dependencies. You can install it with the conda command conda install mamba -c conda-forge and then use it in place of the conda command (e.g., 'mamba install' instead of 'conda install').

I don't know anything about conda-metachannel. Maybe @jasercion can comment on that.

BrunoArsioli commented 3 years ago

Hi @donhorner, I just installed mamba, and tried:

mamba create -n fermi -c conda-forge -c fermi fermitools python=3 clhep=2.4.4.1

I got :

WARNING: A conda environment already exists at '/Users/bs/opt/miniconda3/envs/fermi' Remove existing environment (y/[n])? y

so, I removed envs/fermi (from the previous trial with conda create -n fermi ... )

The installation moved on, downloading some extra packages, and again, it went for many hours with this message:

" Preparing transaction: done Verifying transaction: done Executing transaction: \ "

with the never ending "Executing transaction"

BrunoArsioli commented 3 years ago

After removing Miniconda and installing Anaconda, it does not helped to solve the issue. After running for long time:

conda create -n fermi -c conda-forge -c fermi fermitools python=3 clhep=2.4.4.1

hitting ctrl+c some few times (4x), showed this error message (before interrupting the process): ERROR conda.core.link:_execute(699): An error occurred while installing package 'conda-forge::gdk-pixbuf-2.42.6-h2e6141f_0'.

I am not sure if this helps. Is 'gdk-pixbuf' related to where the process is stuck?

(also, I tried to add -c main, because I read this could simplify the optimization process : conda create -n fermi -c main -c conda-forge -c fermi fermitools python=3 clhep=2.4.4.1 but the situation keeps the same and the same error shows up after ctrl+c)

donhorner commented 3 years ago

We haven't been able to replicate the problem. Can you try installing a development version of the tools using this command?
conda create -n fermi-2.0.24 -c conda-forge -c fermi/label/dev fermitools=2.0.24 The dev version solved a problem with the clhep library so specifying it with the conda install is no longer necessary. I'm wondering if it might work better.

BrunoArsioli commented 3 years ago

Hi @donhorner , thank you for the attention. I tried that, but it didn't work. It is the same issue: never ending "Executing transaction: / "

I also tried to update to MacOS BigSur 11.6, then:

conda update --all

conda update mamba

conda create -n fermi -c conda-forge -c fermi fermitools python=3 clhep=2.4.4.1

or, for the dev version:

conda create -n fermi-2.0.24 -c conda-forge -c fermi/label/dev fermitools=2.0.24

or:

mamba create -n fermi -c conda-forge -c fermi fermitools python=3 clhep=2.4.4.1

All trials get stuck in " executing transaction \ " (> 1h)

when hitting ctrl+c, all show the same error msg:

" ERROR conda.core.link:_execute(699): An error occurred while installing package 'conda-forge::gdk-pixbuf-2.42.6-h2e6141f_0'. " " Rolling back transaction: done "

my uname -v

Darwin Kernel Version 20.6.0: Mon Aug 30 06:12:21 PDT 2021; root:xnu-7195.141.6~3/RELEASE_X86_64

Areustle commented 3 years ago

Hi @BrunoArsioli . I am unsure what the cause of your conda/mamba install issue is, but since it seems to be related to your system's ability to resolve a suitable environment, @donhorner has generated an explicit one for you. I would like you to install this spec using the --file flag of conda create, which you can read about more by running conda create --help. Attached is a text file listing conda package URLs named spec-file.txt. Please download this file to your system, then run conda create -n fermi --file /path/to/spec-file.txt (substituting /path/to/ appropriately). This should skip the environment resolution step. Afterwards you should be able conda activate fermi and proceed with using the fermitools.

The MD5 fingerprint for spec-file.txt is 9dbd3ba14cb8c7d13c2506a56e1ec1d5 which you can check on your mac by running md5 /path/to/spec-file.txt.

BrunoArsioli commented 3 years ago

I just notice now that when I tried conda create -n fermi -c conda-forge -c fermi fermitools python=3 clhep=2.4.4.1 , I get:

Collecting package metadata (current_repodata.json): done Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source. Collecting package metadata (repodata.json): done Solving environment: -

similar to #112 .

after some minutes, it actually goes through the Solving environment: - like this:

=============

Collecting package metadata (current_repodata.json): done Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source. Collecting package metadata (repodata.json): done Solving environment: done

Package Plan

environment location: /opt/anaconda3/envs/fermi

added / updated specs:

The following packages will be downloaded:

package                    |            build
---------------------------|-----------------
hypothesis-6.21.4          |     pyhd8ed1ab_0         249 KB  conda-forge
pyzmq-22.3.0               |   py37h8f778e5_0         482 KB  conda-forge
------------------------------------------------------------
                                       Total:         731 KB

The following NEW packages will be INSTALLED:

afterimage conda-forge/osx-64::afterimage-1.21-h4dd67e6_1003 ape fermi/osx-64::ape-3.0.0-h1de35cc_1 aplpy conda-forge/noarch::aplpy-2.0.3-py_1 appnope conda-forge/osx-64::appnope-0.1.2-py37hf985489_1 argcomplete conda-forge/noarch::argcomplete-1.12.3-pyhd8ed1ab_2 argon2-cffi conda-forge/osx-64::argon2-cffi-20.1.0-py37h271585c_2 astropy conda-forge/osx-64::astropy-3.2.3-py37h0b31af3_0 astropy-healpix conda-forge/osx-64::astropy-healpix-0.6-py37h032687b_0 async_generator conda-forge/noarch::async_generator-1.10-py_0 atk-1.0 conda-forge/osx-64::atk-1.0-2.36.0-he69c4ee_4 attrs conda-forge/noarch::attrs-21.2.0-pyhd8ed1ab_0 backcall conda-forge/noarch::backcall-0.2.0-pyh9f0ad1d_0 backports conda-forge/noarch::backports-1.0-py_2 backports.functoo~ conda-forge/noarch::backports.functools_lru_cache-1.6.4-pyhd8ed1ab_0 blas conda-forge/osx-64::blas-2.17-openblas bleach conda-forge/noarch::bleach-4.1.0-pyhd8ed1ab_0 bzip2 conda-forge/osx-64::bzip2-1.0.8-h0d85af4_4 c-ares conda-forge/osx-64::c-ares-1.17.2-h0d85af4_0 c-compiler conda-forge/osx-64::c-compiler-1.1.2-hc929b4f_0 ca-certificates conda-forge/osx-64::ca-certificates-2021.5.30-h033912b_0 cairo conda-forge/osx-64::cairo-1.16.0-he43a7df_1008 cctools conda-forge/osx-64::cctools-949.0.1-hd9211c8_21 cctools_osx-64 conda-forge/osx-64::cctools_osx-64-949.0.1-h6407bdd_21 certifi conda-forge/osx-64::certifi-2021.5.30-py37hf985489_0 cffi conda-forge/osx-64::cffi-1.14.6-py37h446072c_1 cfitsio conda-forge/osx-64::cfitsio-3.470-h422484a_7 clang conda-forge/osx-64::clang-11.1.0-h694c41f_1 clang-11 conda-forge/osx-64::clang-11-11.1.0-default_he082bbe_1 clang_osx-64 conda-forge/osx-64::clang_osx-64-11.1.0-hb91bd55_3 clangxx conda-forge/osx-64::clangxx-11.1.0-default_he082bbe_1 clangxx_osx-64 conda-forge/osx-64::clangxx_osx-64-11.1.0-h7e1b574_3 clhep conda-forge/osx-64::clhep-2.4.4.1-h046ec9c_0 click conda-forge/osx-64::click-8.0.1-py37hf985489_0 cloudpickle conda-forge/noarch::cloudpickle-2.0.0-pyhd8ed1ab_0 colorama conda-forge/noarch::colorama-0.4.4-pyh9f0ad1d_0 compiler-rt conda-forge/osx-64::compiler-rt-11.1.0-h654b07c_0 compiler-rt_osx-64 conda-forge/noarch::compiler-rt_osx-64-11.1.0-h8c5fa43_0 compilers conda-forge/osx-64::compilers-1.1.2-h694c41f_0 coverage conda-forge/osx-64::coverage-5.5-py37h271585c_0 cppunit conda-forge/osx-64::cppunit-1.15.1-he49afe7_0 cxx-compiler conda-forge/osx-64::cxx-compiler-1.1.2-h9a9d8cb_0 cycler conda-forge/noarch::cycler-0.10.0-py_2 cytoolz conda-forge/osx-64::cytoolz-0.11.0-py37h271585c_3 dask-core conda-forge/noarch::dask-core-2021.9.0-pyhd8ed1ab_0 davix conda-forge/osx-64::davix-0.7.6-h61356a0_0 debugpy conda-forge/osx-64::debugpy-1.4.1-py37hd8d24ac_0 decorator conda-forge/noarch::decorator-5.1.0-pyhd8ed1ab_0 defusedxml conda-forge/noarch::defusedxml-0.7.1-pyhd8ed1ab_0 entrypoints conda-forge/noarch::entrypoints-0.3-pyhd8ed1ab_1003 expat conda-forge/osx-64::expat-2.4.1-he49afe7_0 f2c fermi/osx-64::f2c-0.0.2-h1de35cc_0 fermitools fermi/osx-64::fermitools-2.0.8-hb189bb9_0 fermitools-data fermi/noarch::fermitools-data-0.18-0 fermitools-depend~ fermi/osx-64::fermitools-dependencies-2.0.8-0 fftw conda-forge/osx-64::fftw-3.3.8-nompi_h27769e0_1114 font-ttf-dejavu-s~ conda-forge/noarch::font-ttf-dejavu-sans-mono-2.37-hab24e00_0 font-ttf-inconsol~ conda-forge/noarch::font-ttf-inconsolata-3.000-h77eed37_0 font-ttf-source-c~ conda-forge/noarch::font-ttf-source-code-pro-2.038-h77eed37_0 font-ttf-ubuntu conda-forge/noarch::font-ttf-ubuntu-0.83-hab24e00_0 fontconfig conda-forge/osx-64::fontconfig-2.13.1-h10f422b_1005 fonts-conda-ecosy~ conda-forge/noarch::fonts-conda-ecosystem-1-0 fonts-conda-forge conda-forge/noarch::fonts-conda-forge-1-0 fortran-compiler conda-forge/osx-64::fortran-compiler-1.1.2-hcc3dc05_0 freetype conda-forge/osx-64::freetype-2.10.4-h4cff582_1 fribidi conda-forge/osx-64::fribidi-1.0.10-hbcb3906_0 fsspec conda-forge/noarch::fsspec-2021.8.1-pyhd8ed1ab_0 ftgl conda-forge/osx-64::ftgl-2.4.0-h508da5b_0 gdk-pixbuf conda-forge/osx-64::gdk-pixbuf-2.42.6-h2e6141f_0 geos conda-forge/osx-64::geos-3.9.1-he49afe7_2 gettext conda-forge/osx-64::gettext-0.19.8.1-hd1a6beb_1006 gfortran_impl_osx~ conda-forge/osx-64::gfortran_impl_osx-64-7.5.0-hdadd0f8_23 gfortran_osx-64 conda-forge/osx-64::gfortran_osx-64-7.5.0-h18f7dce_14 giflib conda-forge/osx-64::giflib-5.2.1-hbcb3906_2 gl2ps conda-forge/osx-64::gl2ps-1.4.2-h4cff582_0 glew conda-forge/osx-64::glew-2.1.0-h046ec9c_2 gmp conda-forge/osx-64::gmp-6.2.1-h2e338ed_0 graphite2 conda-forge/osx-64::graphite2-1.3.13-h2e338ed_1001 graphviz conda-forge/osx-64::graphviz-2.49.0-h77de9ca_0 gsl conda-forge/osx-64::gsl-2.6-h71c5fe9_2 gtk2 conda-forge/osx-64::gtk2-2.24.33-h675d97a_1 gts conda-forge/osx-64::gts-0.7.6-hccb3bdf_2 harfbuzz conda-forge/osx-64::harfbuzz-2.9.1-h159f659_0 healpix_cxx conda-forge/osx-64::healpix_cxx-3.31-h37507ba_1003 hypothesis conda-forge/noarch::hypothesis-6.21.4-pyhd8ed1ab_0 icu conda-forge/osx-64::icu-68.1-h74dc148_0 imageio conda-forge/noarch::imageio-2.9.0-py_0 importlib-metadata conda-forge/osx-64::importlib-metadata-4.8.1-py37hf985489_0 importlib_metadata conda-forge/noarch::importlib_metadata-4.8.1-hd8ed1ab_0 iniconfig conda-forge/noarch::iniconfig-1.1.1-pyh9f0ad1d_0 ipykernel conda-forge/osx-64::ipykernel-6.4.0-py37h4c52d7d_0 ipyparallel conda-forge/noarch::ipyparallel-7.0.1-pyhd8ed1ab_0 ipython conda-forge/osx-64::ipython-7.27.0-py37h4c52d7d_0 ipython_genutils conda-forge/noarch::ipython_genutils-0.2.0-py_1 isl conda-forge/osx-64::isl-0.22.1-hb1e8313_2 jedi conda-forge/osx-64::jedi-0.18.0-py37hf985489_2 jinja2 conda-forge/noarch::jinja2-3.0.1-pyhd8ed1ab_0 jpeg conda-forge/osx-64::jpeg-9d-hbcb3906_0 jsonschema conda-forge/noarch::jsonschema-3.2.0-pyhd8ed1ab_3 jupyter_client conda-forge/noarch::jupyter_client-7.0.2-pyhd8ed1ab_0 jupyter_core conda-forge/osx-64::jupyter_core-4.7.1-py37hf985489_0 jupyterlab_pygmen~ conda-forge/noarch::jupyterlab_pygments-0.1.2-pyh9f0ad1d_0 kiwisolver conda-forge/osx-64::kiwisolver-1.3.2-py37h737db71_0 krb5 conda-forge/osx-64::krb5-1.19.2-hcfbf3a7_0 lcms2 conda-forge/osx-64::lcms2-2.12-h577c468_0 ld64 conda-forge/osx-64::ld64-530-hd2e7500_21 ld64_osx-64 conda-forge/osx-64::ld64_osx-64-530-he8994da_21 ldid conda-forge/osx-64::ldid-2.1.2-h7660a38_2 libblas conda-forge/osx-64::libblas-3.8.0-17_openblas libcblas conda-forge/osx-64::libcblas-3.8.0-17_openblas libclang-cpp11.1 conda-forge/osx-64::libclang-cpp11.1-11.1.0-default_he082bbe_1 libcurl conda-forge/osx-64::libcurl-7.79.0-hf45b732_0 libcxx conda-forge/osx-64::libcxx-11.1.0-habf9029_0 libedit conda-forge/osx-64::libedit-3.1.20191231-h0678c8f_2 libev conda-forge/osx-64::libev-4.33-haf1e3a3_1 libffi conda-forge/osx-64::libffi-3.4.2-he49afe7_1 libgd conda-forge/osx-64::libgd-2.3.2-h4e7a7ea_0 libgfortran conda-forge/osx-64::libgfortran-4.0.0-7_5_0_h1a10cd1_23 libgfortran-devel~ conda-forge/noarch::libgfortran-devel_osx-64-7.5.0-h1a10cd1_23 libgfortran4 conda-forge/osx-64::libgfortran4-7.5.0-h1a10cd1_23 libglib conda-forge/osx-64::libglib-2.68.4-hf1fb8c0_1 libiconv conda-forge/osx-64::libiconv-1.16-haf1e3a3_0 liblapack conda-forge/osx-64::liblapack-3.8.0-17_openblas liblapacke conda-forge/osx-64::liblapacke-3.8.0-17_openblas libllvm10 conda-forge/osx-64::libllvm10-10.0.1-h009f743_3 libllvm11 conda-forge/osx-64::libllvm11-11.1.0-hd011deb_2 libnghttp2 conda-forge/osx-64::libnghttp2-1.43.0-h07e645a_0 libopenblas conda-forge/osx-64::libopenblas-0.3.10-openmp_h63d9170_5 libpng conda-forge/osx-64::libpng-1.6.37-h7cec526_2 librsvg conda-forge/osx-64::librsvg-2.50.7-hd2a7919_0 libsodium conda-forge/osx-64::libsodium-1.0.18-hbcb3906_1 libssh2 conda-forge/osx-64::libssh2-1.10.0-h52ee1ee_0 libtiff conda-forge/osx-64::libtiff-4.2.0-h355d032_0 libtool conda-forge/osx-64::libtool-2.4.6-he49afe7_1008 libwebp conda-forge/osx-64::libwebp-1.2.0-h1648767_0 libwebp-base conda-forge/osx-64::libwebp-base-1.2.0-h0d85af4_2 libxcb conda-forge/osx-64::libxcb-1.13-h35c211d_1003 libxml2 conda-forge/osx-64::libxml2-2.9.12-h93ec3fd_0 llvm-openmp conda-forge/osx-64::llvm-openmp-12.0.1-hda6cdc1_1 llvm-tools conda-forge/osx-64::llvm-tools-11.1.0-hd011deb_2 llvmlite conda-forge/osx-64::llvmlite-0.36.0-py37h62ea057_0 locket conda-forge/noarch::locket-0.2.0-py_2 lz4-c conda-forge/osx-64::lz4-c-1.9.2-hb1e8313_3 markupsafe conda-forge/osx-64::markupsafe-2.0.1-py37h271585c_0 matplotlib conda-forge/osx-64::matplotlib-3.4.3-py37hf985489_0 matplotlib-base conda-forge/osx-64::matplotlib-base-3.4.3-py37h3147e9e_0 matplotlib-inline conda-forge/noarch::matplotlib-inline-0.1.3-pyhd8ed1ab_0 metakernel conda-forge/noarch::metakernel-0.27.5-pyhd3deb0d_0 mistune conda-forge/osx-64::mistune-0.8.4-py37h271585c_1004 more-itertools conda-forge/noarch::more-itertools-8.9.0-pyhd8ed1ab_0 mpc conda-forge/osx-64::mpc-1.2.1-hbb51d92_0 mpfr conda-forge/osx-64::mpfr-4.1.0-h0f52abe_1 nbclient conda-forge/noarch::nbclient-0.5.4-pyhd8ed1ab_0 nbconvert conda-forge/osx-64::nbconvert-6.1.0-py37hf985489_1 nbformat conda-forge/noarch::nbformat-5.1.3-pyhd8ed1ab_0 ncurses conda-forge/osx-64::ncurses-6.2-h2e338ed_4 nest-asyncio conda-forge/noarch::nest-asyncio-1.5.1-pyhd8ed1ab_0 networkx conda-forge/noarch::networkx-2.5-py_0 notebook conda-forge/noarch::notebook-6.4.4-pyha770c72_0 numba conda-forge/osx-64::numba-0.53.1-py37h0183300_1 numpy conda-forge/osx-64::numpy-1.21.2-py37haefe36b_0 olefile conda-forge/noarch::olefile-0.46-pyh9f0ad1d_1 openblas conda-forge/osx-64::openblas-0.3.10-openmp_h3f95352_5 openjpeg conda-forge/osx-64::openjpeg-2.4.0-h6e7aa92_1 openssl conda-forge/osx-64::openssl-1.1.1l-h0d85af4_0 packaging conda-forge/noarch::packaging-21.0-pyhd8ed1ab_0 pandoc conda-forge/osx-64::pandoc-2.14.2-h0d85af4_0 pandocfilters conda-forge/noarch::pandocfilters-1.5.0-pyhd8ed1ab_0 pango conda-forge/osx-64::pango-1.48.10-ha05cd14_0 parso conda-forge/noarch::parso-0.8.2-pyhd8ed1ab_0 partd conda-forge/noarch::partd-1.2.0-pyhd8ed1ab_0 pcre conda-forge/osx-64::pcre-8.45-he49afe7_0 pexpect conda-forge/noarch::pexpect-4.8.0-pyh9f0ad1d_2 pgplot conda-forge/osx-64::pgplot-5.2.2-ha010686_1007 pickleshare conda-forge/noarch::pickleshare-0.7.5-py_1003 pillow conda-forge/osx-64::pillow-8.2.0-py37hd4e48bc_1 pip conda-forge/noarch::pip-21.2.4-pyhd8ed1ab_0 pixman conda-forge/osx-64::pixman-0.40.0-hbcb3906_0 pluggy conda-forge/osx-64::pluggy-1.0.0-py37hf985489_1 pmw conda-forge/osx-64::pmw-2.0.1-py37hf985489_1004 portalocker conda-forge/osx-64::portalocker-2.3.2-py37hf985489_0 prometheus_client conda-forge/noarch::prometheus_client-0.11.0-pyhd8ed1ab_0 prompt-toolkit conda-forge/noarch::prompt-toolkit-3.0.20-pyha770c72_0 psutil conda-forge/osx-64::psutil-5.8.0-py37h271585c_1 pthread-stubs conda-forge/osx-64::pthread-stubs-0.4-hc929b4f_1001 ptyprocess conda-forge/noarch::ptyprocess-0.7.0-pyhd3deb0d_0 py conda-forge/noarch::py-1.10.0-pyhd3deb0d_0 pyavm conda-forge/noarch::pyavm-0.9.5-pyhac0dd68_0 pycparser conda-forge/noarch::pycparser-2.20-pyh9f0ad1d_2 pygments conda-forge/noarch::pygments-2.10.0-pyhd8ed1ab_0 pyparsing conda-forge/noarch::pyparsing-2.4.7-pyh9f0ad1d_0 pyregion conda-forge/osx-64::pyregion-2.0-py37h37391d0_1001 pyrsistent conda-forge/osx-64::pyrsistent-0.17.3-py37h271585c_2 pytest conda-forge/osx-64::pytest-6.2.5-py37hf985489_0 pytest-arraydiff conda-forge/noarch::pytest-arraydiff-0.3-py_0 pytest-astropy conda-forge/noarch::pytest-astropy-0.8.0-pyhd8ed1ab_0 pytest-astropy-he~ conda-forge/noarch::pytest-astropy-header-0.1.2-py_0 pytest-cov conda-forge/noarch::pytest-cov-2.12.1-pyhd8ed1ab_0 pytest-doctestplus conda-forge/noarch::pytest-doctestplus-0.10.1-pyhd8ed1ab_0 pytest-filter-sub~ conda-forge/noarch::pytest-filter-subpackage-0.1.1-py_0 pytest-mock conda-forge/noarch::pytest-mock-3.6.1-pyhd8ed1ab_0 pytest-openfiles conda-forge/noarch::pytest-openfiles-0.5.0-py_0 pytest-remotedata conda-forge/noarch::pytest-remotedata-0.3.2-pyh9f0ad1d_0 pythia8 conda-forge/osx-64::pythia8-8.244-py37h54c7649_2 python conda-forge/osx-64::python-3.7.10-haf480d7_101_cpython python-dateutil conda-forge/noarch::python-dateutil-2.8.2-pyhd8ed1ab_0 python_abi conda-forge/osx-64::python_abi-3.7-2_cp37m pywavelets conda-forge/osx-64::pywavelets-1.1.1-py37h032687b_3 pyyaml conda-forge/osx-64::pyyaml-5.4.1-py37h271585c_1 pyzmq conda-forge/osx-64::pyzmq-22.3.0-py37h8f778e5_0 readline conda-forge/osx-64::readline-8.1-h05e3726_0 reproject conda-forge/osx-64::reproject-0.8-py37h032687b_0 root conda-forge/osx-64::root-6.22.6-py37ha0830e8_0 root-binaries conda-forge/osx-64::root-binaries-6.22.6-py37ha0830e8_0 root-dependencies conda-forge/osx-64::root-dependencies-6.22.6-py37ha0830e8_0 root_base conda-forge/osx-64::root_base-6.22.6-py37h1ff80a4_0 scikit-image conda-forge/osx-64::scikit-image-0.16.2-py37h4f17bb1_0 scipy conda-forge/osx-64::scipy-1.5.3-py37h04d6967_0 scitokens-cpp conda-forge/osx-64::scitokens-cpp-0.6.2-h27338ba_1 send2trash conda-forge/noarch::send2trash-1.8.0-pyhd8ed1ab_0 setuptools conda-forge/osx-64::setuptools-58.0.4-py37hf985489_0 shapely conda-forge/osx-64::shapely-1.7.1-py37hf313787_5 six conda-forge/noarch::six-1.16.0-pyh6c4a22f_0 sortedcontainers conda-forge/noarch::sortedcontainers-2.4.0-pyhd8ed1ab_0 sqlite conda-forge/osx-64::sqlite-3.36.0-h23a322b_1 tapi conda-forge/osx-64::tapi-1100.0.11-h9ce4665_0 tbb conda-forge/osx-64::tbb-2020.2-h940c156_4 tbb-devel conda-forge/osx-64::tbb-devel-2020.2-h940c156_4 terminado conda-forge/osx-64::terminado-0.12.1-py37hf985489_0 testpath conda-forge/noarch::testpath-0.5.0-pyhd8ed1ab_0 tk conda-forge/osx-64::tk-8.6.11-h5dbffcc_1 toml conda-forge/noarch::toml-0.10.2-pyhd8ed1ab_0 toolz conda-forge/noarch::toolz-0.11.1-py_0 tornado conda-forge/osx-64::tornado-6.1-py37h271585c_1 tqdm conda-forge/noarch::tqdm-4.62.2-pyhd8ed1ab_0 traitlets conda-forge/noarch::traitlets-5.1.0-pyhd8ed1ab_0 typing_extensions conda-forge/noarch::typing_extensions-3.10.0.0-pyha770c72_0 vdt conda-forge/osx-64::vdt-0.4.3-h046ec9c_0 wcslib conda-forge/osx-64::wcslib-5.20-hed077c5_1002 wcwidth conda-forge/noarch::wcwidth-0.2.5-pyh9f0ad1d_2 webencodings conda-forge/noarch::webencodings-0.5.1-py_1 wheel conda-forge/noarch::wheel-0.37.0-pyhd8ed1ab_1 xerces-c conda-forge/osx-64::xerces-c-3.2.3-h379762d_2 xorg-kbproto conda-forge/osx-64::xorg-kbproto-1.0.7-h35c211d_1002 xorg-libx11 conda-forge/osx-64::xorg-libx11-1.7.2-h0d85af4_0 xorg-libxau conda-forge/osx-64::xorg-libxau-1.0.9-h35c211d_0 xorg-libxdmcp conda-forge/osx-64::xorg-libxdmcp-1.1.3-h35c211d_0 xorg-xproto conda-forge/osx-64::xorg-xproto-7.0.31-h35c211d_1007 xrootd conda-forge/osx-64::xrootd-5.3.0-py37h5394c04_0 xxhash conda-forge/osx-64::xxhash-0.8.0-h35c211d_3 xz conda-forge/osx-64::xz-5.2.5-haf1e3a3_1 yaml conda-forge/osx-64::yaml-0.2.5-haf1e3a3_0 zeromq conda-forge/osx-64::zeromq-4.3.4-he49afe7_1 zipp conda-forge/noarch::zipp-3.5.0-pyhd8ed1ab_0 zlib conda-forge/osx-64::zlib-1.2.11-h7795811_1010 zstd conda-forge/osx-64::zstd-1.4.8-h62961b2_0

Proceed ([y]/n)?

=================

and then, after proceeding and downloading packs, it get stuck at "Executing transaction: \" . (for >1h )

BrunoArsioli commented 3 years ago

Hi @Areustle, thank you for writing. I tried as you said (using conda create -n fermi --file xxxx/spec-file.txt) but the issue remains the same: Stuck at "Executing transaction: \" . (for >1h )

When I hit ctrl+c, I get the same error msg as before:

Executing transaction: - \ | - - done ERROR conda.core.link:_execute(699): An error occurred while installing package 'conda-forge::gdk-pixbuf-2.42.6-h2e6141f_0'. Rolling back transaction: done

BrunoArsioli commented 3 years ago

Hi @Areustle. Trying something different:

now, created fermi env folder by hand:

mkdir /opt/anaconda3/envs/fermi

then, preinstall gdk-pixbuf-2.42.6-h2e6141f_0 pack (from conda-forge channel) at envs/fermi.

conda install gdk-pixbuf=2.42.6=h2e6141f_0 -c conda-forge -n fermi

remove gdk-pixbuf-2.42.6-h2e6141f_0 from spec-file.txt

then, use install instead of create (since env/fermi is already there).

conda install -n fermi --file [path]/spec-file.txt

the process get stuck, and after ctrl+c, got the following msg:

Executing transaction: / \ / \ / done ERROR conda.core.link:_execute(699): An error occurred while installing package 'conda-forge::librsvg-2.50.7-hd2a7919_0'. Rolling back transaction: | \ done

now, remove the env/fermi, start again from mkdir /opt/anaconda3/envs/fermi

and this time, also pre-install two extra pack:

conda install gdk-pixbuf=2.42.6=h2e6141f_0 -c conda-forge -n fermi

conda install librsvg=2.50.7=hd2a7919_0 -c conda-forge -n fermi

remove gdk-pixbuf-2.42.6-h2e6141f_0 and librsvg-2.50.7-hd2a7919_0 from spec-file.txt

and

conda install -n fermi --file [path]/spec-file.txt

the process get stuck, and after ctrl+c, got the following msg:

Executing transaction: / - | done ERROR conda.core.link:_execute(699): An error occurred while installing package 'conda-forge::graphviz-2.49.0-h77de9ca_0'. Rolling back transaction: -

now, remove the envs/fermi, and try again

mkdir /opt/anaconda3/envs/fermi

and this time, pre-install three extra packs:

conda install gdk-pixbuf=2.42.6=h2e6141f_0 -c conda-forge -n fermi

conda install librsvg=2.50.7=hd2a7919_0 -c conda-forge -n fermi

conda install graphviz=2.49.0=h77de9ca_0 -c conda-forge -n fermi

remove gdk-pixbuf-2.42.6-h2e6141f_0 and librsvg-2.50.7-hd2a7919_0 and graphviz=2.49.0=h77de9ca_0 from spec-file.txt

and

conda install -n fermi --file [path]/spec-file.txt

now, the process get stuck, and after ctrl+c, there is NO error msg.

....

Then, tried to install fermi-software with:

conda install fermitools-dependencies=2.0.8-0 -c fermi -n fermi

and

conda install fermitools=2.0.8=hb189bb9_0 -c fermi -n fermi

in both cases there was similar msgs:

Collecting package metadata (current_repodata.json): done Solving environment: failed with initial frozen solve. Retrying with flexible solve. Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source. Collecting package metadata (repodata.json): done Solving environment: failed with initial frozen solve. Retrying with flexible solve. Solving environment: - Found conflicts! Looking for incompatible packages. This can take several minutes. Press CTRL-C to abort. failed

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versions

Package expat conflicts for: expat graphviz==2.49.0=h77de9ca_0 -> expat[version='>=2.4.1,<3.0a0'] libgd -> expat[version='>=2.2.5,<3.0a0|>=2.2.6,<3.0a0|>=2.3.0,<3.0.0a0'] graphviz==2.49.0=h77de9ca_0 -> libgd[version='>=2.3.2,<2.4.0a0'] -> expat[version='>=2.3.0,<3.0.0a0']

Package gettext conflicts for: atk-1.0 -> libglib[version='>=2.64.6,<3.0a0'] -> gettext[version='>=0.19.8.1,<1.0a0'] pango -> libglib[version='>=2.68.4,<3.0a0'] -> gettext[version='>=0.19.8.1,<1.0a0|>=0.21.0,<1.0a0'] gts -> glib[version='>=2.69.1,<3.0a0'] -> gettext[version='>=0.19.8.1,<1.0a0|>=0.21.0,<1.0a0'] atk-1.0 -> gettext[version='>=0.21.0,<1.0a0'] gtk2 -> atk-1.0[version='>=2.36.0'] -> gettext[version='>=0.21.0,<1.0a0'] gettext harfbuzz -> libglib[version='>=2.68.4,<3.0a0'] -> gettext[version='>=0.19.8.1,<1.0a0|>=0.21.0,<1.0a0'] librsvg==2.50.7=hd2a7919_0 -> gdk-pixbuf[version='>=2.42.6,<3.0a0'] -> gettext[version='>=0.21.0,<1.0a0'] gdk-pixbuf==2.42.6=h2e6141f_0 -> libglib[version='>=2.68.0,<3.0a0'] -> gettext[version='>=0.19.8.1,<1.0a0'] cairo -> libglib[version='>=2.68.0,<3.0a0'] -> gettext[version='>=0.19.8.1,<1.0a0|>=0.21.0,<1.0a0'] librsvg==2.50.7=hd2a7919_0 -> gettext[version='>=0.19.8.1,<1.0a0'] libglib -> gettext[version='>=0.19.8.1,<1.0a0'] gtk2 -> gettext[version='>=0.19.8.1,<1.0a0'] graphviz==2.49.0=h77de9ca_0 -> gdk-pixbuf[version='>=2.42.6,<3.0a0'] -> gettext[version='>=0.19.8.1,<1.0a0|>=0.21.0,<1.0a0']

. . very long list with solving conflicts . .

and when I try conda activate fermi it shows no error message. But if I try to call a fermi routine like gtselect , I get 'zsh: command not found: gtselect'

Areustle commented 3 years ago

That is concerning. It suggests you may have a deeper problem with your miniconda installation. The --file method doesn't (or isn't supposed to) do any environment solving.

I wonder if you're running out of storage on your mac? Or running out just in one critical partition? That might explain why the transactions can't complete, but it's total speculation. Check that by running df -h. Pay special attention to /opt or /tmp if they're partitioned separately. You can clean out your conda package cache with conda clean --all.

Barring that you're going to need to do a deep dive into the debug output of the conda create command. Remove the old fermi environment and run the command conda create -n fermi --file .../spec-file.txt -vvv command, but append -vvv to the invocation. Each 'v' will greatly increase the output of the program, 'v' is verbose, 'vv' is debug, and 'vvv' is trace. Going straight for -vvv will give you the most information. I suspect that when your output will either stop at some recognizable point (possibly related to gdk-pixbuf) or loop forever.

Please do not post the full output of that command here, as it may contain private system information that you do not want made public. But do let us know if it gives you any extra insight into this issue.

BrunoArsioli commented 3 years ago

Hi @Areustle, thank you for writing. I will try to debug as you say. About the storage, there is still ~60Gb free in my disk. df -h shows: Filesystem Size Used Avail Capacity iused ifree %iused Mounted on /dev/disk1s5s1 466Gi 14Gi 32Gi 31% 553788 4882923132 0% / devfs 190Ki 190Ki 0Bi 100% 658 0 100% /dev /dev/disk1s4 466Gi 1.0Gi 32Gi 4% 2 4883476918 0% /System/Volumes/VM /dev/disk1s2 466Gi 357Mi 32Gi 2% 1532 4883475388 0% /System/Volumes/Preboot /dev/disk1s6 466Gi 2.8Mi 32Gi 1% 19 4883476901 0% /System/Volumes/Update /dev/disk1s1 466Gi 417Gi 32Gi 93% 1326714 4882150206 0% /System/Volumes/Data map auto_home 0Bi 0Bi 0Bi 100% 0 0 100% /System/Volumes/Data/home

BrunoArsioli commented 3 years ago

About the Miniconda installation, I've done the full Anaconda installation, and the issue remains the same.

Areustle commented 3 years ago

Do you have any more information from your debug output @BrunoArsioli ?

BrunoArsioli commented 3 years ago

Hi @Areustle. Yes. I just clean the conda cache sudo conda clean --all , and then removed conda completely with sudo rm -rf /op/anaconda3following uninstall-anaconda; and removed conda initializer lines from my home ~./bash_profile. Rebot. Installed Anaconda full version for mac, from here. Installed anaconda3 in my home folder (full version Anaconda3-2021.05-MacOSX-x86_64.pkg) with no issues. Rebot. At this point, my storage shows 70 Gb of free space. Then, in terminal

conda create -n fermi -c conda-forge -c fermi fermitools python=3 clhep=2.4.4.1 -vvv , after some 20 min, it stops here:

. . TRACE conda.gateways.disk.permissions:make_executable(79): chmod +x /Users/xxxxx/opt/anaconda3/envs/fermi/bin/jupyter-bundlerextension DEBUG conda.core.link:run_script(1203): for conda-forge::gdk-pixbuf-2.42.6-h2e6141f_0 at /Users/xxxxx/opt/anaconda3/envs/fermi, executing script: $ /bin/bash /Users/xxxxx/opt/anaconda3/envs/fermi/.tmp189snezu DEBUG conda.gateways.subprocess:subprocess_call(71): executing>> /bin/bash /Users/xxxxx/opt/anaconda3/envs/fermi/.tmp189snezu

The same issue as I reported before, stuck at conda-forge::gdk-pixbuf-2.42.6-h2e6141f_0 .... then, ctrl+c, stops at: . . TRACE conda.core.path_actions:reverse(381): reversing link creation /Users/xxxxx/opt/anaconda3/envs/fermi DEBUG conda.core.link:run_script(1203): for conda-forge::librsvg-2.50.7-hd2a7919_0 at /Users/xxxxx/opt/anaconda3/envs/fermi, executing script: $ /bin/bash /Users/xxxxx/opt/anaconda3/envs/fermi/.tmpiqfd9sbg DEBUG conda.gateways.subprocess:subprocess_call(71): executing>> /bin/bash /Users/xxxxx/opt/anaconda3/envs/fermi/.tmpiqfd9sbg

The same issue as I reported before with conda-forge::librsvg-2.50.7-hd2a7919_0 , then, ctrl+c, stops at: . . TRACE conda.core.path_actions:reverse(381): reversing link creation /Users/xxxxx/opt/anaconda3/envs/fermi DEBUG conda.core.link:run_script(1203): for conda-forge::graphviz-2.49.0-h77de9ca_0 at /Users/xxxxx/opt/anaconda3/envs/fermi, executing script: $ /bin/bash /Users/xxxxx/opt/anaconda3/envs/fermi/.tmpz4vp9dlw DEBUG conda.gateways.subprocess:subprocess_call(71): executing>> /bin/bash /Users/xxxxx/opt/anaconda3/envs/fermi/.tmpz4vp9dlw

The same issue as I reported before with conda-forge::graphviz-2.49.0-h77de9ca_0 , then, ctrl+c, stops at: . . " TRACE conda.core.path_actions:execute(562): compiling ....[very long list of file paths] TRACE conda.gateways.disk.create:compile_multiplepyc(399): ['/Users/xxxxx/opt/anaconda3/envs/fermi/bin/python3.7', '-Wi', '-m', 'compileall', '-q', '-l', '-i', '/var/folders/2/1jjw_1g13w7gxhcwr71wk1jh0000gn/T/tmpymc4lv3w', '-j', '0']

then, if I hit ctrl+c again, it is over, with: . . conda.CondaMultiError: Signal interrupt SIGINT ()

BrunoArsioli commented 3 years ago

Then, another trial: I clean the cache sudo conda clean --all, and remove fermi env rm -rf /Users/xxxxx/opt/anaconda3/envs/fermi. Rebot. One of the messages I saw through the -vvv comments was a suggestion to update conda using conda update -n base -c defaults conda , so, I just did that now. Implementing the following: conda 4.10.1-py38hecd8cb5_1 --> 4.10.3-py38hecd8cb5_0 and xmltodict 0.12.0-py_0 --> 0.12.0-pyhd3eb1b0_0 . Then conda create -n fermi --file [path]/spec-file.txt -vvv

Again, the process stops at conda-forge::gdk-pixbuf-2.42.6-h2e6141f_0 ; same msg, then stops at: conda-forge::librsvg-2.50.7-hd2a7919_0 , and again at conda-forge::graphviz-2.49.0-h77de9ca_0 . Then, it stops again at " TRACE conda.gateways.disk.create:compile_multiplepyc(399): ['/Users/xxxxx/opt/anaconda3/envs/fermi/bin/python3.7', '-Wi', '-m', 'compileall', '-q', '-l', '-i', '/var/folders/2/1jjw_1g13w7gxhcwr71wk1jh0000gn/T/tmpiwml47bn', '-j', '0'] " , and after ctrl+c , it is over. just as in previous trials.

BrunoArsioli commented 2 years ago

Found a fix. Now with Mac Bigsur 11.6.2

First, I downgraded Xcode to 13.0 , following #116

Then, reading: link1 , link2 , link3 (check the comment from "glass-ships") , link4

I tried:

conda update conda -c conda-canary

conda config --set channel_priority false

( if there is already a xxx/env/fermi , remove with rm -r xxx/env/fermi )

conda create --name fermi

conda activate fermi

conda install -c fermi fermitools python=3 clhep=2.4.4.1

then: conda activate fermi and the fermitools seems to be working now.

maybe this is enough to solve #116 as well.

kerrm commented 2 years ago

Wow, nice detective work!

On Sun, 23 Jan 2022 at 00:45, Bruno Arsioli @.***> wrote:

Found a fix. Now with Mac Bigsur 11.6.2

First, I downgraded Xcode to 13.0 , following #116 https://github.com/fermi-lat/Fermitools-conda/issues/116

Then, reading: link1 https://exerror.com/solving-environment-failed-with-initial-frozen-solve-retrying-with-flexible-solve/

link2 https://programmerah.com/solve-condas-solving-environment-failed-problem-4018/

link3-check comment from "glass-ships" https://github.com/conda/conda/issues/9004

link4 https://dizzycoding.com/solving-environment-failed-with-repodata-from-current_repodata-json-will-retry-with-next-repodata-source-code-answer/

I tried:

conda update conda -c conda-canary

conda config --set channel_priority false

( if there is already a xxx/env/fermi , remove with rm -r xxx/env/fermi )

conda create --name fermi

conda activate fermi

conda install -c fermi fermitools python=3 clhep=2.4.4.1

then: conda activate fermi and the fermitools seems to be working now.

— Reply to this email directly, view it on GitHub https://github.com/fermi-lat/Fermitools-conda/issues/111#issuecomment-1019418269, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF3LAG6XA6PUCKFSYMSTV2LUXOIYVANCNFSM5DYLZWGA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you are subscribed to this thread.Message ID: @.***>

donhorner commented 2 years ago

We have a development version of the Fermitools that we hope to release soon that should fix most installation problems because we removed the ROOT package, which was the source of many of these issues. If you like to test it, you can install the dev version with the command: conda create -n fermitools-2.1.30 -c conda-forge -c fermi/label/dev fermitools=2.1.30

donhorner commented 2 years ago

Fermitools 2.2.0 has been released which fixes this issue.