search

conda-forge / miniforge

A conda-forge distribution.
https://conda-forge.org/miniforge
Other
5.76k stars 306 forks source link

Crashes using `mamba` on macOS Sonoma #512

Open pauldmccarthy opened 8 months ago

pauldmccarthy commented 8 months ago

Solution to issue cannot be found in the documentation.

Issue

Howdy,

(Reporting here for posterity - I think the main issue is https://github.com/curl/curl/issues/11893).

I have a couple of users reporting crashes when running mamba env update on macOS Sonoma. I don't have access to a Sonoma machine, so can only provide a few more details (if necessary), but I think the most relevant message that is emitted is:

*** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[__NSCFString stringByStandardizingPath]: unrecognized selector sent to instance 0x600003af0050'

Searching around the internet, I stumbled across a recently fixed issue in curl.

So I suspect that this issue will be resolved with a new miniforge/mambaforge release that bundles the latest curl 8.4.0.

Judging by #511, it sounds like this is already on the cards, so hopefully this be resolved without any further effort.

Thanks!

Installed packages

Environment installed via [`Mambaforge-23.3.1-1-MacOSX-arm64.sh`](https://github.com/conda-forge/miniforge/releases/download/23.3.1-1/Mambaforge-23.3.1-1-MacOSX-arm64.sh)

Environment info

Environment installed via [`Mambaforge-23.3.1-1-MacOSX-arm64.sh`](https://github.com/conda-forge/miniforge/releases/download/23.3.1-1/Mambaforge-23.3.1-1-MacOSX-arm64.sh)
victorlin commented 7 months ago

Just came across this. It seems to affect any attempt to install a package using mamba. Here's relevant snippets of output from a simple command mamba create -n tmp python=3.10 --yes:

Output ``` Package Version Build Channel Size ──────────────────────────────────────────────────────────────────────── Install: ──────────────────────────────────────────────────────────────────────── + bzip2 1.0.8 h1de35cc_0 pkgs/main Cached + ncurses 6.4 hcec6c5f_0 pkgs/main Cached + zlib 1.2.13 h4dc903c_0 pkgs/main Cached + libffi 3.4.4 hecd8cb5_0 pkgs/main Cached + xz 5.4.5 h6c40b1e_0 pkgs/main 374kB + ca-certificates 2023.08.22 hecd8cb5_0 pkgs/main Cached + readline 8.2 hca72f7f_0 pkgs/main Cached + tk 8.6.12 h5d9f67b_0 pkgs/main Cached + openssl 3.0.12 hca72f7f_0 pkgs/main Cached + sqlite 3.41.2 h6c40b1e_0 pkgs/main Cached + tzdata 2023c h04d1e81_0 pkgs/main Cached + python 3.10.13 h5ee71fb_0 pkgs/main 14MB + wheel 0.41.2 py310hecd8cb5_0 pkgs/main 113kB + setuptools 68.0.0 py310hecd8cb5_0 pkgs/main 978kB + pip 23.3.1 py310hecd8cb5_0 pkgs/main 3MB Summary: Install: 15 packages Total download: 18MB ──────────────────────────────────────────────────────────────────────── [+] 0.1s Downloading (5) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 59.2kB / 18.1MB pip 0.0s Extracting ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0 / 5 0.0s*** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[__NSCFString stringByStandardizingPath]: unrecognized selector sent to instance 0x600000dac680' *** First throw call stack: ( 0 CoreFoundation 0x00007ff807dadaae __exceptionPreprocess + 242 1 libobjc.A.dylib 0x00007ff8078a45ff objc_exception_throw + 48 2 CoreFoundation 0x00007ff807e529dd -[NSObject(NSObject) __retain_OA] + 0 3 CoreFoundation 0x00007ff807d1bff8 ___forwarding___ + 1379 4 CoreFoundation 0x00007ff807d1ba08 _CF_forwarding_prep_0 + 120 5 Foundation 0x00007ff808be0b8f -[NSProcessInfo arguments] + 203 6 CoreFoundation 0x00007ff807e20572 __getDefaultArguments_block_invoke + 101 7 libdispatch.dylib 0x00007ff807ab959a _dispatch_client_callout + 8 8 libdispatch.dylib 0x00007ff807aba7cb _dispatch_once_callout + 20 9 CoreFoundation 0x00007ff807e1ff83 _addBackstopValuesForIdentifierAndSource + 626 10 CoreFoundation 0x00007ff807ce7fc4 __81-[_CFXPreferences(SourceAdditions) withNamedVolatileSourceForIdentifier:perform:]_block_invoke + 151 11 CoreFoundation 0x00007ff807e1fc4c -[_CFXPreferences withNamedVolatileSourceForIdentifier:perform:] + 241 12 CoreFoundation 0x00007ff807ceec00 -[CFPrefsSearchListSource addNamedVolatileSourceForIdentifier:] + 98 13 CoreFoundation 0x00007ff807e5a479 __108-[_CFXPreferences(SearchListAdditions) withSearchListForIdentifier:container:cloudConfigurationURL:perform:]_block_invoke.160 + 310 14 CoreFoundation 0x00007ff807e5a146 -[_CFXPreferences withSearchLists:] + 60 15 CoreFoundation 0x00007ff807ce95e6 __108-[_CFXPreferences(SearchListAdditions) withSearchListForIdentifier:container:cloudConfigurationURL:perform:]_block_invoke + 278 16 CoreFoundation 0x00007ff807e5a2be -[_CFXPreferences withSearchListForIdentifier:container:cloudConfigurationURL:perform:] + 349 17 CoreFoundation 0x00007ff807ce906b -[_CFXPreferences copyAppValueForKey:identifier:container:configurationURL:] + 124 18 CoreFoundation 0x00007ff807ce8fb5 _CFPreferencesCopyAppValueWithContainerAndConfiguration + 101 19 SystemConfiguration 0x00007ff8088978bb SCDynamicStoreCopyProxiesWithOptions + 155 20 libcurl.4.dylib 0x000000010e4aa34b Curl_macos_init + 11 21 libcurl.4.dylib 0x000000010e487d31 global_init + 145 22 libcurl.4.dylib 0x000000010e487c88 curl_global_init + 56 23 libmamba.2.0.0.dylib 0x000000010d872b31 _GLOBAL__sub_I_singletons.cpp + 17 24 dyld 0x0000000204a7ffea ___ZZNK5dyld46Loader25findAndRunAllInitializersERNS_12RuntimeStateEENK3$_0clEv_block_invoke + 182 25 dyld 0x0000000204ac15d4 ___ZNK5dyld313MachOAnalyzer18forEachInitializerER11DiagnosticsRKNS0_15VMAddrConverterEU13block_pointerFvjEPKv_block_invoke.175 + 133 26 dyld 0x0000000204ab5963 ___ZNK5dyld39MachOFile14forEachSectionEU13block_pointerFvRKNS0_11SectionInfoEbRbE_block_invoke + 543 27 dyld 0x0000000204a6307f _ZNK5dyld39MachOFile18forEachLoadCommandER11DiagnosticsU13block_pointerFvPK12load_commandRbE + 249 28 dyld 0x0000000204ab4b2c _ZNK5dyld39MachOFile14forEachSectionEU13block_pointerFvRKNS0_11SectionInfoEbRbE + 176 29 dyld 0x0000000204ab7154 _ZNK5dyld39MachOFile32forEachInitializerPointerSectionER11DiagnosticsU13block_pointerFvjjRbE + 116 30 dyld 0x0000000204ac130a _ZNK5dyld313MachOAnalyzer18forEachInitializerER11DiagnosticsRKNS0_15VMAddrConverterEU13block_pointerFvjEPKv + 390 31 dyld 0x0000000204a7cd1c _ZNK5dyld46Loader25findAndRunAllInitializersERNS_12RuntimeStateE + 222 32 dyld 0x0000000204a825eb _ZNK5dyld416JustInTimeLoader15runInitializersERNS_12RuntimeStateE + 21 33 dyld 0x0000000204a7cf11 _ZNK5dyld46Loader23runInitializersBottomUpERNS_12RuntimeStateERN5dyld35ArrayIPKS0_EE + 181 34 dyld 0x0000000204a7cef3 _ZNK5dyld46Loader23runInitializersBottomUpERNS_12RuntimeStateERN5dyld35ArrayIPKS0_EE + 151 35 dyld 0x0000000204a80060 _ZZNK5dyld46Loader38runInitializersBottomUpPlusUpwardLinksERNS_12RuntimeStateEENK3$_1clEv + 98 36 dyld 0x0000000204a7cfa7 _ZNK5dyld46Loader38runInitializersBottomUpPlusUpwardLinksERNS_12RuntimeStateE + 93 37 dyld 0x0000000204a9e9ad _ZN5dyld44APIs25runAllInitializersForMainEv + 317 38 dyld 0x0000000204a68261 _ZN5dyld4L7prepareERNS_4APIsEPKN5dyld313MachOAnalyzerE + 3397 39 dyld 0x0000000204a6733f start + 1839 ) libc++abi: terminating due to uncaught exception of type NSException xz 374.4kB @ 2.5MB/s 0.2s wheel 112.7kB @ 587.6kB/s 0.2s xz-5.4.5-h6c40b1e_0.conda extraction failed [+] 0.2s # the above error is repeated for a few packages # - victorlin libc++abi: terminating due to uncaught exception of type NSException python-3.10.13-h5ee71fb_0.conda extraction failed Found incorrect download: xz. Aborting # >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<< Traceback (most recent call last): File "/Users/victor/miniconda3/lib/python3.11/site-packages/conda/exception_handler.py", line 17, in __call__ return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/victor/miniconda3/lib/python3.11/site-packages/mamba/mamba.py", line 956, in exception_converter raise e File "/Users/victor/miniconda3/lib/python3.11/site-packages/mamba/mamba.py", line 949, in exception_converter exit_code = _wrapped_main(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/victor/miniconda3/lib/python3.11/site-packages/mamba/mamba.py", line 895, in _wrapped_main result = do_call(parsed_args, p) ^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/victor/miniconda3/lib/python3.11/site-packages/mamba/mamba.py", line 764, in do_call exit_code = create(args, parser) ^^^^^^^^^^^^^^^^^^^^ File "/Users/victor/miniconda3/lib/python3.11/site-packages/mamba/mamba.py", line 602, in create return install(args, parser, "create") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/victor/miniconda3/lib/python3.11/site-packages/mamba/mamba.py", line 558, in install transaction.fetch_extract_packages() RuntimeError: Found incorrect download: xz. Aborting `$ /Users/victor/miniconda3/bin/mamba create -n tmp python=3.10 --yes` environment variables: CIO_TEST= CONDA_DEFAULT_ENV=base CONDA_EXE=/Users/victor/miniconda3/bin/conda CONDA_PREFIX=/Users/victor/miniconda3 CONDA_PROMPT_MODIFIER=(base) CONDA_PYTHON_EXE=/Users/victor/miniconda3/bin/python CONDA_ROOT=/Users/victor/miniconda3 CONDA_SHLVL=1 CURL_CA_BUNDLE= GEM_PATH=/Users/victor/.gem/ruby/3.1.3:/Users/victor/.rubies/ruby- 3.1.3/lib/ruby/gems/3.1.0 INFOPATH=/opt/homebrew/share/info: LD_PRELOAD= MANPATH=/opt/homebrew/share/man:: PATH=/Users/victor/miniconda3/bin:/Users/victor/.micromamba/condabin:/Users /victor/.gem/ruby/3.1.3/bin:/Users/victor/.rubies/ruby- 3.1.3/lib/ruby/gems/3.1.0/bin:/Users/victor/.rubies/ruby- 3.1.3/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/Users/victor/.bin:/usr /local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin :/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local /bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/b in:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/app leinternal/bin:/Library/Apple/usr/bin REQUESTS_CA_BUNDLE= SSL_CERT_FILE= active environment : base active env location : /Users/victor/miniconda3 shell level : 1 user config file : /Users/victor/.condarc populated config files : conda version : 23.11.0 conda-build version : not installed python version : 3.11.5.final.0 solver : libmamba (default) virtual packages : __archspec=1=m1 __conda=23.11.0=0 __osx=10.16=0 __unix=0=0 base environment : /Users/victor/miniconda3 (writable) conda av data dir : /Users/victor/miniconda3/etc/conda conda av metadata url : None channel URLs : https://repo.anaconda.com/pkgs/main/osx-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/osx-64 https://repo.anaconda.com/pkgs/r/noarch package cache : /Users/victor/miniconda3/pkgs /Users/victor/.conda/pkgs envs directories : /Users/victor/miniconda3/envs /Users/victor/.conda/envs platform : osx-64 user-agent : conda/23.11.0 requests/2.31.0 CPython/3.11.5 Darwin/23.1.0 OSX/10.16 solver/libmamba conda-libmamba-solver/23.11.1 libmambapy/1.5.4 UID:GID : 501:20 netrc file : None offline mode : False An unexpected error has occurred. Conda has prepared the above report. If you suspect this error is being caused by a malfunctioning plugin, consider using the --no-plugins option to turn off plugins. Example: conda --no-plugins install Alternatively, you can set the CONDA_NO_PLUGINS environment variable on the command line to run the command without plugins enabled. Example: CONDA_NO_PLUGINS=true conda install ```
sorenwacker commented 5 months ago

Same problem.

fxcoudert commented 4 months ago

This crash is typically because some library loaded (here, probably curl) should link to CoreServices but doesn't. This was fixed in Curl in this commit: https://github.com/curl/curl/commit/6ab7e1990bd548059f08c471d20537bca13c67b6

So this should be fixed by moving to curl >= 8.4.0

jaimergp commented 4 months ago

Does it work with conda too? With the new solver, it should be more or less the same result, but the download happens via a Python code path, so maybe it's not affected.

This should be the same (--solver libmamba not needed, just added for clarity):

$ conda create -n tmp python=3.10 --yes --solver=libmamba
pauldmccarthy commented 4 months ago

I'm afraid I don't have access to a Sonoma machine, so cannot be of much use. However I am still getting reports of this occurring with the latest Mambaforge (23.11.0). Confusingly I am also getting reports of successful installations on Sonoma, so there is another factor at play somewhere.

We have solved this issue in our system by switching to the latest Miniconda release (also versioned 23.11.0), and calling conda rather than mamba (which as @jaimergp points out is likely to be using libmamba under the hood).

For what it's worth, here is a log file of our installer script which uses mambaforge to set up a conda environment containing our software, in which the error has occurred. The installer does the following:

  1. Downloads and installs Mambaforge-23.11.0-0-MacOSX-arm64.sh
  2. Downloads our environment specification
  3. Runs $PREFIX/bin/mamba env update -p $PREFIX -f <env>.yml (We're aware that installing directly into the base environment is discouraged, but it has worked well for us in practice so far).

Note that our installer script is written in Python, and so a separate Python interpreter is required to run our script and to bootstrap the installation. One common theme amongst all of the reports I have received are that the Python interpreter used for running the installer script is an Anaconda installation (e.g. /Users/<username>/anaconda3/bin/python).

I had a wild theory that perhaps the Anaconda installation has an out-dated libcurl library, and that this library is somehow magically being loaded by the mambaforge Python interpreter, instead of the one installed into the mambaforge base environment. I asked a colleague of mine (with a M1 mac running Sonoma) to test this theory (install an old version of Anaconda with a broken libcurl, then use it to run our installer), but he wasn't able to reproduce the error.

So I'm kind of stumped. But as I mentioned, this is not a problem for us, as with the switch to miniconda our users are no longer experiencing the error.