search

mamba-org / mamba

The Fast Cross-Platform Package Manager
https://mamba.readthedocs.io
BSD 3-Clause "New" or "Revised" License
6.84k stars 353 forks source link

Error while building micromamba from source on MacOS (linking libmamba.a step) #3541

Closed anutosh491 closed 1 hour ago

anutosh491 commented 3 hours ago

Troubleshooting docs

Anaconda default channels

How did you install Mamba?

Micromamba

Search tried in issue tracker

Similar issue not found

Latest version of Mamba

Tried in Conda?

Not applicable

Describe your issue

I tried building micromamba from source on two relatively fresh MacOS devices and end up with the same error on both.

What I did 1) Cloned the repo 2) Installed micromamba using automatic install 3) setup the env

micromamba create -n mamba-dev -f dev/environment-dev.yml -y
micromamba install -n mamba-dev -f dev/environment-micromamba-static.yml -y

4) Executed the following 

(mamba-dev) anutosh491@Anutoshs-MacBook-Air mamba % cmake -B build/ \
    -G "Unix Makefiles" \
    ${CMAKE_ARGS} \
    -D CMAKE_INSTALL_PREFIX="${CONDA_PREFIX}" \
    -D CMAKE_BUILD_TYPE="Release" \
    -D BUILD_LIBMAMBA=ON \
    -D BUILD_STATIC=ON \
    -D BUILD_MICROMAMBA=ON
-- The C compiler identification is Clang 17.0.6
-- The CXX compiler identification is Clang 17.0.6
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Users/anutosh491/micromamba/envs/mamba-dev/bin/arm64-apple-darwin20.0.0-clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Users/anutosh491/micromamba/envs/mamba-dev/bin/arm64-apple-darwin20.0.0-clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Building libmamba v2.0.2
-- libmamba binary version: v2.0.0
-- Found Python3: /Users/anutosh491/micromamba/envs/mamba-dev/bin/python3.12 (found version "3.12.7") found components: Interpreter
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Found nlohmann_json: /Users/anutosh491/micromamba/envs/mamba-dev/share/cmake/nlohmann_json/nlohmann_jsonConfig.cmake (found version "3.11.3")
-- Found PkgConfig: /Users/anutosh491/micromamba/envs/mamba-dev/bin/pkg-config (found version "0.29.2")
-- Found Libsolv: /Users/anutosh491/micromamba/envs/mamba-dev/include (found version "0.7.30")
-- Setting LTO for target libmamba::solv-cpp
-- Adding static libmamba target
-- Setting LTO for target libmamba::libmamba-static
   -> Statically linking against libmamba (static) dependencies
Found library: /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/Security.framework
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/CoreFoundation.framework
   -> Found static dependencies:
      - /Users/anutosh491/micromamba/envs/mamba-dev/lib/libcurl.a
      - /Users/anutosh491/micromamba/envs/mamba-dev/lib/libssh2.a
      - /Users/anutosh491/micromamba/envs/mamba-dev/lib/libgssapi_krb5.a
      - /Users/anutosh491/micromamba/envs/mamba-dev/lib/libkrb5.a
      - /Users/anutosh491/micromamba/envs/mamba-dev/lib/libk5crypto.a
      - /Users/anutosh491/micromamba/envs/mamba-dev/lib/libkrb5support.a
      - /Users/anutosh491/micromamba/envs/mamba-dev/lib/libcom_err.a
      - /Users/anutosh491/micromamba/envs/mamba-dev/lib/libssl.a
      - /Users/anutosh491/micromamba/envs/mamba-dev/lib/libcrypto.a
      - /Users/anutosh491/micromamba/envs/mamba-dev/lib/libarchive.a
      - /Users/anutosh491/micromamba/envs/mamba-dev/lib/libiconv.a
      - /Users/anutosh491/micromamba/envs/mamba-dev/lib/libbz2.a
      - /Users/anutosh491/micromamba/envs/mamba-dev/lib/liblz4.a
      - /Users/anutosh491/micromamba/envs/mamba-dev/lib/libzstd.a
      - /Users/anutosh491/micromamba/envs/mamba-dev/lib/libz.a
      - /Users/anutosh491/micromamba/envs/mamba-dev/lib/liblzma.a
      - /Users/anutosh491/micromamba/envs/mamba-dev/lib/libnghttp2.a
      - /Users/anutosh491/micromamba/envs/mamba-dev/lib/libc++.a
      - /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/CoreFoundation.framework
      - /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/Security.framework
      - /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/SystemConfiguration.framework
-- Adding executable micromamba
-- Setting LTO for target micromamba::micromamba
-- Configuring done (3.3s)
-- Generating done (0.1s)
-- Build files have been written to: /Users/anutosh491/work/mamba/build
make -C build/ -j8 
.......
[ 86%] Linking CXX static library libmamba.a
Error running link command: name too longmake[2]: *** [libmamba/CMakeFiles/libmamba-static.dir/build.make:1983: libmamba/libmamba.a] Error 2
make[1]: *** [CMakeFiles/Makefile2:133: libmamba/CMakeFiles/libmamba-static.dir/all] Error 2
make: *** [Makefile:136: all] Error 2

This tells me that is has something to do with link.txt might be framed 

"/Users/anutosh491/micromamba/envs/mamba-dev/bin/arm64-apple-darwin20.0.0-ar -DCMAKE_CXX_COMPILER_AR=/Users/anutosh491/micromamba/envs/mamba-dev/bin/arm64-apple-darwin20.0.0-ar -DCMAKE_C_COMPILER_AR=/Users/anutosh491/micromamba/envs/mamba-dev/bin/arm64-apple-darwin20.0.0-ar -DCMAKE_RANLIB=/Users/anutosh491/micromamba/envs/mamba-dev/bin/arm64-apple-darwin20.0.0-ranlib -DCMAKE_CXX_COMPILER_RANLIB=/Users/anutosh491/micromamba/envs/mamba-dev/bin/arm64-apple-darwin20.0.0-ranlib -DCMAKE_C_COMPILER_RANLIB=/Users/anutosh491/micromamba/envs/mamba-dev/bin/arm64-apple-darwin20.0.0-ranlib -DCMAKE_LINKER=/Users/anutosh491/micromamba/envs/mamba-dev/bin/arm64-apple-darwin20.0.0-ld -DCMAKE_STRIP=/Users/anutosh491/micromamba/envs/mamba-dev/bin/arm64-apple-darwin20.0.0-strip -DCMAKE_INSTALL_NAME_TOOL=/Users/anutosh491/micromamba/envs/mamba-dev/bin/arm64-apple-darwin20.0.0-install_name_tool -DCMAKE_LIBTOOL=/Users/anutosh491/micromamba/envs/mamba-dev/bin/arm64-apple-darwin20.0.0-libtool -DCMAKE_BUILD_TYPE=Release -DCMAKE_OSX_SYSROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk" qc libmamba.a "CMakeFiles/libmamba-static.dir/src/version.cpp.o" "CMakeFiles/libmamba-static.dir/src/fs/filesystem.cpp.o" "CMakeFiles/libmamba-static.dir/src/util/cfile.cpp.o" "CMakeFiles/libmamba-static.dir/src/util/cryptography.cpp.o" "CMakeFiles/libmamba-static.dir/src/util/encoding.cpp.o" "CMakeFiles/libmamba-static.dir/src/util/environment.cpp.o" "CMakeFiles/libmamba-static.dir/src/util/os_linux.cpp.o" "CMakeFiles/libmamba-static.dir/src/util/os_osx.cpp.o" "CMakeFiles/libmamba-static.dir/src/util/os_unix.cpp.o" "CMakeFiles/libmamba-static.dir/src/util/os_win.cpp.o" "CMakeFiles/libmamba-static.dir/src/util/parsers.cpp.o" "CMakeFiles/libmamba-static.dir/src/util/path_manip.cpp.o" "CMakeFiles/libmamba-static.dir/src/util/random.cpp.o" "CMakeFiles/libmamba-static.dir/src/util/string.cpp.o" "CMakeFiles/libmamba-static.dir/src/util/url.cpp.o" "CMakeFiles/libmamba-static.dir/src/util/url_manip.cpp.o" "CMakeFiles/libmamba-static.dir/src/specs/archive.cpp.o" "CMakeFiles/libmamba-static.dir/src/specs/authentication_info.cpp.o" "CMakeFiles/libmamba-static.dir/src/specs/build_number_spec.cpp.o" "CMakeFiles/libmamba-static.dir/src/specs/channel.cpp.o" "CMakeFiles/libmamba-static.dir/src/specs/chimera_string_spec.cpp.o" "CMakeFiles/libmamba-static.dir/src/specs/conda_url.cpp.o" "CMakeFiles/libmamba-static.dir/src/specs/glob_spec.cpp.o" "CMakeFiles/libmamba-static.dir/src/specs/match_spec.cpp.o" "CMakeFiles/libmamba-static.dir/src/specs/package_info.cpp.o" "CMakeFiles/libmamba-static.dir/src/specs/platform.cpp.o" "CMakeFiles/libmamba-static.dir/src/specs/regex_spec.cpp.o" "CMakeFiles/libmamba-static.dir/src/specs/repo_data.cpp.o" "CMakeFiles/libmamba-static.dir/src/specs/unresolved_channel.cpp.o" "CMakeFiles/libmamba-static.dir/src/specs/version.cpp.o" "CMakeFiles/libmamba-static.dir/src/specs/version_spec.cpp.o" "CMakeFiles/libmamba-static.dir/src/solver/helpers.cpp.o" "CMakeFiles/libmamba-static.dir/src/solver/problems_graph.cpp.o" "CMakeFiles/libmamba-static.dir/src/solver/libsolv/database.cpp.o" "CMakeFiles/libmamba-static.dir/src/solver/libsolv/helpers.cpp.o" "CMakeFiles/libmamba-static.dir/src/solver/libsolv/matcher.cpp.o" "CMakeFiles/libmamba-static.dir/src/solver/libsolv/parameters.cpp.o" "CMakeFiles/libmamba-static.dir/src/solver/libsolv/repo_info.cpp.o" "CMakeFiles/libmamba-static.dir/src/solver/libsolv/solver.cpp.o" "CMakeFiles/libmamba-static.dir/src/solver/libsolv/unsolvable.cpp.o" "CMakeFiles/libmamba-static.dir/src/validation/tools.cpp.o" "CMakeFiles/libmamba-static.dir/src/validation/errors.cpp.o" "CMakeFiles/libmamba-static.dir/src/validation/keys.cpp.o" "CMakeFiles/libmamba-static.dir/src/validation/update_framework.cpp.o" "CMakeFiles/libmamba-static.dir/src/validation/update_framework_v0_6.cpp.o" "CMakeFiles/libmamba-static.dir/src/validation/update_framework_v1.cpp.o" "CMakeFiles/libmamba-static.dir/src/validation/repo_checker.cpp.o" "CMakeFiles/libmamba-static.dir/src/download/compression.cpp.o" "CMakeFiles/libmamba-static.dir/src/download/curl.cpp.o" "CMakeFiles/libmamba-static.dir/src/download/downloader.cpp.o" "CMakeFiles/libmamba-static.dir/src/download/mirror.cpp.o" "CMakeFiles/libmamba-static.dir/src/download/mirror_map.cpp.o" "CMakeFiles/libmamba-static.dir/src/download/mirror_impl.cpp.o" "CMakeFiles/libmamba-static.dir/src/download/request.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/singletons.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/activation.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/package_database_loader.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/channel_context.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/context.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/download_progress_bar.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/environments_manager.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/error_handling.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/transaction_context.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/link.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/history.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/menuinst.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/output.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/package_handling.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/package_cache.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/prefix_data.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/progress_bar.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/progress_bar_impl.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/pinning.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/package_fetcher.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/package_paths.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/query.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/repo_checker_store.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/run.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/shell_init.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/subdirdata.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/thread_utils.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/transaction.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/util.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/fsutil.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/util_os.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/virtual_packages.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/env_lockfile.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/execution.cpp.o" "CMakeFiles/libmamba-static.dir/src/core/timeref.cpp.o" "CMakeFiles/libmamba-static.dir/src/api/c_api.cpp.o" "CMakeFiles/libmamba-static.dir/src/api/channel_loader.cpp.o" "CMakeFiles/libmamba-static.dir/src/api/clean.cpp.o" "CMakeFiles/libmamba-static.dir/src/api/config.cpp.o" "CMakeFiles/libmamba-static.dir/src/api/configuration.cpp.o" "CMakeFiles/libmamba-static.dir/src/api/create.cpp.o" "CMakeFiles/libmamba-static.dir/src/api/info.cpp.o" "CMakeFiles/libmamba-static.dir/src/api/install.cpp.o" "CMakeFiles/libmamba-static.dir/src/api/list.cpp.o" "CMakeFiles/libmamba-static.dir/src/api/pip_utils.cpp.o" "CMakeFiles/libmamba-static.dir/src/api/remove.cpp.o" "CMakeFiles/libmamba-static.dir/src/api/repoquery.cpp.o" "CMakeFiles/libmamba-static.dir/src/api/shell.cpp.o" "CMakeFiles/libmamba-static.dir/src/api/update.cpp.o" "CMakeFiles/libmamba-static.dir/shell_scripts/mamba.sh.cpp.o" "CMakeFiles/libmamba-static.dir/shell_scripts/mamba.csh.cpp.o" "CMakeFiles/libmamba-static.dir/shell_scripts/mamba.bat.cpp.o" "CMakeFiles/libmamba-static.dir/shell_scripts/activate.bat.cpp.o" "CMakeFiles/libmamba-static.dir/shell_scripts/_mamba_activate.bat.cpp.o" "CMakeFiles/libmamba-static.dir/shell_scripts/mamba_hook.bat.cpp.o" "CMakeFiles/libmamba-static.dir/shell_scripts/mamba_hook.ps1.cpp.o" "CMakeFiles/libmamba-static.dir/shell_scripts/Mamba.psm1.cpp.o" "CMakeFiles/libmamba-static.dir/shell_scripts/mamba.xsh.cpp.o" "CMakeFiles/libmamba-static.dir/shell_scripts/mamba.fish.cpp.o" "CMakeFiles/libmamba-static.dir/shell_scripts/compile_pyc.py.cpp.o" "CMakeFiles/libmamba-static.dir/shell_scripts/mamba_completion.posix.cpp.o" "ext/solv-cpp/CMakeFiles/solv-cpp.dir/src/pool.cpp.o" "ext/solv-cpp/CMakeFiles/solv-cpp.dir/src/queue.cpp.o" "ext/solv-cpp/CMakeFiles/solv-cpp.dir/src/repo.cpp.o" "ext/solv-cpp/CMakeFiles/solv-cpp.dir/src/solvable.cpp.o" "ext/solv-cpp/CMakeFiles/solv-cpp.dir/src/solver.cpp.o" "ext/solv-cpp/CMakeFiles/solv-cpp.dir/src/transaction.cpp.o"
/Users/anutosh491/micromamba/envs/mamba-dev/bin/llvm-ranlib libmamba.a

The file has a couple duplications like the compiler archiver, ranlib etc (don't think that should be an issue but ..) .Just keeping one instance of those and trying to build again I get 

[  5%] Linking CXX static library libmamba.a
Error running link command: no such file or directorymake[2]: *** [libmamba/CMakeFiles/libmamba-static.dir/build.make:1983: libmamba/libmamba.a] Error 2
make[2]: Leaving directory '/Users/anutosh491/work/mamba/build'
make[1]: *** [CMakeFiles/Makefile2:133: libmamba/CMakeFiles/libmamba-static.dir/all] Error 2
make[1]: Leaving directory '/Users/anutosh491/work/mamba/build'

That being said I don't have any issues while executing a shared build (maybe through the following)

cmake -B build/ -G Ninja --preset mamba-unix-shared-debug-dev
cmake --build build/ --parallel

mamba info / micromamba info

(mamba-dev) anutosh491@Anutoshs-MacBook-Air mamba % micromamba info

       libmamba version : 2.0.2
     micromamba version : 2.0.2
           curl version : libcurl/8.9.1 (OpenSSL/3.3.2) SecureTransport zlib/1.3.1 zstd/1.5.6 libssh2/1.11.0 nghttp2/1.58.0
     libarchive version : libarchive 3.7.4 zlib/1.2.13 bz2lib/1.0.8 libzstd/1.5.6
       envs directories : /Users/anutosh491/micromamba/envs
          package cache : /Users/anutosh491/micromamba/pkgs
                          /Users/anutosh491/.mamba/pkgs
            environment : mamba-dev (active)
           env location : /Users/anutosh491/micromamba/envs/mamba-dev
      user config files : /Users/anutosh491/.mambarc
 populated config files : /Users/anutosh491/.condarc
       virtual packages : __unix=0=0
                          __osx=14.5=0
                          __archspec=1=arm64
               channels : https://conda.anaconda.org/conda-forge/noarch
                          https://conda.anaconda.org/conda-forge/osx-arm64
                          https://conda.anaconda.org/nodefaults/noarch
                          https://conda.anaconda.org/nodefaults/osx-arm64
       base environment : /Users/anutosh491/micromamba
               platform : osx-arm64

Logs

No response

environment.yml

No response

~/.condarc

(mamba-dev) anutosh491@Anutoshs-MacBook-Air mamba % cat ~/.condarc                                                               
channels:
  - conda-forge
  - nodefaults
channel_priority: strict
jjerphan commented 2 hours ago

Adapting the build script given on the feedstock might resolve the problem.

anutosh491 commented 2 hours ago

Adapting the build script given on the feedstock might resolve the problem.

Looking into it :)

Also just confirmed the sdk I have been using. Feedstock says we need version 11 or above (the one I am using is 14.4)

(base) anutosh491@Anutoshs-MacBook-Air SDKs % ls -l /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
lrwxr-xr-x  1 root  wheel  14 Sep 12 13:03 /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -> MacOSX14.4.sdk
anutosh491 commented 1 hour ago

Closing as everything boils down to these flags.

Simply making an executable called build.sh, copy pasting the contents from the build script and running it should solve the issues.