Installation error on Mac OS arm64 architecture

[andreadelprete]

Hi @Tim-Salzmann ,

I hope you're doing fine! I am trying to install l4casadi on an M2 MacBook, arm64 architecture. I have installed all the required dependencies via pip, but when I try to install l4casadi with pip install . --no-build-isolation I get the following error message:

Processing /Users/adelprete/devel/docker/orc23/l4casadi
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: torch in /Users/adelprete/miniconda3/lib/python3.10/site-packages (from l4casadi==1.4.1) (2.1.2)
Requirement already satisfied: jinja2>=3.1 in /Users/adelprete/miniconda3/lib/python3.10/site-packages (from l4casadi==1.4.1) (3.1.4)
Requirement already satisfied: casadi>=3.6 in /Users/adelprete/miniconda3/lib/python3.10/site-packages (from l4casadi==1.4.1) (3.6.5)
Requirement already satisfied: numpy in /Users/adelprete/miniconda3/lib/python3.10/site-packages (from casadi>=3.6->l4casadi==1.4.1) (1.26.2)
Requirement already satisfied: MarkupSafe>=2.0 in /Users/adelprete/miniconda3/lib/python3.10/site-packages (from jinja2>=3.1->l4casadi==1.4.1) (2.1.3)
Requirement already satisfied: filelock in /Users/adelprete/miniconda3/lib/python3.10/site-packages (from torch->l4casadi==1.4.1) (3.13.1)
Requirement already satisfied: typing-extensions in /Users/adelprete/miniconda3/lib/python3.10/site-packages (from torch->l4casadi==1.4.1) (4.11.0)
Requirement already satisfied: sympy in /Users/adelprete/miniconda3/lib/python3.10/site-packages (from torch->l4casadi==1.4.1) (1.12)
Requirement already satisfied: networkx in /Users/adelprete/miniconda3/lib/python3.10/site-packages (from torch->l4casadi==1.4.1) (3.3)
Requirement already satisfied: fsspec in /Users/adelprete/miniconda3/lib/python3.10/site-packages (from torch->l4casadi==1.4.1) (2023.10.0)
Requirement already satisfied: mpmath>=0.19 in /Users/adelprete/miniconda3/lib/python3.10/site-packages (from sympy->torch->l4casadi==1.4.1) (1.3.0)
Building wheels for collected packages: l4casadi
  Building wheel for l4casadi (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for l4casadi (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [257 lines of output]
      /Users/adelprete/miniconda3/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py:108: _BetaConfiguration: Support for `[tool.setuptools]` in `pyproject.toml` is still *beta*.
        warnings.warn(msg, _BetaConfiguration)

      --------------------------------------------------------------------------------
      -- Trying 'Ninja' generator
      --------------------------------
      ---------------------------
      ----------------------
      -----------------
      ------------
      -------
      --
      Not searching for unused variables given on the command line.
      -- The C compiler identification is AppleClang 15.0.0.15000100
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- The CXX compiler identification is AppleClang 15.0.0.15000100
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Configuring done (3.5s)
      -- Generating done (0.0s)
      -- Build files have been written to: /Users/adelprete/devel/docker/orc23/l4casadi/_cmake_test_compile/build
      --
      -------
      ------------
      -----------------
      ----------------------
      ---------------------------
      --------------------------------
      -- Trying 'Ninja' generator - success
      --------------------------------------------------------------------------------

      Configuring Project
        Working directory:
          /Users/adelprete/devel/docker/orc23/l4casadi/_skbuild/macosx-13.0-x86_64-3.10/cmake-build
        Command:
          /usr/local/bin/cmake /Users/adelprete/devel/docker/orc23/l4casadi/libl4casadi -G Ninja -DCMAKE_MAKE_PROGRAM:FILEPATH=/Users/adelprete/miniconda3/lib/python3.10/site-packages/ninja/data/bin/ninja --no-warn-unused-cli -DCMAKE_INSTALL_PREFIX:PATH=/Users/adelprete/devel/docker/orc23/l4casadi/_skbuild/macosx-13.0-x86_64-3.10/cmake-install -DPYTHON_VERSION_STRING:STRING=3.10.13 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/Users/adelprete/miniconda3/lib/python3.10/site-packages/skbuild/resources/cmake -DPYTHON_EXECUTABLE:PATH=/Users/adelprete/miniconda3/bin/python -DPYTHON_INCLUDE_DIR:PATH=/Users/adelprete/miniconda3/include/python3.10 -DPYTHON_LIBRARY:PATH=/Users/adelprete/miniconda3/lib/libpython3.10.dylib -DPython_EXECUTABLE:PATH=/Users/adelprete/miniconda3/bin/python -DPython_ROOT_DIR:PATH=/Users/adelprete/miniconda3 -DPython_FIND_REGISTRY:STRING=NEVER -DPython_INCLUDE_DIR:PATH=/Users/adelprete/miniconda3/include/python3.10 -DPython_NumPy_INCLUDE_DIRS:PATH=/Users/adelprete/miniconda3/lib/python3.10/site-packages/numpy/core/include -DPython3_EXECUTABLE:PATH=/Users/adelprete/miniconda3/bin/python -DPython3_ROOT_DIR:PATH=/Users/adelprete/miniconda3 -DPython3_FIND_REGISTRY:STRING=NEVER -DPython3_INCLUDE_DIR:PATH=/Users/adelprete/miniconda3/include/python3.10 -DPython3_NumPy_INCLUDE_DIRS:PATH=/Users/adelprete/miniconda3/lib/python3.10/site-packages/numpy/core/include -DCMAKE_MAKE_PROGRAM:FILEPATH=/Users/adelprete/miniconda3/lib/python3.10/site-packages/ninja/data/bin/ninja -DCMAKE_TORCH_PATH=/Users/adelprete/miniconda3/lib/python3.10/site-packages/torch -DCMAKE_OSX_DEPLOYMENT_TARGET:STRING=13.0 -DCMAKE_OSX_ARCHITECTURES:STRING=x86_64 -DCMAKE_BUILD_TYPE:STRING=Release

      CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
        Compatibility with CMake < 3.5 will be removed from a future version of
        CMake.

        Update the VERSION argument <min> value or use a ...<max> suffix to tell
        CMake that the project does not need compatibility with older versions.

      Not searching for unused variables given on the command line.

      -- The C compiler identification is AppleClang 15.0.0.15000100
      -- The CXX compiler identification is AppleClang 15.0.0.15000100
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Could not find nvcc, please set CUDAToolkit_ROOT.
      CMake Warning (dev) at CMakeLists.txt:6 (find_package):
        Policy CMP0146 is not set: The FindCUDA module is removed.  Run "cmake
        --help-policy CMP0146" for policy details.  Use the cmake_policy command to
        set the policy and suppress this warning.

      This warning is for project developers.  Use -Wno-dev to suppress it.

      CUDA_TOOLKIT_ROOT_DIR not found or specified
      -- Could NOT find CUDA (missing: CUDA_TOOLKIT_ROOT_DIR CUDA_NVCC_EXECUTABLE CUDA_INCLUDE_DIRS CUDA_CUDART_LIBRARY)
      CMake Warning at /Users/adelprete/miniconda3/lib/python3.10/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:22 (message):
        static library kineto_LIBRARY-NOTFOUND not found.
      Call Stack (most recent call first):
        /Users/adelprete/miniconda3/lib/python3.10/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:127 (append_torchlib_if_found)
        CMakeLists.txt:16 (find_package)

      -- Found Torch: /Users/adelprete/miniconda3/lib/python3.10/site-packages/torch/lib/libtorch.dylib
      -- Configuring done (3.7s)
      -- Generating done (0.0s)
      -- Build files have been written to: /Users/adelprete/devel/docker/orc23/l4casadi/_skbuild/macosx-13.0-x86_64-3.10/cmake-build
      [1/3] Building CXX object CMakeFiles/l4casadi.dir/src/l4casadi.cpp.o
      [2/3] Linking CXX shared library libl4casadi.dylib
      FAILED: libl4casadi.dylib
      : && /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -O3 -DNDEBUG -arch x86_64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.2.sdk -mmacosx-version-min=13.0 -dynamiclib -Wl,-headerpad_max_install_names  -o libl4casadi.dylib -install_name @rpath/libl4casadi.dylib CMakeFiles/l4casadi.dir/src/l4casadi.cpp.o  -Wl,-rpath,/Users/adelprete/miniconda3/lib/python3.10/site-packages/torch/lib  /Users/adelprete/miniconda3/lib/python3.10/site-packages/torch/lib/libc10.dylib  /Users/adelprete/miniconda3/lib/python3.10/site-packages/torch/lib/libtorch.dylib  /Users/adelprete/miniconda3/lib/python3.10/site-packages/torch/lib/libtorch_cpu.dylib  /Users/adelprete/miniconda3/lib/python3.10/site-packages/torch/lib/libc10.dylib && :
      ld: warning: ignoring file '/Users/adelprete/miniconda3/lib/python3.10/site-packages/torch/lib/libc10.dylib': found architecture 'arm64', required architecture 'x86_64'
      ld: warning: ignoring file '/Users/adelprete/miniconda3/lib/python3.10/site-packages/torch/lib/libc10.dylib': found architecture 'arm64', required architecture 'x86_64'
      ld: warning: ignoring file '/Users/adelprete/miniconda3/lib/python3.10/site-packages/torch/lib/libtorch_cpu.dylib': found architecture 'arm64', required architecture 'x86_64'
      ld: warning: ignoring file '/Users/adelprete/miniconda3/lib/python3.10/site-packages/torch/lib/libtorch.dylib': found architecture 'arm64', required architecture 'x86_64'
      ld: Undefined symbols:
...

The error message keeps going, but I think the interesting part is the last few lines of what I have copied above. In particular, it complains about finding arm64 libraries, while it was expecting x86_64 ones. I think this is due to the compilation flag -arch x86_64. However, I cannot figure out how to change it by modifying the project configuration files. I guess the problem comes from the fact that skbuild thinks my machine has an x86 architecture, while instead it is arm. Any idea how to fix this?

[andreadelprete]

I've just managed to fix it by adding the following lines in the setup.py script:

import skbuild
skbuild.constants.set_skbuild_plat_name('macosx-13.0-arm64')

It's ugly but it worked. No idea if there is a better way to do it though.

[Tim-Salzmann]

Hi Andrea,

Thanks for the message. My best guess is that maybe your Xcode Developer Tools are x86 running via Rosetta? I am using a M2 too and have not seen this problem before. Would you mind checking so we know if this was the underlying problem?

Glad you managed to fix it!

Best Tim

[andreadelprete]

Hi Tim, thanks for the answer. That's a good guess. I've checked by running gcc -v, but from the output I get it doesn't seem the Xcode Developer Tools are x86:

Apple clang version 15.0.0 (clang-1500.1.0.2.5)
Target: arm64-apple-darwin22.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

[Tim-Salzmann]

Could you check the same with the installed cmake version?

Thanks

[andreadelprete]

Not sure how to do that. Running cmake --version does not give me any info. If I create an empty cmake project and check the default value of the CMAKE variables I don't see anything useful. I was expecting to see a value for CMAKE_OSX_ARCHITECTURES, but by default it's empty. Any suggestion?

However, if I run make --version I get the following:

make --version
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for i386-apple-darwin11.3.0

I guess this confirms your guess. The installed make seems to be for x86 rather than for arm64.

[Tim-Salzmann]

I guess this confirms your guess. The installed make seems to be for x86 rather than for arm64.

I think this would explain it. Feel free to close the issue if you believe it is resolved.

Best Tim

[andreadelprete]

Ok, thanks for the help!