search

openxla / stablehlo

Backward compatible ML compute opset inspired by HLO/MHLO
Apache License 2.0
402 stars 110 forks source link

Trouble building StableHLO on macOS #2001

Closed johnmatter closed 8 months ago

johnmatter commented 8 months ago

What happened?

I'm having trouble building on macOS 14.2.1. MLIR seems to build correctly. When I cmake StableHLO I get the following errors:

CMake Deprecation Warning at CMakeLists.txt:34 (cmake_policy):
  The OLD behavior for policy CMP0116 will be removed from a future version
  of CMake.

  The cmake-policies(7) manual explains that the OLD behaviors of all
  policies are deprecated and that a policy should be set to OLD only under
  specific short-term circumstances.  Projects should be ported to the NEW
  behavior and not rely on setting a policy to OLD.

-- Building StableHLO with an installed MLIR
-- Using MLIRConfig.cmake in: /opt/homebrew/opt/llvm/lib/cmake/mlir
-- Using LLVMConfig.cmake in: /opt/homebrew/opt/llvm/lib/cmake/llvm
-- Building with -fPIC
-- Found python include dirs: /opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/include/python3.11
-- Found python libraries:
-- Found numpy v1.24.4: /opt/homebrew/lib/python3.11/site-packages/numpy/core/include
-- Checking for pybind11 in python path...
-- found (/opt/homebrew/lib/python3.11/site-packages/pybind11/share/cmake/pybind11)
-- Found pybind11: /opt/homebrew/lib/python3.11/site-packages/pybind11/include (found version "2.11.1")
-- Found pybind11 v2.11.1: /opt/homebrew/lib/python3.11/site-packages/pybind11/include
-- Python prefix = '', suffix = '', extension = '.cpython-311-darwin.so
CMake Error at /opt/homebrew/opt/llvm/lib/cmake/mlir/AddMLIRPython.cmake:498 (get_target_property):
  get_target_property() called with non-existent target "MLIRPythonSources".
Call Stack (most recent call first):
  /opt/homebrew/opt/llvm/lib/cmake/mlir/AddMLIRPython.cmake:448 (_flatten_mlir_python_targets)
  stablehlo/integrations/python/CMakeLists.txt:113 (add_mlir_python_common_capi_library)

CMake Error at /opt/homebrew/opt/llvm/lib/cmake/mlir/AddMLIRPython.cmake:499 (get_target_property):
  get_target_property() called with non-existent target "MLIRPythonSources".
Call Stack (most recent call first):
  /opt/homebrew/opt/llvm/lib/cmake/mlir/AddMLIRPython.cmake:448 (_flatten_mlir_python_targets)
  stablehlo/integrations/python/CMakeLists.txt:113 (add_mlir_python_common_capi_library)

CMake Error at /opt/homebrew/opt/llvm/lib/cmake/mlir/AddMLIRPython.cmake:498 (get_target_property):
  get_target_property() called with non-existent target
  "MLIRPythonExtension.RegisterEverything".
Call Stack (most recent call first):
  /opt/homebrew/opt/llvm/lib/cmake/mlir/AddMLIRPython.cmake:448 (_flatten_mlir_python_targets)
  stablehlo/integrations/python/CMakeLists.txt:113 (add_mlir_python_common_capi_library)

CMake Error at /opt/homebrew/opt/llvm/lib/cmake/mlir/AddMLIRPython.cmake:499 (get_target_property):
  get_target_property() called with non-existent target
  "MLIRPythonExtension.RegisterEverything".
Call Stack (most recent call first):
  /opt/homebrew/opt/llvm/lib/cmake/mlir/AddMLIRPython.cmake:448 (_flatten_mlir_python_targets)
  stablehlo/integrations/python/CMakeLists.txt:113 (add_mlir_python_common_capi_library)

CMake Error at /opt/homebrew/opt/llvm/lib/cmake/mlir/AddMLIRPython.cmake:498 (get_target_property):
  get_target_property() called with non-existent target "MLIRPythonSources".
Call Stack (most recent call first):
  /opt/homebrew/opt/llvm/lib/cmake/mlir/AddMLIRPython.cmake:246 (_flatten_mlir_python_targets)
  stablehlo/integrations/python/CMakeLists.txt:129 (add_mlir_python_modules)

CMake Error at /opt/homebrew/opt/llvm/lib/cmake/mlir/AddMLIRPython.cmake:499 (get_target_property):
  get_target_property() called with non-existent target "MLIRPythonSources".
Call Stack (most recent call first):
  /opt/homebrew/opt/llvm/lib/cmake/mlir/AddMLIRPython.cmake:246 (_flatten_mlir_python_targets)
  stablehlo/integrations/python/CMakeLists.txt:129 (add_mlir_python_modules)

CMake Error at /opt/homebrew/opt/llvm/lib/cmake/mlir/AddMLIRPython.cmake:498 (get_target_property):
  get_target_property() called with non-existent target
  "MLIRPythonExtension.RegisterEverything".
Call Stack (most recent call first):
  /opt/homebrew/opt/llvm/lib/cmake/mlir/AddMLIRPython.cmake:246 (_flatten_mlir_python_targets)
  stablehlo/integrations/python/CMakeLists.txt:129 (add_mlir_python_modules)

CMake Error at /opt/homebrew/opt/llvm/lib/cmake/mlir/AddMLIRPython.cmake:499 (get_target_property):
  get_target_property() called with non-existent target
  "MLIRPythonExtension.RegisterEverything".
Call Stack (most recent call first):
  /opt/homebrew/opt/llvm/lib/cmake/mlir/AddMLIRPython.cmake:246 (_flatten_mlir_python_targets)
  stablehlo/integrations/python/CMakeLists.txt:129 (add_mlir_python_modules)

-- Configuring incomplete, errors occurred!

Steps to reproduce your issue

# Build LLVM
cd stablehlo
cmake -GNinja \
   "-Hllvm-project/llvm" \
   "-Bllvm-build" \
   -DLLVM_INSTALL_UTILS=ON \
   -DLLVM_ENABLE_LLD=OFF \
   -DLLVM_ENABLE_PROJECTS=mlir \
   -DLLVM_TARGETS_TO_BUILD=host \
   -DLLVM_INCLUDE_TOOLS=ON \
   -DMLIR_ENABLE_BINDINGS_PYTHON=ON \
   -DLLVM_ENABLE_BINDINGS=OFF \
   -DLLVM_VERSION_SUFFIX="" \
   -DCMAKE_PLATFORM_NO_VERSIONED_SONAME:BOOL=ON \
   -DLLVM_BUILD_TOOLS=OFF \
   -DLLVM_INCLUDE_TESTS=OFF \
   -DCMAKE_BUILD_TYPE=RelWithDebInfo \
   -DLLVM_USE_SPLIT_DWARF=ON \
   -DLLVM_ENABLE_ASSERTIONS=ON \
   -DPython3_EXECUTABLE=/opt/homebrew/bin/python3 \
   -DPython3_FIND_STRATEGY=LOCATION \
   -DPython3_FIND_UNVERSIONED_NAMES=FIRST

cmake --build "$build_dir" --target all

# Build StableHLO
cd build
cmake .. -GNinja \
  -DLLVM_ENABLE_LLD=OFF \
  -DCMAKE_BUILD_TYPE=Release \
  -DLLVM_ENABLE_ASSERTIONS=ON \
  -DSTABLEHLO_ENABLE_BINDINGS_PYTHON=ON \
  -DMLIR_DIR=/llvm-build/lib/cmake/mlir \
  -DPython3_EXECUTABLE=/opt/homebrew/bin/python3 \
  -DPython3_FIND_STRATEGY=LOCATION \
  -DPython3_FIND_UNVERSIONED_NAMES=FIRST

Version information

I have the following commits checked out:

c4c0465d71f20717e522a7011a6ca55ef654e9ec - stablehlo
d6c2cbbc6513bd412b34f3bf70e21b5a363b2fd9 - llvm-project
mlevesquedion commented 8 months ago

Thank you for taking the time to file a bug report!

It seems the issue is coming from building the Python bindings. If you are not planning to use those, one option would be to avoid building the bindings: -DSTABLEHLO_ENABLE_BINDINGS_PYTHON=OFF. I'm noticing that in the README we suggest setting this to ON, which IMO is not the correct default. I will send a PR for that shortly.

I was able to reproduce your issue on my machine (I'm on Linux, so it seems this is not Mac-specific) and am currently looking to find a fix.

fzakaria commented 8 months ago

Usually the error you posted is when the llvm build didn't have MLIR_ENABLE_BINDINGS_PYTHON enabled. I see that your snippet says it has it... which is confusing. Can you try to re-run the building of MLIR again ? We have a script build_mlir.sh in the repository you can use to do that.

I'l think more about what it could be.

mlevesquedion commented 8 months ago

I can't reproduce your issue locally anymore. I think the commands may be out of order?

What is the value of build_dir in cmake --build "$build_dir" --target all?

The below works on my machine:

cmake -Bllvm-build -GNinja \
   -Hllvm-project/llvm \
   -DLLVM_INSTALL_UTILS=ON \
   -DLLVM_ENABLE_LLD=OFF \
   -DLLVM_ENABLE_PROJECTS=mlir \
   -DLLVM_TARGETS_TO_BUILD=host \
   -DLLVM_INCLUDE_TOOLS=ON \
   -DMLIR_ENABLE_BINDINGS_PYTHON=ON \
   -DLLVM_ENABLE_BINDINGS=OFF \
   -DLLVM_VERSION_SUFFIX="" \
   -DCMAKE_PLATFORM_NO_VERSIONED_SONAME:BOOL=ON \
   -DLLVM_BUILD_TOOLS=OFF \
   -DLLVM_INCLUDE_TESTS=OFF \
   -DCMAKE_BUILD_TYPE=RelWithDebInfo \
   -DLLVM_USE_SPLIT_DWARF=ON \
   -DLLVM_ENABLE_ASSERTIONS=ON \
   -DPython3_EXECUTABLE=$(which python3) \
   -DPython3_FIND_STRATEGY=LOCATION \
   -DPython3_FIND_UNVERSIONED_NAMES=FIRST

cmake -Bbuild -GNinja \
  -DLLVM_ENABLE_LLD=OFF \
  -DCMAKE_BUILD_TYPE=Release \
  -DLLVM_ENABLE_ASSERTIONS=ON \
  -DSTABLEHLO_ENABLE_BINDINGS_PYTHON=ON \
  -DMLIR_DIR=/llvm-build/lib/cmake/mlir \
  -DPython3_EXECUTABLE=$(which python3) \
  -DPython3_FIND_STRATEGY=LOCATION \
  -DPython3_FIND_UNVERSIONED_NAMES=FIRST

cmake --build build --target all

Another question for you: why these commits specifically? Does building at HEAD with the instructions in the README work for you?

johnmatter commented 8 months ago

Thank you both for the quick reply! I git pull-ed the stablehlo repo and have an update. First, answers to your questions.

It seems the issue is coming from building the Python bindings. If you are not planning to use those, one option would be to avoid building the bindings

I'd like to enable the Python bindings if possible. For context, I'm trying to bridge StableHLO and C++ homomorphic encryption libraries. We're still figuring out the appropriate line to draw between the Python (also includes tensorflow, Jax, PyTorch) and C++ parts of the project. We're thinking about an interpreter/compiler that takes StableHLO as input and maps that to operators that the homomorphic encryption libraries understand.

What is the value of build_dir in cmake --build "$build_dir" --target all?

The value is llvm-build. I meant to replace bash variable names with their values for clarity but I missed that one. Sorry about that.

Another question for you: why these commits specifically?

No reason for these specific commits. They just happened to be the ones I had in my clones of the repos and I wanted to include that info for completeness :)

Does building at HEAD with the instructions in the README work for you?

Previously, it did not. build_mlir.sh failed because it couldn't find the correct python. I believe I have one from macOS itself and the one I actually use from homebrew (which is like apt or yum). Not sure yet if pyenv/conda would have helped here. I added the -DPython3_EXECUTABLE lines to what build_mlir.sh was doing (as pasted above) which seemed to get it building MLIR.

After git pull for stablehlo just now and git checkout $(cat ../build_tools/llvm_version.txt) for llvm-project, build_mlir.sh seems to work fine when I run it like this to enable the bindings:

matter stablehlo $ MLIR_ENABLE_BINDINGS_PYTHON=ON build_tools/build_mlir.sh ${PWD}/llvm-project/ ${PWD}/llvm-build

This part still gives me the same errors as in my original post:

matter build $ cmake .. -GNinja \
  -DLLVM_ENABLE_LLD=OFF \
  -DCMAKE_BUILD_TYPE=Release \
  -DLLVM_ENABLE_ASSERTIONS=ON \
  -DSTABLEHLO_ENABLE_BINDINGS_PYTHON=ON \
  -DMLIR_DIR=../llvm-build/lib/cmake/mlir

If I run it with STABLEHLO_ENABLE_BINDINGS_PYTHON=OFF I get the following:

matter build $ cmake .. -GNinja \
>   -DLLVM_ENABLE_LLD=OFF \
>   -DCMAKE_BUILD_TYPE=Release \
>   -DLLVM_ENABLE_ASSERTIONS=ON \
>   -DSTABLEHLO_ENABLE_BINDINGS_PYTHON=OFF \
>   -DMLIR_DIR=../llvm-build/lib/cmake/mlir
CMake Deprecation Warning at CMakeLists.txt:34 (cmake_policy):
  The OLD behavior for policy CMP0116 will be removed from a future version
  of CMake.

  The cmake-policies(7) manual explains that the OLD behaviors of all
  policies are deprecated and that a policy should be set to OLD only under
  specific short-term circumstances.  Projects should be ported to the NEW
  behavior and not rely on setting a policy to OLD.

-- Building StableHLO with an installed MLIR
-- Using MLIRConfig.cmake in: /opt/homebrew/opt/llvm/lib/cmake/mlir
-- Using LLVMConfig.cmake in: /opt/homebrew/opt/llvm/lib/cmake/llvm
-- Building with -fPIC
-- Configuring done (0.1s)
CMake Error at /opt/homebrew/opt/llvm/lib/cmake/llvm/AddLLVM.cmake:556 (add_dependencies):
  The dependency target "MLIRFunctionInterfaces" of target
  "obj.StablehloPasses" does not exist.
Call Stack (most recent call first):
  /opt/homebrew/opt/llvm/lib/cmake/mlir/AddMLIR.cmake:347 (llvm_add_library)
  /opt/homebrew/opt/llvm/lib/cmake/mlir/AddMLIR.cmake:626 (add_mlir_library)
  stablehlo/transforms/CMakeLists.txt:23 (add_mlir_dialect_library)

-- Generating done (0.0s)
CMake Generate step failed.  Build files cannot be regenerated correctly.
fzakaria commented 8 months ago

Whoa smoking 🔫 found I think ... (famous last words?)

I see 

-- Using MLIRConfig.cmake in: /opt/homebrew/opt/llvm/lib/cmake/mlir
-- Using LLVMConfig.cmake in: /opt/homebrew/opt/llvm/lib/cmake/llvm
CMake Error at /opt/homebrew/opt/llvm/lib/cmake/llvm/AddLLVM.cmake:556 (add_dependencies):

which means its not using the llvm-build directory.

This could be a function of cmake homebrew has some auto adding maybe that you installed llvm also.

I'm sure we should be able to handle this in our CMake but I believe this seems to be the issue.

johnmatter commented 8 months ago

llvm and cmake both show up in brew list so that could be the issue. Thanks for spotting that. Do you know if there are flags to specify a different llvm directory?

fzakaria commented 8 months ago

I'll try to read the documentation a bit more and see what else is missing -- should be just setting MLIR_DIR....

i asked: https://discourse.cmake.org/t/ideal-way-to-force-find-package-to-use-a-given-folder/10046 to see if there are better suggestions as well.

johnmatter commented 8 months ago

I just realized I have export PATH="/opt/homebrew/opt/llvm/bin:$PATH" in my .bash_profile. Humongous whoops! I usually only put stuff in .bashrc so it snuck past me. Taking that out of .bash_profile, it looks like it builds now. Thank you so much for your time.

I ran the following from the top level stablehlo dir:

cmake -Bbuild -GNinja \
  -DLLVM_ENABLE_LLD=OFF \
  -DCMAKE_BUILD_TYPE=Release \
  -DLLVM_ENABLE_ASSERTIONS=ON \
  -DSTABLEHLO_ENABLE_BINDINGS_PYTHON=ON \
  -DMLIR_DIR=llvm-build/lib/cmake/mlir

Output:

-- Building StableHLO with an installed MLIR
-- Using MLIRConfig.cmake in: llvm-build/lib/cmake/mlir
-- Using LLVMConfig.cmake in: llvm-build/lib/cmake/llvm
-- Building with -fPIC
-- Found python include dirs: /opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/include/python3.11
-- Found python libraries:
-- Found numpy v1.24.4: /opt/homebrew/lib/python3.11/site-packages/numpy/core/include
-- Checking for pybind11 in python path...
-- found (/opt/homebrew/lib/python3.11/site-packages/pybind11/share/cmake/pybind11)
-- Found pybind11: /opt/homebrew/lib/python3.11/site-packages/pybind11/include (found version "2.11.1")
-- Found pybind11 v2.11.1: /opt/homebrew/lib/python3.11/site-packages/pybind11/include
-- Python prefix = '', suffix = '', extension = '.cpython-311-darwin.so
-- Configuring done (0.4s)
-- Generating done (0.1s)
-- Build files have been written to: build
fzakaria commented 8 months ago

Great to hear. I thought something was odd.

@johnmatter we also publish nightly python wheels (see README) if you'd like to just pip install and avoid building StableHLO yourself.