Closed johnmatter closed 8 months ago
Thank you for taking the time to file a bug report!
It seems the issue is coming from building the Python bindings. If you are not planning to use those, one option would be to avoid building the bindings: -DSTABLEHLO_ENABLE_BINDINGS_PYTHON=OFF
. I'm noticing that in the README we suggest setting this to ON
, which IMO is not the correct default. I will send a PR for that shortly.
I was able to reproduce your issue on my machine (I'm on Linux, so it seems this is not Mac-specific) and am currently looking to find a fix.
Usually the error you posted is when the llvm build didn't have MLIR_ENABLE_BINDINGS_PYTHON enabled.
I see that your snippet says it has it... which is confusing.
Can you try to re-run the building of MLIR again ?
We have a script build_mlir.sh
in the repository you can use to do that.
I'l think more about what it could be.
I can't reproduce your issue locally anymore. I think the commands may be out of order?
What is the value of build_dir
in cmake --build "$build_dir" --target all
?
The below works on my machine:
cmake -Bllvm-build -GNinja \
-Hllvm-project/llvm \
-DLLVM_INSTALL_UTILS=ON \
-DLLVM_ENABLE_LLD=OFF \
-DLLVM_ENABLE_PROJECTS=mlir \
-DLLVM_TARGETS_TO_BUILD=host \
-DLLVM_INCLUDE_TOOLS=ON \
-DMLIR_ENABLE_BINDINGS_PYTHON=ON \
-DLLVM_ENABLE_BINDINGS=OFF \
-DLLVM_VERSION_SUFFIX="" \
-DCMAKE_PLATFORM_NO_VERSIONED_SONAME:BOOL=ON \
-DLLVM_BUILD_TOOLS=OFF \
-DLLVM_INCLUDE_TESTS=OFF \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DLLVM_USE_SPLIT_DWARF=ON \
-DLLVM_ENABLE_ASSERTIONS=ON \
-DPython3_EXECUTABLE=$(which python3) \
-DPython3_FIND_STRATEGY=LOCATION \
-DPython3_FIND_UNVERSIONED_NAMES=FIRST
cmake -Bbuild -GNinja \
-DLLVM_ENABLE_LLD=OFF \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_ENABLE_ASSERTIONS=ON \
-DSTABLEHLO_ENABLE_BINDINGS_PYTHON=ON \
-DMLIR_DIR=/llvm-build/lib/cmake/mlir \
-DPython3_EXECUTABLE=$(which python3) \
-DPython3_FIND_STRATEGY=LOCATION \
-DPython3_FIND_UNVERSIONED_NAMES=FIRST
cmake --build build --target all
Another question for you: why these commits specifically? Does building at HEAD with the instructions in the README work for you?
Thank you both for the quick reply! I git pull
-ed the stablehlo repo and have an update. First, answers to your questions.
It seems the issue is coming from building the Python bindings. If you are not planning to use those, one option would be to avoid building the bindings
I'd like to enable the Python bindings if possible. For context, I'm trying to bridge StableHLO and C++ homomorphic encryption libraries. We're still figuring out the appropriate line to draw between the Python (also includes tensorflow, Jax, PyTorch) and C++ parts of the project. We're thinking about an interpreter/compiler that takes StableHLO as input and maps that to operators that the homomorphic encryption libraries understand.
What is the value of build_dir in cmake --build "$build_dir" --target all?
The value is llvm-build
. I meant to replace bash variable names with their values for clarity but I missed that one. Sorry about that.
Another question for you: why these commits specifically?
No reason for these specific commits. They just happened to be the ones I had in my clones of the repos and I wanted to include that info for completeness :)
Does building at HEAD with the instructions in the README work for you?
Previously, it did not. build_mlir.sh
failed because it couldn't find the correct python. I believe I have one from macOS itself and the one I actually use from homebrew (which is like apt or yum). Not sure yet if pyenv/conda would have helped here. I added the -DPython3_EXECUTABLE
lines to what build_mlir.sh
was doing (as pasted above) which seemed to get it building MLIR.
After git pull
for stablehlo just now and git checkout $(cat ../build_tools/llvm_version.txt)
for llvm-project, build_mlir.sh
seems to work fine when I run it like this to enable the bindings:
matter stablehlo $ MLIR_ENABLE_BINDINGS_PYTHON=ON build_tools/build_mlir.sh ${PWD}/llvm-project/ ${PWD}/llvm-build
This part still gives me the same errors as in my original post:
matter build $ cmake .. -GNinja \
-DLLVM_ENABLE_LLD=OFF \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_ENABLE_ASSERTIONS=ON \
-DSTABLEHLO_ENABLE_BINDINGS_PYTHON=ON \
-DMLIR_DIR=../llvm-build/lib/cmake/mlir
If I run it with STABLEHLO_ENABLE_BINDINGS_PYTHON=OFF
I get the following:
matter build $ cmake .. -GNinja \
> -DLLVM_ENABLE_LLD=OFF \
> -DCMAKE_BUILD_TYPE=Release \
> -DLLVM_ENABLE_ASSERTIONS=ON \
> -DSTABLEHLO_ENABLE_BINDINGS_PYTHON=OFF \
> -DMLIR_DIR=../llvm-build/lib/cmake/mlir
CMake Deprecation Warning at CMakeLists.txt:34 (cmake_policy):
The OLD behavior for policy CMP0116 will be removed from a future version
of CMake.
The cmake-policies(7) manual explains that the OLD behaviors of all
policies are deprecated and that a policy should be set to OLD only under
specific short-term circumstances. Projects should be ported to the NEW
behavior and not rely on setting a policy to OLD.
-- Building StableHLO with an installed MLIR
-- Using MLIRConfig.cmake in: /opt/homebrew/opt/llvm/lib/cmake/mlir
-- Using LLVMConfig.cmake in: /opt/homebrew/opt/llvm/lib/cmake/llvm
-- Building with -fPIC
-- Configuring done (0.1s)
CMake Error at /opt/homebrew/opt/llvm/lib/cmake/llvm/AddLLVM.cmake:556 (add_dependencies):
The dependency target "MLIRFunctionInterfaces" of target
"obj.StablehloPasses" does not exist.
Call Stack (most recent call first):
/opt/homebrew/opt/llvm/lib/cmake/mlir/AddMLIR.cmake:347 (llvm_add_library)
/opt/homebrew/opt/llvm/lib/cmake/mlir/AddMLIR.cmake:626 (add_mlir_library)
stablehlo/transforms/CMakeLists.txt:23 (add_mlir_dialect_library)
-- Generating done (0.0s)
CMake Generate step failed. Build files cannot be regenerated correctly.
Whoa smoking 🔫 found I think ... (famous last words?)
I see
-- Using MLIRConfig.cmake in: /opt/homebrew/opt/llvm/lib/cmake/mlir
-- Using LLVMConfig.cmake in: /opt/homebrew/opt/llvm/lib/cmake/llvm
CMake Error at /opt/homebrew/opt/llvm/lib/cmake/llvm/AddLLVM.cmake:556 (add_dependencies):
which means its not using the llvm-build directory.
This could be a function of cmake homebrew has some auto adding maybe that you installed llvm also.
I'm sure we should be able to handle this in our CMake but I believe this seems to be the issue.
llvm and cmake both show up in brew list
so that could be the issue. Thanks for spotting that. Do you know if there are flags to specify a different llvm directory?
I'll try to read the documentation a bit more and see what else is missing -- should be just setting MLIR_DIR
....
i asked: https://discourse.cmake.org/t/ideal-way-to-force-find-package-to-use-a-given-folder/10046 to see if there are better suggestions as well.
I just realized I have export PATH="/opt/homebrew/opt/llvm/bin:$PATH"
in my .bash_profile. Humongous whoops! I usually only put stuff in .bashrc so it snuck past me. Taking that out of .bash_profile, it looks like it builds now. Thank you so much for your time.
I ran the following from the top level stablehlo dir:
cmake -Bbuild -GNinja \
-DLLVM_ENABLE_LLD=OFF \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_ENABLE_ASSERTIONS=ON \
-DSTABLEHLO_ENABLE_BINDINGS_PYTHON=ON \
-DMLIR_DIR=llvm-build/lib/cmake/mlir
Output:
-- Building StableHLO with an installed MLIR
-- Using MLIRConfig.cmake in: llvm-build/lib/cmake/mlir
-- Using LLVMConfig.cmake in: llvm-build/lib/cmake/llvm
-- Building with -fPIC
-- Found python include dirs: /opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/include/python3.11
-- Found python libraries:
-- Found numpy v1.24.4: /opt/homebrew/lib/python3.11/site-packages/numpy/core/include
-- Checking for pybind11 in python path...
-- found (/opt/homebrew/lib/python3.11/site-packages/pybind11/share/cmake/pybind11)
-- Found pybind11: /opt/homebrew/lib/python3.11/site-packages/pybind11/include (found version "2.11.1")
-- Found pybind11 v2.11.1: /opt/homebrew/lib/python3.11/site-packages/pybind11/include
-- Python prefix = '', suffix = '', extension = '.cpython-311-darwin.so
-- Configuring done (0.4s)
-- Generating done (0.1s)
-- Build files have been written to: build
Great to hear. I thought something was odd.
@johnmatter we also publish nightly python wheels (see README) if you'd like to just pip install
and avoid building StableHLO yourself.
What happened?
I'm having trouble building on macOS 14.2.1. MLIR seems to build correctly. When I
cmake
StableHLO I get the following errors:Steps to reproduce your issue
Version information
I have the following commits checked out: