iree-org / iree

A retargetable MLIR-based machine learning compiler and runtime toolkit.
http://iree.dev/
Apache License 2.0
2.84k stars 611 forks source link

[CMAKE] [BAZEL] Need to import some pipelines in other MLIR Project #18588

Closed mingzheTerapines closed 1 month ago

mingzheTerapines commented 1 month ago

Request description

Remove Bazel and use MLIR/CIRCT project style cmakelist.txt.

What component(s) does this issue relate to?

No response

Additional context

We are experimentaly deploying iree to some dataflow devices Is there any plan to remove Bazel which is not friendly for other MLIR poject? I tried on libireecompiler.so but failed. Is there any way to import some of the pipelines from IREE such as buildStableHLOInputConversionPassPipeline() in iree/compiler/plugins/input/StableHLO/Conversion/Passes.h? I always got error like: ld.lld: error: undefined symbol because it is hidden:

~/workspace/tasks/magic$ readelf -sW ./third_party/iree/build/install/lib/[libIREECompiler.so](http://libireecompiler.so/) | grep registerStableHLOConversionPasses
  5465: 0000000001d38b60   664 FUNC    LOCAL  HIDDEN    13 _ZN4mlir13iree_compiler9stablehlo33registerStableHLOConversionPassesEv
benvanik commented 1 month ago

You don't have to use bazel - we fully support cmake.

mingzheTerapines commented 1 month ago

@benvanik Really aprreciate. But for MLIR Project, I can get link name by having name just after function add_mlir_library in cmakelists.txt .

add_mlir_library(MLIRIR
  AffineExpr.cpp
  AffineMap.cpp
  AsmPrinter.cpp

So should I use string after name in function iree_tablegen_library?

iree_tablegen_library(
  NAME
    CHLODecompositionPatterns
  TD_FILE

So I get CHLODecompositionPatterns as link name for lld, am I right?

ScottTodd commented 1 month ago

libireecompiler.so exposes our compiler API, not all symbols. Docs on the compiler API are here: https://iree.dev/reference/bindings/c-api/#compiler-api and a template repo showing usage is here: https://github.com/iree-org/iree-template-compiler-cmake.

If you want to use internal implementation details of the compiler, you'll need to statically link against the project and depend on the CMake (or Bazel, if you really want) libraries/targets.

You can use your CMake generator or debugger of choice to see which target names we create.

mingzheTerapines commented 1 month ago

Sorry for missing these pages. I will have a look. Really appreciate.

mingzheTerapines commented 1 month ago

We do want to use internal implementation details of the compiler. We were trying to expose these symbol. If we failed, we may model amd-aie adding plugins for IREE for dataflow devices.

stellaraccident commented 1 month ago

libIREECompiler.so only exposes public symbols. You are welcome to link to the internals as needed directly via cmake or bazel static deps (cmake is actively maintained, bazel gets some effort but none of the core devs can help much with it). You will of course need to deal with the usual issues that come with integrating llvm based projects.

If trying to build an iree backend that will ship with iree, there is a way to set up your project as a plug-in that will build together vs forcing you to figure something out. This is how IREE's air backend is made: https://github.com/nod-ai/iree-amd-aie

mingzheTerapines commented 1 month ago

We just set IREE_STATIC_LIB_DIR and include those *.a files, it works, but we have to carefully check which to be used.