finch-tensor / Finch-mlir

Rewriting Finch in mlir
0 stars 0 forks source link

BLD: Initial linux wheel build setup #5

Open mtsokol opened 6 days ago

mtsokol commented 6 days ago

This PR introduces linux wheel build process. In the CI there are two jobs:

  1. build-wheel:
    • builds LLVM in a build dir
    • installs LLVM in an install dir
    • builds Finch dialect in a build dir in a standalone mode
    • installs Finch dialect to setuptools-install dir
    • uploads build artifact
  2. release-wheel:
    • downloads build artifact
    • creates a release
mtsokol commented 5 days ago

@willow-ahrens I did my homework yesterday and building Finch dialect in a standalone mode yields a wheel of size 200MB.

The CI run first builds LLVM and then Finch dialect in an out-of-tree mode. Only the Finch part build is placed in a wheel and uploaded as a release artifact: https://github.com/nullplay/Finch-mlir/releases

Then the wheel can be directly installed via pip, here are example shell commands:

conda create --name temp-env python=3.10 --yes

conda activate temp-env

pip install PyYAML numpy

pip install https://github.com/nullplay/Finch-mlir/releases/download/latest/mlir_finch-0.0.1-cp310-cp310-linux_x86_64.whl

python -c 'from mlir_finch.dialects import finch, arith, func, linalg, sparse_tensor, tensor'

conda env remove --name temp-env --yes

I think mlir_c_runner_utils lib is missing from the CMake setup - it should be included in the build - it's required to run ExecutionEngine from the Python layer (but I think it just a few MBs only).

This build setup will be run periodically. Now it's only python 3.10, linux, x86_64 but target build matrix could be [3.10, 3.11, 3.12] x [linux, macos, win64] x [x86_64, arm64].

CC @nullplay @hameerabbasi

mtsokol commented 5 days ago

It's for a later discussion but I see that libFinchCAPITestLib.so is also shipped in the install dir (150MB file size). Assuming by its name it might not be necessary to include it.