Enabling Python binding on Windows causes build failure

oroppas commented 2 years ago

What happened?

The compilation of both compiler\src\iree\compiler\API\python\IREECTransforms.cpp and llvm-external-projects\iree-dialects\python\IREEDialectsModule.cpp failed:

IREECTransforms.cpp

[15/60] Building CXX object compiler\src\iree\compiler\API\python\CMakeFiles...ilerPythonModules.extension._ireecTransforms.dso.dir\IREECTransforms.cpp.obj
FAILED: compiler/src/iree/compiler/API/python/CMakeFiles/IREECompilerPythonModules.extension._ireecTransforms.dso.dir/IREECTransforms.cpp.obj
C:\PROGRA~1\MIB055~1\2022\COMMUN~1\VC\Tools\MSVC\1433~1.316\bin\Hostx64\x64\cl.exe  /nologo /TP -DIREECompilerPythonModules_extension__ireecTransforms_dso_EXPORTS -DMLIR_PYTHON_PACKAGE_PREFIX=iree.compiler. -ID:\packages\iree\iree\compiler\src -ID:\packages\iree\iree\llvm-external-projects\iree-dialects\include -ID:\packages\iree\iree\third_party\llvm-project\mlir\include -ID:\packages\iree\iree\third_party\mlir-hlo\include -ID:\packages\iree\iree\third_party\llvm-project\llvm\include -ID:\packages\iree\build\third_party\llvm-project\llvm\include -ID:\packages\iree\build\third_party\llvm-project\llvm\tools\mlir\include -ID:\packages\iree\iree\third_party\llvm-project\lld\include -ID:\packages\iree\build\third_party\llvm-project\llvm\tools\lld\include -external:IC:\Users\cd4mcq\AppData\Local\Programs\Python\Python39\include -external:ID:\packages\iree\iree\third_party\pybind11\include -external:W0 /DWIN32 /D_WINDOWS /EHsc /Zi /O2 /Ob1 -MD /EHsc /GR /bigobj /MP -std:c++17 /showIncludes /Focompiler\src\iree\compiler\API\python\CMakeFiles\IREECompilerPythonModules.extension._ireecTransforms.dso.dir\IREECTransforms.cpp.obj /Fdcompiler\src\iree\compiler\API\python\CMakeFiles\IREECompilerPythonModules.extension._ireecTransforms.dso.dir\ /FS -c D:\packages\iree\iree\compiler\src\iree\compiler\API\python\IREECTransforms.cpp
D:\packages\iree\iree\third_party\llvm-project\mlir\include\mlir-c/Bindings/Python/Interop.h(125): error C2143: syntax error: missing ';' before '*'
...

IREEDialectsModule.cpp

[46/60] Building CXX object compiler\src\iree\compiler\API\python\CMakeFiles...__\__\llvm-external-projects\iree-dialects\python\IREEDialectsModule.cpp.obj
FAILED: compiler/src/iree/compiler/API/python/CMakeFiles/IREECompilerPythonModules.extension._ireeDialects.dso.dir/__/__/__/__/__/__/llvm-external-projects/iree-dialects/python/IREEDialectsModule.cpp.obj
C:\PROGRA~1\MIB055~1\2022\COMMUN~1\VC\Tools\MSVC\1433~1.316\bin\Hostx64\x64\cl.exe  /nologo /TP -DIREECompilerPythonModules_extension__ireeDialects_dso_EXPORTS -DMLIR_PYTHON_PACKAGE_PREFIX=iree.compiler. -ID:\packages\iree\iree\compiler\src -ID:\packages\iree\iree\llvm-external-projects\iree-dialects\include -ID:\packages\iree\iree\third_party\llvm-project\mlir\include -ID:\packages\iree\iree\third_party\mlir-hlo\include -ID:\packages\iree\iree\third_party\llvm-project\llvm\include -ID:\packages\iree\build\third_party\llvm-project\llvm\include -ID:\packages\iree\build\third_party\llvm-project\llvm\tools\mlir\include -ID:\packages\iree\iree\third_party\llvm-project\lld\include -ID:\packages\iree\build\third_party\llvm-project\llvm\tools\lld\include -external:IC:\Users\cd4mcq\AppData\Local\Programs\Python\Python39\include -external:ID:\packages\iree\iree\third_party\pybind11\include -external:W0 /DWIN32 /D_WINDOWS /EHsc /Zi /O2 /Ob1 -MD /EHsc /GR /bigobj /MP -std:c++17 /showIncludes /Focompiler\src\iree\compiler\API\python\CMakeFiles\IREECompilerPythonModules.extension._ireeDialects.dso.dir\__\__\__\__\__\__\llvm-external-projects\iree-dialects\python\IREEDialectsModule.cpp.obj /Fdcompiler\src\iree\compiler\API\python\CMakeFiles\IREECompilerPythonModules.extension._ireeDialects.dso.dir\ /FS -c D:\packages\iree\iree\llvm-external-projects\iree-dialects\python\IREEDialectsModule.cpp
D:\packages\iree\iree\third_party\llvm-project\mlir\include\mlir-c/Bindings/Python/Interop.h(125): error C2143: syntax error: missing ';' before '*'
...

Steps to reproduce your issue

Build with -DIREE_BUILD_PYTHON_BINDINGS=ON on Windows with VC++ 19.33.31629

What component(s) does this issue relate to?

Python

Version information

No response

Additional context

According to the error message

third_party\llvm-project\mlir\include\mlir-c/Bindings/Python/Interop.h(125): error C2143: syntax error: missing ';' before '*'

and https://reviews.llvm.org/D125284?id=428267, simply deleting #include "mlir-c/Bindings/Python/Interop.h" in both compiler\src\iree\compiler\API\python\IREECTransforms.cpp and llvm-external-projects\iree-dialects\python\IREEDialectsModule.cpp fixes the compilation issue.

ScottTodd commented 2 years ago

To set expectations a bit:

We have a few developers actively using Windows for work on the compiler and runtime (but not usually the Python bindings)
- -DIREE_BUILD_PYTHON_BINDINGS=ON has been broken with different error messages on my development machine for months-years
We only have continuous builds running on Linux, so Windows can break occasionally
We only publish releases (including Python wheels to pypi) on macOS and Linux at the moment

Personally, I use the Python bindings through Colab when on Windows and test changes to them from Linux.

We'd be happy to improve the situation there, it's just a matter of investing proportional to the demand. So thanks for the issue report - that's one more vote for fixing things and keeping them working :). Applying that header change from the upstream MLIR Python bindings seems reasonable to me. We should also be able to add Windows CI builds somewhat soon - we're just finishing up migrating from one CI provider (Kokoro) to another (GitHub Actions).

ScottTodd commented 1 year ago

I think this particular issue was fixed, and we now have some continuous builds running on Windows with the Python bindings enabled 🥳

That configuration still isn't as well supported as Linux, and we aren't publishing official Python packages for Windows yet, but we're getting closer to that.

iree-org / iree