Incompatible version of Polygaist is installed if build-with-polygeist.sh is used

[geexie]

Does this script is still valid? It checks out version of Polygeist from polygeist-version.txt which seems to be incompatible and miss some required headers so I'm getting the error

[10/33] Building CXX object tools/mlir/tools/polymer/lib/Target/CMakeFiles/obj.PolymerTargetOpenScop.dir/OpenScop/ConvertFromOpenScop.cc.o
FAILED: tools/mlir/tools/polymer/lib/Target/CMakeFiles/obj.PolymerTargetOpenScop.dir/OpenScop/ConvertFromOpenScop.cc.o 
/usr/bin/c++  -DGTEST_HAS_RTTI=0 -DMLIR_CUDA_CONVERSIONS_ENABLED=0 -DMLIR_INCLUDE_TESTS -DMLIR_ROCM_CONVERSIONS_ENABLED=0 -D_DEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Itools/mlir/tools/polymer/lib/Target -I/home/geek/py/base/Polygeist-polymer/mlir/tools/polymer/lib/Target -Iinclude -I/home/geek/py/base/Polygeist-polymer/llvm/include -I/home/geek/py/base/Polygeist-polymer/mlir/include -Itools/mlir/include -Itools/mlir/tools/polymer/pluto/include -I/home/geek/py/base/Polygeist-polymer/mlir/tools/polymer/include -Itools/mlir/tools/polymer/include -fPIC -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wno-comment -fdiagnostics-color -ffunction-sections -fdata-sections -O3     -fno-exceptions -fno-rtti -UNDEBUG -std=c++14 -MD -MT tools/mlir/tools/polymer/lib/Target/CMakeFiles/obj.PolymerTargetOpenScop.dir/OpenScop/ConvertFromOpenScop.cc.o -MF tools/mlir/tools/polymer/lib/Target/CMakeFiles/obj.PolymerTargetOpenScop.dir/OpenScop/ConvertFromOpenScop.cc.o.d -o tools/mlir/tools/polymer/lib/Target/CMakeFiles/obj.PolymerTargetOpenScop.dir/OpenScop/ConvertFromOpenScop.cc.o -c /home/geek/py/base/Polygeist-polymer/mlir/tools/polymer/lib/Target/OpenScop/ConvertFromOpenScop.cc
/home/geek/py/base/Polygeist-polymer/mlir/tools/polymer/lib/Target/OpenScop/ConvertFromOpenScop.cc:31:10: fatal error: mlir/Dialect/Arithmetic/IR/Arithmetic.h: No such file or directory
 #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
[11/33] Building CXX object tools/mlir/tools/polymer/lib/Transforms/CMakeFiles/obj.PolymerTransforms.dir/Reg2Mem.cc.o
FAILED: tools/mlir/tools/polymer/lib/Transforms/CMakeFiles/obj.PolymerTransforms.dir/Reg2Mem.cc.o 
/usr/bin/c++  -DGTEST_HAS_RTTI=0 -DMLIR_CUDA_CONVERSIONS_ENABLED=0 -DMLIR_INCLUDE_TESTS -DMLIR_ROCM_CONVERSIONS_ENABLED=0 -D_DEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Itools/mlir/tools/polymer/lib/Transforms -I/home/geek/py/base/Polygeist-polymer/mlir/tools/polymer/lib/Transforms -Iinclude -I/home/geek/py/base/Polygeist-polymer/llvm/include -I/home/geek/py/base/Polygeist-polymer/mlir/include -Itools/mlir/include -Itools/mlir/tools/polymer/pluto/include -I/home/geek/py/base/Polygeist-polymer/mlir/tools/polymer/include -Itools/mlir/tools/polymer/include -fPIC -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wno-comment -fdiagnostics-color -ffunction-sections -fdata-sections -O3     -fno-exceptions -fno-rtti -UNDEBUG -std=c++14 -MD -MT tools/mlir/tools/polymer/lib/Transforms/CMakeFiles/obj.PolymerTransforms.dir/Reg2Mem.cc.o -MF tools/mlir/tools/polymer/lib/Transforms/CMakeFiles/obj.PolymerTransforms.dir/Reg2Mem.cc.o.d -o tools/mlir/tools/polymer/lib/Transforms/CMakeFiles/obj.PolymerTransforms.dir/Reg2Mem.cc.o -c /home/geek/py/base/Polygeist-polymer/mlir/tools/polymer/lib/Transforms/Reg2Mem.cc
/home/geek/py/base/Polygeist-polymer/mlir/tools/polymer/lib/Transforms/Reg2Mem.cc:15:10: fatal error: mlir/Dialect/Arithmetic/IR/Arithmetic.h: No such file or directory
 #include "mlir/Dialect/Arithmetic/IR/Arithmetic.h"
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

Which version of Polygeist should I compile against of?
``

[wsmoses]

CI appears to use daeb952678b1d46c867c94ed2f20d78e1a3d5ec5

That said @kumasento we should probably rebase this onto polygeist/mlir main.

[kumasento]

Thanks @geexie and @wsmoses . Sorry for the inconvenience.

I've populated the commit ID from what the CI uses. Hope this will help.

[geexie]

@wsmoses, @kumasento thanks for the link! do you actually build Polygeist to get Polymer working? On you CI you just build LLVM and then Polymer. I'm a bit confused about relation ship of those 2 projects and repos (Polygeist and Polymer)

[kumasento]

@wsmoses, @kumasento thanks for the link! do you actually build Polygeist to get Polymer working? On you CI you just build LLVM and then Polymer. I'm a bit confused about relation ship of those 2 projects and repos (Polygeist and Polymer)

Thanks for the Q. It depends on what your input is. If it is MLIR, then just building Polymer + LLVM is fine. But if you need to pass in C/C++ code, you need Polygeist to translate them into MLIR first.

Hope this helps.

[wsmoses]

Generalizing the answer from Ruizhe a bit further:

Polygeist is the name of the overall project as well as the repo for the frontend and raising pipelines (Eg takes C/C++, and also converts cfg to loops, loops to affine, etc).

This repo is a component of the overall framework that does the bidirectional conversion to/from openscop.

[geexie]

@wsmoses, thanks for your comment. This means that if I want just go from c++ to affine the Polygeist is perfect for this task and if I want a full pipeline with conversion and polyhedral tuning let's say run some c++ workload I need to use Polygeist to generate MLIR and than pass this MLIR to Polymer in 2 separate commands

[wsmoses]

Correct!

[paldebjit]

@kumasento @wsmoses I was trying to follow the steps in CI. I got the following error:

[dp638@zhang-26 build]$ cat /work/zhang-x1/users/dp638/Tools/MLIR_Tools/polymer/build/CMakeFiles/CMakeError.log
Performing C++ SOURCE FILE Test CXX_SUPPORTS_CUSTOM_LINKER failed with the following output:
Change Dir: /work/zhang-x1/users/dp638/Tools/MLIR_Tools/polymer/build/CMakeFiles/CMakeTmp

Run Build Command(s):/work/zhang-x1/users/dp638/Tools/Ninja/ninja cmTC_30574 && [1/2] Building CXX object CMakeFiles/cmTC_30574.dir/src.cxx.o
clang-14: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument]
[2/2] Linking CXX executable cmTC_30574
FAILED: cmTC_30574 
: && /work/zhang-x1/users/dp638/Tools/MLIR_Tools/Polygeist/llvm-project/install/bin/clang++ -fuse-ld=lld  CMakeFiles/cmTC_30574.dir/src.cxx.o -o 
cmTC_30574   && :
clang-14: error: invalid linker name in argument '-fuse-ld=lld'
ninja: build stopped: subcommand failed.

Source file was:
int main() { return 0; }

I have built llvm-project inside Polygeist at the commit daeb952678b1d46c867c94ed2f20d78e1a3d5ec5. Any suggestion what could be causing this? My gcc version is the following: gcc version 8.3.1 20190311 (Red Hat 8.3.1-3) (GCC).

[kumasento]

Based on the error message: It seems to me that you haven't installed lld.

You could try to install one through package manager, or build from scratch after removing this line: https://github.com/kumasento/polymer/blob/17f40fcadae4b1932aab04e7bb6e315b54dd4924/.github/workflows/buildAndTest.yml#L114

On Thu, 27 Jan 2022 at 00:24, Debjit Pal @.***> wrote:

@kumasento https://github.com/kumasento @wsmoses https://github.com/wsmoses I was trying to follow the steps in CI. I got the following error:

@.*** build]$ cat /work/zhang-x1/users/dp638/Tools/MLIR_Tools/polymer/build/CMakeFiles/CMakeError.log Performing C++ SOURCE FILE Test CXX_SUPPORTS_CUSTOM_LINKER failed with the following output: Change Dir: /work/zhang-x1/users/dp638/Tools/MLIR_Tools/polymer/build/CMakeFiles/CMakeTmp

Run Build Command(s):/work/zhang-x1/users/dp638/Tools/Ninja/ninja cmTC_30574 && [1/2] Building CXX object CMakeFiles/cmTC_30574.dir/src.cxx.o clang-14: warning: argument unused during compilation: '-fuse-ld=lld' [-Wunused-command-line-argument] [2/2] Linking CXX executable cmTC_30574 FAILED: cmTC_30574 : && /work/zhang-x1/users/dp638/Tools/MLIR_Tools/Polygeist/llvm-project/install/bin/clang++ -fuse-ld=lld CMakeFiles/cmTC_30574.dir/src.cxx.o -o cmTC_30574 && : clang-14: error: invalid linker name in argument '-fuse-ld=lld' ninja: build stopped: subcommand failed.

Source file was: int main() { return 0; }

I have built llvm-project inside Polygeist at the commit daeb952678b1d46c867c94ed2f20d78e1a3d5ec5. Any suggestion what could be causing this? My gcc version is the following: gcc version 8.3.1 20190311 (Red Hat 8.3.1-3) (GCC).

— Reply to this email directly, view it on GitHub https://github.com/kumasento/polymer/issues/122#issuecomment-1022727001, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACC42R3MOXBVSK7KYDPFTKDUYCGEBANCNFSM5LZXYVRQ . You are receiving this because you were mentioned.Message ID: @.***>

[paldebjit]

@kumasento Here is a screenshot of my setup. It matches the CI setup (checkout-wise). However, polymer compilation still fails. I would appreciate your suggestions. Thanks once again.

[kumasento]

Right, it seems that the LLVM/MLIR version you're using is more recent and some APIs have been changed.

I will make some updates later.

On Thu, 27 Jan 2022 at 22:09, Debjit Pal @.***> wrote:

@kumasento https://github.com/kumasento Here is a screenshot of my setup. It matches the CI setup (checkout-wise). However, polymer compilation still fails. I would appreciate your suggestions. Thanks once again. [image: Screenshot from 2022-01-27 17-06-24] https://user-images.githubusercontent.com/49922297/151451898-850f52c0-1804-4871-86d0-166c6b52ea8e.png

— Reply to this email directly, view it on GitHub https://github.com/kumasento/polymer/issues/122#issuecomment-1023682892, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACC42RYBDRD6VXVBU7BGWKTUYG7BVANCNFSM5LZXYVRQ . You are receiving this because you were mentioned.Message ID: @.***>

[paldebjit]

I am trying Polygeist daeb952678b1d46c867c94ed2f20d78e1a3d5ec5and also trying to checkout the LLVM that is linked to that checkout of Polygeist. I will post my findings soon.

[kumasento]

125 should do the trick

[wsmoses]

The lld issue is weird and shouldn't occur regardless. That in particular feels like some messed up cmake cache/config? In any case once #125 lands try a clean build

[paldebjit]

@kumasento I was able to build polymer-opt and polymer-translate using the daeb952678b1d46c867c94ed2f20d78e1a3d5ec5 commit. I will try #125 later as I guess this build will meet my need. I have one more question for you.

The polymer-opt has a pass --extract-scop-stmt. In my understanding that is needed to transform the optimized Pluto OpenSCOP to a MLIR function format. However, what I want is to extract the OpenSCOP for a given MLIR code in (Affine Dialect for the time being) which I guess is done by polymer-translate with --export-scop option. However, when I use polymer-translate --export-scop matmul.mlir, I get an error

polymer-translate --export-scop matmul.mlir 
matmul.mlir:2:8: error: custom op 'memref.alloc' is unknown
  %A = memref.alloc() : memref<64x64xf32>

Maybe because matmul is not entirely in affine dialect? I would appreciate your suggestions in this regard.

The --extract-scop-stmt with --pluto-opt does spit out an OpenSCOP in the terminal but I guess that is post pluto optimization.

[kumasento]

Hi @paldebjit, I'm not quite sure about your question: are you trying to dump MLIR from Pluto-optimised (or original?) OpenScop, or generate OpenScop from an MLIR program?

In either case, polymer-translate should be the right tool for you.

• --import-scop translates OpenScop to MLIR
• --export-scop translates MLIR to OpenScop

However, polymer-translate hasn't been maintained for a while, and it could possibly be obsolete.

The error you saw is due to that polymer-translate is not updated for a while and the memref dialect is not registered by it yet. It should be an easy fix, and I would do that soon this weekend.

But if you're in a hurry, you could try printing it out to the terminal. The whole pluto transformation is in this function:

https://github.com/kumasento/polymer/blob/b2f5469ab41c4458b98ff8e96c5b617700b43f0f/lib/Transforms/PlutoTransform.cc#L72-L133

[paldebjit]

Hi @kumasento thanks. I was able to compile. The issue currently is on my side, my code base depends on LLVM-13 which does not have Arithmetic dialect and the mlir::FlatAffineValueConstraints whereas many of your code fragments do use Arith dialect and the that MLIR class. I am trying to backtrack and see which commit of your code I can reliably use along with my LLVM 13 based codebase. Thanks once again for your kind help. I appreciate it very mcuh.

[kumasento]

TBH I don't know if that is viable: changes in Polymer are not just version related, they may improve functionality as well.

The latest commit that doesn't use FlatAffineValueConstraint is 9a59b6b3ace91e4914f2532930a205678b1799fe.

I'm curious if there is a way to make a codebase compatible with multiple MLIR versions.