search

bondhugula / pluto

Pluto: An automatic polyhedral parallelizer and locality optimizer
http://pluto-compiler.sourceforge.net
MIT License
268 stars 65 forks source link

What compiler can I use to compile pluto? #83

Closed thisiscam closed 2 years ago

thisiscam commented 2 years ago

Hi,

I was wondering what's the compiler requirement for compiling pluto? Does it have to be gcc, or I can use clang? Currently if I use clang to do the compilation, I get:

ld: library not found for -lgomp

If I'm understanding correctly, doesn't gomp only come with gcc? This linker flag appears in places such as: https://github.com/bondhugula/pluto/blob/master/Makefile.am#L58

bondhugula commented 2 years ago

Thanks for reporting this. pluto is just set up to build with GCC. The Github CI test also only tests build with GCC.

thisiscam commented 2 years ago

Thanks for the quick response!

I guess then I have to compile LLVM from source with gcc first..

bondhugula commented 2 years ago

LLVM and clang itself could be built with clang or gcc. But Pluto will have to be built with GCC.

On Sat, 30 Oct, 2021, 10:57 am Cambridge Yang, @.***> wrote:

Thanks for the quick response!

I guess then I have to compile LLVM from source with gcc first..

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bondhugula/pluto/issues/83#issuecomment-955151579, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABVPBELGGKUYOS4FQIMSQETUJN65VANCNFSM5HAUTX7A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

thisiscam commented 2 years ago

LLVM and clang itself could be built with clang or gcc. But Pluto will have to be built with GCC.

Sorry, I wasn't clear. The problem I ran into was that I cannot use a pre-built version of LLVM (e.g., one I get from brew) that is compiled with clang. This is because, as you said, I need to compile pluto with gcc, and I can't get gcc to link libpluto to a clang built LLVM.

After some minor tweaks, I actually just got pluto to successfully compile on macOS, with clang as the compiler.

I think there are two things that I did that mattered.

bondhugula commented 2 years ago

The piplib under candl should not be used at all. Candl and all components are configured to use the piplib submodule of Pluto.

Could you post if possible what errors you ran into when linking libpluto using clang/llvm libraries built with clang. Which version of clang are you using?

Good to know that you were able to get Pluto to build with clang as well. We can support both. If you have a patch, I can review and we can merge it. Or I'll try to support this myself in the next few days.

-Uday

On Sat, 30 Oct, 2021, 11:25 am Cambridge Yang, @.***> wrote:

LLVM and clang itself could be built with clang or gcc. But Pluto will have to be built with GCC.

Sorry, I wasn't clear. The problem I ran into was that I cannot use a pre-built version of LLVM (e.g., one I get from brew https://formulae.brew.sh/formula/llvm) that is compiled with clang. This is because, as you said, I need to compile pluto with gcc, and I can't get gcc to link libpluto to a clang built LLVM.

After some minor tweaks, I actually just got pluto to successfully compile on macOS, with clang as the compiler.

I think there are two things that I did that mattered.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bondhugula/pluto/issues/83#issuecomment-955154523, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABVPBEIGGHCEX6CDHVEDU7DUJOCDZANCNFSM5HAUTX7A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

thisiscam commented 2 years ago

Here's the duplicated symbol error:

Making all in tool
  CC       pluto-ast_transform.o
  CC       pluto-constraints_polylib.o
  CXX      pluto-main.o
  CC       pluto-osl_pluto.o
  CXX      pluto-pet_to_pluto.o
  CC       pluto-pluto_codegen_if.o
  CXXLD    pluto
duplicate symbol '_inptrf_xx' in:
    ../piplib/.libs/libpiplib_dp.a(libpiplib_dp_la-piplib.o)
    /Users/xxx/xxx/xxx/pluto/candl/piplib/.libs/libpiplib_gmp.a(libpiplib_gmp_la-piplib.o)
ld: 1 duplicate symbol for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Some info to help reproduce a successful build

Some configurations other than the two tweaks I mentioned above:

I was using Apple's built-in clang as CC (aliased to gcc):

$ gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
Apple clang version 13.0.0 (clang-1300.0.29.3)
Target: x86_64-apple-darwin20.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

I was using LLVM@11 from homebrew (which also happens to have FileCheck):

$ brew install llvm@11
$ brew info llvm@11 # Do all the environment variable exports suggested in the caveats

In particular, the latest LLVM 13 from homebrew does not work, which I got this error during compilation:

pet.cc:1028:40: error: too few arguments to function call, expected at least 4, have 3
                                            LangStandard::lang_unspecified);
                                                                          ^
/usr/local/opt/llvm/include/clang/Frontend/CompilerInvocation.h:226:3: note: 'setLangDefaults' declared here
  setLangDefaults(LangOptions &Opts, InputKind IK, const llvm::Triple &T,
  ^

I was using bison==3.8.2, texinfo== 6.8, libtool==2.4.6 from homebrew:

$ brew install bison
$ brew info bisin # Do all the environment variable exports suggested in the caveats
$ brew install texinfo
$ brew info texinfo # Do all the environment variable exports suggested in the caveats
$ brew install libtool
$ brew info libtool # Export the PATH

Then, I also had to add -Wno-implicit-function-declaration to CFLAGS, which is the same issue as, for example, https://github.com/inducer/islpy/issues/39.

Finally, I have to add $(CPPFLAGS) to the two rules for test_libpluto and unit_tests https://github.com/bondhugula/pluto/blob/master/Makefile.am#L57 (no idea why these use CPPFLAGS at all -- aren't these all C sources?), to ensure make test passes. Otherwise, I get not found (used by ./lib/math_support.h).

thisiscam commented 2 years ago

I removed all occurrences of -lgomp from Makefile.ams.

BTW, I think this is ok because pluto is not using openmp anyways (https://github.com/bondhugula/pluto/issues/73)?

bondhugula commented 2 years ago

Here's the duplicated symbol error:

Making all in tool
  CC       pluto-ast_transform.o
  CC       pluto-constraints_polylib.o
  CXX      pluto-main.o
  CC       pluto-osl_pluto.o
  CXX      pluto-pet_to_pluto.o
  CC       pluto-pluto_codegen_if.o
  CXXLD    pluto
duplicate symbol '_inptrf_xx' in:
    ../piplib/.libs/libpiplib_dp.a(libpiplib_dp_la-piplib.o)
    /Users/xxx/xxx/xxx/pluto/candl/piplib/.libs/libpiplib_gmp.a(libpiplib_gmp_la-piplib.o)
ld: 1 duplicate symbol for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

This is strange. The configure includes the following snippet and so candl/piplib shouldn't have been even looked at unless someone reran configure without any options inside of piplib.

dnl Configuring Candl-osl
dnl can't enable the gmp version without modifying
dnl the precision the is used elsewhere in pluto
echo ""
echo "=========================="
echo "Configuring Candl"
echo "=========================="
configureopts="--enable-llint-version \
--prefix=$prefix \
--with-piplib=build \
--with-piplib-builddir=../piplib \
--with-osl=build \
--with-osl-builddir=../openscop"
(mkdir -p candl
 cd candl/
 $REL_SRCDIR_PREFIX/candl/configure ${configureopts} ${archopts}
 )

In particular, things should work even after a rm -rf candl/piplib.

thisiscam commented 2 years ago

I tried again on a fresh repo, and I'm seeing 

Making all in candl
make[2]: *** No rule to make target `all'.  Stop.
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

I now remembered -- I overcame this by cd candl and manually invoke ./configure. I guess this is the issue.

Any idea of how to fix the above error? Seems like the same problem as: #79 (which was closed for some reason, and the suggested answer about texinfo doesn't fix the problem)

thisiscam commented 2 years ago

Ah hah, I think I found the problem:

==========================
Configuring Candl
==========================
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
.
.
.
checking for __gmpz_init in -lgmp... yes
checking for gmp.h... no
configure: error: "Can't find gmp headers."

Is it possible that --with-gmp-prefix is not propagated into candl's configure?

thisiscam commented 2 years ago

85 fixes this problem.

The only missing thing that enables a complete build on macOS is really just the removal of all -lgomp now.

bondhugula commented 2 years ago

Okay. That explains it. The automake/autogen in candl most likely failed due to another issue. You'll have to look at the log of the top-level configure as to why it failed generating the build conf.

-Uday

On Sun, 31 Oct, 2021, 11:42 am Cambridge Yang, @.***> wrote:

I tried again on a fresh repo, and I'm seeing

Making all in candl make[2]: No rule to make target `all'. Stop. make[1]: [all-recursive] Error 1 make: *** [all] Error 2

I now remembered -- I overcame this by cd candl and manually invoke ./configure. I guess this is the issue.

Any idea of how to fix the above error? Seems like the same problem as:

79 https://github.com/bondhugula/pluto/issues/79 (which was closed for

some reason, and the the suggested answer about texinfo doesn't fix the problem)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bondhugula/pluto/issues/83#issuecomment-955643599, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABVPBEOA2I2DF7MSWQ64BU3UJTM6DANCNFSM5HAUTX7A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

bondhugula commented 2 years ago

I just discovered pluto.h is missing an include for stdint.h which would make the C build for test_libpluto fail. The CI was failing as well but falsely reporting success. This is also fixed in https://github.com/bondhugula/pluto/pull/86

thisiscam commented 2 years ago

The automake/autogen in candl most likely failed due to another issue.

To clarify, I think this is just the above issue with the missing GMP_PREFIX option to candl?

To summarize, I believe what needs to be done for a macOS build are:

bondhugula commented 2 years ago

All of the things you list above sound good. We don't use openmp in Pluto and so it's fine to drop it. It can be later added properly if needed in a way that supports both gcc and clang. If you have a patch and are able to test it on MacOS, I'll be happy to review and merge.