search

bscarlet / llvm-general

Rich LLVM bindings for Haskell (with transfer of LLVM IR to and from C++, detailed compilation pass control, etc.)
http://hackage.haskell.org/package/llvm-general
132 stars 38 forks source link

Build problems on OSX #169

Closed ghost closed 8 years ago

ghost commented 8 years ago

I am trying to use the llvm-3.8 branch with llvm38 from homebrew-versions with stack (lts-6.2), ghc 7.10.2 (ghc 8.0.1 did not work).

I get the following build error:

llvm-general-3.8.0.0: build
Progress: 1/2
--  While building package llvm-general-3.8.0.0 using:
      /Users/Shared/mylang/llvm-general/llvm-general/.stack-work/dist/x86_64-osx/Cabal-1.22.5.0/setup/setup --builddir=.stack-work/dist/x86_64-osx/Cabal-1.22.5.0 build lib:llvm-general --ghc-options " -ddump-hi -ddump-to-file"
    Process exited with code: ExitFailure 1
    Logs have been written to: /Users/Shared/mylang/.stack-work/logs/llvm-general-3.8.0.0.log

    Preprocessing library llvm-general-3.8.0.0...
    error: invalid argument '-std=c++11' not allowed with 'C/ObjC'
    compiling .stack-work/dist/x86_64-osx/Cabal-1.22.5.0/build/LLVM/General/Internal/LibraryFunction_hsc_make.c failed (exit code 1)
    command was: /usr/bin/gcc -c .stack-work/dist/x86_64-osx/Cabal-1.22.5.0/build/LLVM/General/Internal/LibraryFunction_hsc_make.c -o .stack-work/dist/x86_64-osx/Cabal-1.22.5.0/build/LLVM/General/Internal/LibraryFunction_hsc_make.o -m64 -fno-stack-protector -D__GLASGOW_HASKELL__=710 -Ddarwin_BUILD_OS=1 -Dx86_64_BUILD_ARCH=1 -Ddarwin_HOST_OS=1 -Dx86_64_HOST_ARCH=1 -Isrc -I/usr/local/Cellar/llvm38/3.8.0/lib/llvm-3.8/include -I/usr/local/Cellar/llvm38/3.8.0/lib/llvm-3.8/include -D_DEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -O3 -std=c++11 -fvisibility-inlines-hidden -fno-exceptions -fno-common -Wcast-qual -I.stack-work/dist/x86_64-osx/Cabal-1.22.5.0/build/autogen -include .stack-work/dist/x86_64-osx/Cabal-1.22.5.0/build/autogen/cabal_macros.h -I/Users/myuser/.stack/programs/x86_64-osx/ghc-7.10.3/lib/ghc-7.10.3/unix_KZL8h98IqDM57kQSPo1mKx/include -I/Users/myuser/.stack/programs/x86_64-osx/ghc-7.10.3/lib/ghc-7.10.3/time_FTheb6LSxyX1UABIbBXRfn/include -I/Users/myuser/.stack/programs/x86_64-osx/ghc-7.10.3/lib/ghc-7.10.3/bytes_6VWy06pWzJq9evDvK2d4w6/include -I/Users/myuser/.stack/programs/x86_64-osx/ghc-7.10.3/lib/ghc-7.10.3/base_HQfYBxpPvuw8OunzQu6JGM/include -I/Users/myuser/.stack/programs/x86_64-osx/ghc-7.10.3/lib/ghc-7.10.3/integ_2aU3IZNMF9a7mQ0OzsZ0dS/include -I/Users/myuser/.stack/programs/x86_64-osx/ghc-7.10.3/lib/ghc-7.10.3/include -I/Users/myuser/.stack/programs/x86_64-osx/ghc-7.10.3/lib/ghc-7.10.3/include/

I am not sure whether this is a problem with llvm-general or my setup, does anyone know what the problem is?

bscarlet commented 8 years ago

llvm-general has glue code written in C++ which needs the -std=c++11 flag, and .hsc files which produce .c files. Cabal makes it hard to specify different flags for the two cases. Your gcc is complaining that it's been invoked with the -std=c++11 flag (which only makes sense for C++), but it's inferred that it's building C from the filename of the hsc-generated .c file, in this case. For some versions of gcc that complaint is a warning, for others (possibly for the OS-X gcc which is really clang) it's an error. You're getting the error.

There's code in the llvm-general Setup.hs file which is supposed to filter that flag when hsc2hs is run, but it's apparently not working for you. The problem may be that Cabal has changed, breaking that code. The plumbing to control the C++ flags for specific cases in Cabal is quite awful. I believe I most recently upgraded my version of Cabal from 1.22.2 to 1.22.4, but you're using 1.22.5. You might try downgrading your Cabal to see if the problem goes away just to narrow the problem down, or you might try just digging into the Cabal code used by Setup.hs to figure out why the -std=c++11 flag hasn't been filtered out as it should.

ghost commented 8 years ago

I tried to remove all occurrences of c++11 (cc-flags in .cabal and in Setup.hs) but still got the error. Maybe this flag comes from llvm-config and is used for the c files too:

llvm-config-3.8 --cxxflags
-I/usr/local/Cellar/llvm38/3.8.0/lib/llvm-3.8/include  -D_DEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -O3  -std=c++11 -fvisibility-inlines-hidden -fno-exceptions -fno-common -Wcast-qual
cocreature commented 8 years ago

This sounds like it might have been caused by https://github.com/bscarlet/llvm-general/commit/045a1a1c6d3f5c0d3bae138b5648e16ec04698b5 which is definitely a problematic change since you can’t assume that the flags used to build LLVM make sense for llvm-general. They might even use a different compiler.

bscarlet commented 8 years ago

maun: Yes, it comes from llvm-config.

cocreature: However unpleasant you may find it, LLVM requires that C++ code which uses it be configured with their special flags, and llvm-general can't do what it does without such C++ code. The LLVM C API is not sufficient. Thus building LLVM with one C++ compiler and llvm-general with another is unsupported.

Please see my previous comment for more details about what's going wrong. The code in Setup.hs to remove the flag isn't working, either because it's broken or because Cabal changed.

cocreature commented 8 years ago

@bscarlet Sorry I should have been more precise. I know that LLVM requires you to use special flags. The problem I was trying to get at is that llvm-config --cxxflags does show you the flags used to compile LLVM which is a superset of the flags required. So simply using all flags is too conservative. An example of a problematic flag is -Wno-switch-default/-Wcovered-switch-default. The former is the clang flag while the latter is the gcc flag. That can break the build of llvm-general even if that flag is completely unnecessary and linking against LLVM build with clang while compiling with gcc is completely fine. See also the corresponding mailing list thread. The solution would be to filter the flags returned by llvm-config to the necessary ones.

That said this is probably not what’s causing the issue here I should have looked more closely.

ghost commented 8 years ago

llvm-config --cflags also exits, maybe it can help you.

llvm-config-3.8 --cxxflags
-I/usr/local/Cellar/llvm38/3.8.0/lib/llvm-3.8/include  -D_DEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -O3  -std=c++11 -fvisibility-inlines-hidden -fno-exceptions -fno-common -Wcast-qual

llvm-config-3.8 --cflags
-I/usr/local/Cellar/llvm38/3.8.0/lib/llvm-3.8/include  -D_DEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -O3  -fno-common
lialan commented 8 years ago

Seems to me that in the Setup.hs, there is a line for setting ccOptions with "-std=c++11". The option should be set to cppOptions rather than ccOptions as to only apply to preprocessing cpp files?

Look for (cc-options/cpp-options) in: https://www.haskell.org/cabal/users-guide/developing-packages.html

lialan commented 8 years ago

Ahh I think Cabal will merge ccOptions and cppOptions before sending to hsc2hs...

bscarlet commented 8 years ago

I think that's options for the preprocessor, not c++.

bscarlet commented 8 years ago

Are you still having this problem? I have been unable to reproduce the behavior, where cabal passes the -std=c++11 flag through when building the c files, with cabal 1.24.

With cabal 1.22.5 I run into other weird build problems. Since those problems seem to be peculiar to that older version of cabal, I'll only dig into it more if can still see the problem.

ghost commented 8 years ago

I am currently not working on this and do not have the same environment anymore. When it works for you it should be fine, thanks for your help.

cocreature commented 8 years ago

Maybe there is a different issue on the LLVM-3.8 branch (not sure what you tried) but I am pretty sure it is not fixed there. Matthew Pickering encountered this when he tried building llvm-general on macOS a few days ago and on Linux I am also seeing hsc2hs warnings because it is getting flags intended for C++. I’ll try to take a more detailed look in the next few days.

bscarlet commented 8 years ago

Please let me know

(<os>, <branch|tag|git-revision>, <cabal-version>, <ghc-version>, <c-compiler-version>)

tuples for which the c-compiler gets passed the -std=c++11 flag when compiling the hsc2hs intermediate c-files, whether that compilation fails or not. The Setup.hs code is supposed to keep that flag (necessary for the real c++ files) from getting passed through for that c-code, and I know it's happened sometimes, but right now I can't reproduce it with the combinations I can easily test.

typedrat commented 8 years ago

I can confirm that it's happening on OS X 10.11.5 with the llvm-3.8 branch using Cabal 1.22.5 and GHC 7.10.3 or Cabal 1.24.0 and GHC 8.0.1 and with clang 703.0.31 as the C compiler.

cocreature commented 8 years ago

GCC only outputs a warning for these flags so the build works for me.

cocreature commented 8 years ago

Alright I looked into it. For me the problem is that std=c++11 appears twice in the flags and \\ removes only the first occurence. This is easily fixed by removing the cc-options line from the cabal file, we are setting that option in Setup.hs already. I am actually not quite sure why this works because it looks like we should be overwriting the cc-options completely in Setup.hs but apparently we aren’t.

However this is not enough. Since we started adding all flags by default that llvm-config provides we also need to filter more of these. This includes at least -Wdelete-non-virtual-dtor and -fvisibility-inlines-hidden but possibly also others. What we should really be doing is setting the options for hsc2hs to the result of llvm-config --cflags.

typedrat commented 8 years ago

I can confirm that this fixes it for me.

tmcdonell commented 8 years ago

Can we have https://github.com/bscarlet/llvm-general/pull/188/commits/3c172712ee519ef07b8f3526c624a5af036b746e cherry-picked onto the llvm-3.8 branch as well please?