Port to LLVM 11 - Githubissues

anatol commented 4 years ago

I am trying to compile tinygo with LLVM 11 release and I get following compilation failure:

# tinygo.org/x/go-llvm
backports.cpp:10:28: error: redefinition of ‘llvm::PassManagerBuilder* llvm::unwrap(LLVMPassManagerBuilderRef)’
   10 | inline PassManagerBuilder *unwrap(LLVMPassManagerBuilderRef P) {
      |                            ^~~~~~
In file included from backports.cpp:4:
/usr/include/llvm/Transforms/IPO/PassManagerBuilder.h:255:28: note: ‘llvm::PassManagerBuilder* llvm::unwrap(LLVMPassManagerBuilderRef)’ previously defined here
  255 | inline PassManagerBuilder *unwrap(LLVMPassManagerBuilderRef P) {
      |                            ^~~~~~
backports.cpp:14:34: error: redefinition of ‘LLVMOpaquePassManagerBuilder* llvm::wrap(llvm::PassManagerBuilder*)’
   14 | inline LLVMPassManagerBuilderRef wrap(PassManagerBuilder *P) {
      |                                  ^~~~
In file included from backports.cpp:4:
/usr/include/llvm/Transforms/IPO/PassManagerBuilder.h:259:34: note: ‘LLVMOpaquePassManagerBuilder* llvm::wrap(llvm::PassManagerBuilder*)’ previously defined here
  259 | inline LLVMPassManagerBuilderRef wrap(PassManagerBuilder *P) {
      |                                  ^~~~
make: *** [Makefile:187: tinygo] Error 2
make: *** Waiting for unfinished jobs....

aykevl commented 4 years ago

As you've noticed, TinyGo does not yet support statically linking against LLVM 11. This will probably be supported in the near future, when LLVM 11 becomes more widely available.

Related to that, you can already use LLVM 11 when using prebuilt binaries with #1056 (which is not yet merged).

anatol commented 4 years ago

I am actually building tinygo package for Arch Linux that started moving towards LLVM 11. Is it the only patch that is needed for LLVM 11? https://github.com/tinygo-org/tinygo/pull/1056/commits/cc3ec220e79057a24f68b395e4d9f235ff0ca765 Or more work is needed?

aykevl commented 4 years ago

That depends on whether you want to link statically or dynamically. Most distributions prefer dynamic linking and that should work fine with https://github.com/tinygo-org/tinygo/commit/cc3ec220e79057a24f68b395e4d9f235ff0ca765.

anatol commented 4 years ago

Thank you for your help @aykevl.

I applied cc3ec220e79057a24f68b395e4d9f235ff0ca765 on top of 0.15.0 release and it gives me following build problem:

==> Installing missing dependencies...
resolving dependencies...
looking for conflicting packages...

Packages (5) compiler-rt-11.0.0-1  libedit-20191231_3.1-2  clang-11.0.0-1  lld-11.0.0-1  llvm-libs-11.0.0-1

Total Installed Size:  305.71 MiB

:: Proceed with installation? [Y/n] 
(5/5) checking keys in keyring           

.......

lib/cmsis-svd/data/Atmel/ATSAMR21E18A.svd
lib/cmsis-svd/data/Atmel/ATSAMR21E19A.svd
lib/cmsis-svd/data/Atmel/ATSAMR21G16A.svd
lib/cmsis-svd/data/Atmel/ATSAMR21G17A.svd
lib/cmsis-svd/data/Atmel/ATSAMR21G18A.svd
lib/cmsis-svd/data/Atmel/ATSAMV71J19.svd
lib/cmsis-svd/data/Atmel/ATSAMV71J19B.svd
lib/cmsis-svd/data/Atmel/ATSAMV71J20.svd
lib/cmsis-svd/data/Atmel/ATSAMV71J20B.svd
lib/cmsis-svd/data/Atmel/ATSAMV71J21.svd
lib/cmsis-svd/data/Atmel/ATSAMV71J21B.svd
lib/cmsis-svd/data/Atmel/ATSAMV71N19.svd
lib/cmsis-svd/data/Atmel/ATSAMV71N19B.svd
# tinygo.org/x/go-llvm
/usr/bin/ld: cannot find -lLLVM-10
collect2: error: ld returned 1 exit status
make: *** [Makefile:187: tinygo] Error 2
make: *** Waiting for unfinished jobs....
lib/cmsis-svd/data/Atmel/ATSAMV71N20.svd
lib/cmsis-svd/data/Atmel/ATSAMV71N20B.svd
lib/cmsis-svd/data/Atmel/ATSAMV71N21.svd
lib/cmsis-svd/data/Atmel/ATSAMV71N21B.svd
lib/cmsis-svd/data/Atmel/ATSAMV71Q19.svd
lib/cmsis-svd/data/Atmel/ATSAMV71Q19B.svd
lib/cmsis-svd/data/Atmel/ATSAMV71Q20.svd
lib/cmsis-svd/data/Atmel/ATSAMV71Q20B.svd
lib/cmsis-svd/data/Atmel/ATSAMV71Q21.svd
lib/cmsis-svd/data/Atmel/ATSAMV71Q21B.svd
GO111MODULE=off go fmt ./src/device/sam
src/device/sam/at91sam9g10.go
src/device/sam/at91sam9cn12.go
src/device/sam/at91sam9g20.go
src/device/sam/at91sam9cn11.go
src/device/sam/at91sam9g15.go

I do not see LLVM-10 anywhere in the checked out code, so it must be coming from tinygo.org/x/go-llvm dependency. Should this go library be linked with LLVM 11 instead?

deadprogram commented 4 years ago

The current dev branch now has this support merged. Please give a try from that branch.

aykevl commented 4 years ago

You will also need the llvm11 build tag to make this work (-tags=llvm11). Otherwise it will by default try to link to LLVM 10.

anatol commented 4 years ago

Hi @aykevl I am a bit confused now. The comments above state that LLVM11 support has been added in commit cc3ec22. But it turns out that some extra manual changes are required to the build process. Would it make sense to add this proposed -tags=llvm11 to the build system instead?

aykevl commented 4 years ago

We will likely switch the default from LLVM 10 to LLVM 11 in the near future, avoiding this issue. However, LLVM 11 may not have propagated everywhere so I'd rather wait a month or so before doing that.

To use the build tag:

go install -tags=llvm11

It appears that you're building LLVM from source instead of linking it dynamically, in which case using LLVM 11 is not yet supported (but that isn't necessary anyway as there is no need to match the distro LLVM version in this case).

anatol commented 4 years ago

I am actually trying not to build LLVM from sources an use llvm-11 from my system instead. I had to patch Makefile to disable building LLVM from sources, here is the patch https://github.com/archlinux/svntogit-community/blob/e0369803ed783f8f3f664d4a05fec999cdfb3c7f/trunk/disable_static_llvm.patch

This patch worked with LLVM-10, but does not work with LLVM-11. It look like I need extra patching.

Though I would personally prefer avoid patching sources. Is there a chance to make dynamic build configured by some build envvar instead? (the documentation only mentions static build https://github.com/tinygo-org/tinygo/blob/release/BUILDING.md)

anatol commented 4 years ago

By patching Makefile and adding -tags llvm11 to tinygo: build target I was able to compile the project with system LLVM11 library. The build seems fine. We are going to continue verification of tinygo+LLVM11 in Arch Linux testing repository. So my original question is resolved.

But having a simpler way to enable dynamic LLVM build without patching Makefile is still something desirable for us.

aykevl commented 4 years ago

Though I would personally prefer avoid patching sources. Is there a chance to make dynamic build configured by some build envvar instead? (the documentation only mentions static build https://github.com/tinygo-org/tinygo/blob/release/BUILDING.md)

The way to do that is to not use make, but instead call the Go toolchain directly. For example:

go build -tags=llvm11 -o build/tinygo
make gen-device

And to run the tests:

go test
make smoketest XTENSA=0

I've added XTENSA=0 above as the Xtensa backend (for esp32/esp8266) is not yet supported in upstream LLVM.

anatol commented 4 years ago

Could you please update the official build instructions https://github.com/tinygo-org/tinygo/blob/release/BUILDING.md ?? Once it is done I'll modify Arch package build as well.

aykevl commented 3 years ago

It's already more or less included in BUILDING.md:

TinyGo depends on LLVM and libclang, which are both big C++ libraries. It can also optionally use a built-in lld to ease cross compiling. There are two ways these can be linked: dynamically and statically. An install with go install is dynamic linking because it is fast and works almost out of the box on Debian-based systems with the right packages installed.

I'm not sure what you expect there to be? If you want a build with a dynamically linked tinygo binary, you can simply replace the tinygo binary in the build directory with the one produced by go build (or go install).

zonghaishang commented 3 years ago

Though I would personally prefer avoid patching sources. Is there a chance to make dynamic build configured by some build envvar instead? (the documentation only mentions static build https://github.com/tinygo-org/tinygo/blob/release/BUILDING.md)

The way to do that is to not use make, but instead call the Go toolchain directly. For example:
go build -tags=llvm11 -o build/tinygo
make gen-device
And to run the tests:
go test
make smoketest XTENSA=0
I've added XTENSA=0 above as the Xtensa backend (for esp32/esp8266) is not yet supported in upstream LLVM.

➜  tinygo git:(dev) go build -tags=llvm11 -o build/tinygo
# tinygo.org/x/go-llvm
../../../../pkg/mod/tinygo.org/x/go-llvm@v0.0.0-20200503224449-70c558526021/analysis.go:16:10: fatal error: 'llvm-c/Analysis.h' file not found
#include "llvm-c/Analysis.h" // If you are getting an error here read bindings/go/README.txt
         ^~~~~~~~~~~~~~~~~~~
1 error generated.
➜  tinygo git:(dev) brew install llvm
Updating Homebrew...
Warning: llvm 11.0.0 is already installed and up-to-date
To reinstall 11.0.0, run `brew reinstall llvm`

This has been resolved by compiling LLVM10

deadprogram commented 3 years ago

The just merged #1581 should address this in a simpler way. Marking this issue to be closed on next release.

deadprogram commented 3 years ago

This has been released with v0.17.0 so now closing. Thanks!

tinygo-org / tinygo

Port to LLVM 11 #1435