cmd/go: error: unrecognized command line option '-rdynamic' when cross-compiling to ARM with -pie

daenney commented 4 years ago

I'm not sure if this is supposed to work, i.e -buildmode pie for ARM, but nothing seems to suggest it shouldn't.

What version of Go are you using (`go version`)?

$ go version
go version go1.13.6 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/daenney/.cache/go-build"
GOENV="/home/daenney/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/data/Development/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/daenney/Applications/asdf/installs/golang/1.13.6/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/daenney/Applications/asdf/installs/golang/1.13.6/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build205825653=/tmp/go-build -gno-record-gcc-switches"

What did you do?

main.go:

package main

func main() {
        print("hello")
}

$ env GOOS=linux env GOARCH=arm go build -o test -buildmode pie

What did you expect to see?

No error, and a resulting test binary.

What did you see instead?

/home/daenney/Applications/asdf/installs/golang/1.13.6/go/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
gcc: error: unrecognized command line option ‘-marm’; did you mean ‘-mabm’?

The error also occurs if I add GOARM=7. It triggers on a combination of -buildmode pie and GOARCH=arm. Dropping -buildmode pie the build succeeds.

$ gcc --version
gcc (GCC) 9.2.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ uname -a
Linux monoceros 5.4.8-arch1-1 #1 SMP PREEMPT Sat, 04 Jan 2020 23:46:18 +0000 x86_64 GNU/Linux

$ pacman -Qq | grep gcc
arm-none-eabi-gcc
gcc
gcc-libs

ALTree commented 4 years ago

To cross-compile with pie, you need to set CC to a cross-compiler toolchain, e.g. CC=arm-none-eabi-gccin your case. Are you doing this?

daenney commented 4 years ago

I was not, thank you for pointing this out. However, the error now changed:

$ env GOOS=linux env GOARCH=arm env GOARM=7 env CC=arm-none-eabi-gcc go build -o test -buildmode pie

/home/daenney/Applications/asdf/installs/golang/1.13.6/go/pkg/tool/linux_amd64/link: running arm-none-eabi-gcc failed: exit status 1
arm-none-eabi-gcc: error: unrecognized command line option '-rdynamic'

ALTree commented 4 years ago

What is arm-none-eabi-gcc --version ?

daenney commented 4 years ago

Same as my GCC version, 9.2.0.

But I think I found the issue:

arm-none-eabi-gcc --help

  -pie                     Create a dynamically linked position independent
                           executable.

I can't find a -r flag for it, but I did find -pie. It appears that this might've changed between GCC versions?

daenney commented 4 years ago

Hmm, no, I'm misunderstanding things I think.

-rdynamic comes from https://github.com/golang/go/blob/master/src/cmd/link/internal/ld/lib.go#L1374-L1376 -pie does get passed, by https://github.com/golang/go/blob/master/src/cmd/link/internal/ld/lib.go#L1262-L1269

daenney commented 4 years ago

For completeness sake:

$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-pc-linux-gnu/9.2.0/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /build/gcc/src/gcc/configure --prefix=/usr --libdir=/usr/lib --libexecdir=/usr/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=https://bugs.archlinux.org/ --enable-languages=c,c++,ada,fortran,go,lto,objc,obj-c++,d --enable-shared --enable-threads=posix --with-system-zlib --with-isl --enable-__cxa_atexit --disable-libunwind-exceptions --enable-clocale=gnu --disable-libstdcxx-pch --disable-libssp --enable-gnu-unique-object --enable-linker-build-id --enable-lto --enable-plugin --enable-install-libiberty --with-linker-hash-style=gnu --enable-gnu-indirect-function --enable-multilib --disable-werror --enable-checking=release --enable-default-pie --enable-default-ssp --enable-cet=auto gdc_include_dir=/usr/include/dlang/gdc
Thread model: posix
gcc version 9.2.0 (GCC)

$ arm-none-eabi-gcc -v
Using built-in specs.
COLLECT_GCC=arm-none-eabi-gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/arm-none-eabi/9.2.0/lto-wrapper
Target: arm-none-eabi
Configured with: /build/arm-none-eabi-gcc/src/gcc-9.2.0/configure --target=arm-none-eabi --prefix=/usr --with-sysroot=/usr/arm-none-eabi --with-native-system-header-dir=/include --libexecdir=/usr/lib --enable-languages=c,c++ --enable-plugins --disable-decimal-float --disable-libffi --disable-libgomp --disable-libmudflap --disable-libquadmath --disable-libssp --disable-libstdcxx-pch --disable-nls --disable-shared --disable-threads --disable-tls --with-gnu-as --with-gnu-ld --with-system-zlib --with-newlib --with-headers=/usr/arm-none-eabi/include --with-python-dir=share/gcc-arm-none-eabi --with-gmp --with-mpfr --with-mpc --with-isl --with-libelf --enable-gnu-indirect-function --with-host-libstdcxx='-static-libgcc -Wl,-Bstatic,-lstdc++,-Bdynamic -lm' --with-pkgversion='Arch Repository' --with-bugurl=https://bugs.archlinux.org/ --with-multilib-list=rmprofile
Thread model: single
gcc version 9.2.0 (Arch Repository)

daenney commented 4 years ago

I can also trigger this using -buildmode plugin it turns out.

env GOOS=linux env GOARCH=arm env GOARM=7 env CC=arm-none-eabi-gcc go build -buildmode plugin

/home/daenney/Applications/asdf/installs/golang/1.13.6/go/pkg/tool/linux_amd64/link: running arm-none-eabi-gcc failed: exit status 1
arm-none-eabi-gcc: error: unrecognized command line option '-rdynamic'

ALTree commented 4 years ago

I'm a little confused. The error says that the arm-none-eabi-gcc invocation with -rdynamic is failing because -rdynamic is not a recognized option.

The gcc9.2 documentation says that -rdynamic is supported, so it hasn't been removed:

-rdynamic

Pass the flag -export-dynamic to the ELF linker, on targets that support it.

If you manually invoke arm-none-eabi-gcc with -rdynamic, does it fail?

daenney commented 4 years ago

It does, in my case:

$ arm-none-eabi-gcc -rdynamic
arm-none-eabi-gcc: error: unrecognized command line option '-rdynamic'
arm-none-eabi-gcc: fatal error: no input files
compilation terminated.

Interestingly, the Arch manpage for it does have the option: https://jlk.fjfi.cvut.cz/arch/manpages/man/gcc.1. The one thing I'm seeing is that it seems to need an invocation of <object-file-name> -rdynamic so I wonder if we're getting the order of argument wrong?

ALTree commented 4 years ago

Adding -x to the Go build command will make it print the exact gcc commands it is running, that may help with the debugging.

daenney commented 4 years ago

I might be adding the -x in the wrong place, but all I seem to get from it is:

/home/daenney/Applications/asdf/installs/golang/1.13.6/go/pkg/tool/linux_amd64/link -o $WORK/b001/exe/a.out -importcfg $WORK/b001/importcfg.link -installsuffix shared -buildmode=pie -buildid=R8nBSB93JaMvrM2xsyrO/1-WJp2sTmpzMqiDxJUbL/-svvvBDZWEBTMwJydl1o/R8nBSB93JaMvrM2xsyrO -extld=arm-none-eabi-gcc /home/daenney/.cache/go-build/74/74a893b4cd5c0830e9f773af20b07c81429188d5afbfccdbaa7878e9329c478f-d

ALTree commented 4 years ago

env GOOS=linux env GOARCH=arm go build -x [...] and if the executable test exists already delete it before running the command.

daenney commented 4 years ago

$  env GOOS=linux env GOARCH=arm env GOARM=7 env CC=arm-none-eabi-gcc go build -x -o test -buildmode pie main.go
WORK=/tmp/go-build048415045
mkdir -p $WORK/b001/
cat >$WORK/b001/importcfg.link << 'EOF' # internal
packagefile command-line-arguments=/home/daenney/.cache/go-build/74/74a893b4cd5c0830e9f773af20b07c81429188d5afbfccdbaa7878e9329c478f-d
packagefile runtime=/home/daenney/.cache/go-build/c6/c65897af6473754413f8930f1ade7692d71e24a176e78e4d87b0f47f1964f0c5-d
packagefile math=/home/daenney/.cache/go-build/8a/8ac1e42be9f20855149a4f57fc06b9cd09f4b87469660f3a2f195335ddb4768c-d
packagefile internal/bytealg=/home/daenney/.cache/go-build/e5/e553582227b9b61ee7eaa669bd5c753bdbaf897f700909a26e774ef9ad5052bc-d
packagefile internal/cpu=/home/daenney/.cache/go-build/be/be8b55e27bfd9a8d3e5c40ae0e3a61a5388019039863fb19d461c849a7bf98d4-d
packagefile runtime/internal/atomic=/home/daenney/.cache/go-build/de/de33b9513fa0572f490cc74c48514f43abf17617b4982cf5a63a5536f28b8d9b-d
packagefile runtime/internal/math=/home/daenney/.cache/go-build/74/742fe70977dc1066d498a6daa262f78fb011720c4758315a124012efc3346fd1-d
packagefile runtime/internal/sys=/home/daenney/.cache/go-build/e1/e10af778fbdcbbb256cf554e5c44ab6469cb3095575839116b62170e91154e65-d
packagefile math/bits=/home/daenney/.cache/go-build/73/73631338431639e3d4f0be7f7b8fcf9eb60e4d16c29fd352e4db41ba35150608-d
EOF
mkdir -p $WORK/b001/exe/
cd .
/home/daenney/Applications/asdf/installs/golang/1.13.6/go/pkg/tool/linux_amd64/link -o $WORK/b001/exe/a.out -importcfg $WORK/b001/importcfg.link -installsuffix shared -buildmode=pie -buildid=R8nBSB93JaMvrM2xsyrO/1-WJp2sTmpzMqiDxJUbL/-svvvBDZWEBTMwJydl1o/R8nBSB93JaMvrM2xsyrO -extld=arm-none-eabi-gcc /home/daenney/.cache/go-build/74/74a893b4cd5c0830e9f773af20b07c81429188d5afbfccdbaa7878e9329c478f-d
# command-line-arguments
/home/daenney/Applications/asdf/installs/golang/1.13.6/go/pkg/tool/linux_amd64/link: running arm-none-eabi-gcc failed: exit status 1
arm-none-eabi-gcc: error: unrecognized command line option '-rdynamic'

ianlancetaylor commented 4 years ago

Although -rdynamic is documented as being supported by GCC in general, in fact support for it is added on a target-specific basis. It looks like nobody has added support for -rdynamic for the arm-none-eabi target.

We could probably fix this in the Go linker by using -Wl,-export-dynamic instead of -rdynamic in cmd/link/internal/ld/lib.go, but I wonder if we should file a bug against GCC instead.

daenney commented 4 years ago

It might be worthwhile to consider both. Even if we file a bug against GCC and that gets fixed, it'll still cause problems for people on older GCC versions that do not have the patch. Some distros might backport it, but I doubt it'll be done consistently.

If you're willing to accept a patch, I'm happy to attempt the changes myself and submit it.

ianlancetaylor commented 4 years ago

My main concern with a patch is that on Solaris GCC does not always translate -rdynamic to -Wl,-export-dynamic. I think it is consistent on other ELF systems. But in principle a patch is OK.

Marcel2508 commented 4 years ago

Any updates or possible solutions for this?

ianlancetaylor commented 4 years ago

No updates. A possible patch is sketched out above. Anybody want to work on that? Thanks.

golang / go