Allow to statically link Go programs, even with cgo

[probonopd]

Use zig as a cc substitute to statically link Go programs, even with cgo, using musl libc. Doing it this way has the advantage that no Docker container or chroots are needed for building multiple architectures.

Continuation of https://github.com/probonopd/go-appimage/pull/203

[probonopd]

Looks like a couple of AppImages get built successfully, but then:

# github.com/probonopd/go-appimage/src/appimaged
loadinternal: cannot find runtime/cgo
/opt/hostedtoolcache/go/1.17.9/x64/pkg/tool/linux_amd64/link: running /usr/local/musl/bin/musl-gcc failed: exit status 1
/usr/bin/ld: i386 architecture of input file `/tmp/go-link-752300456/go.o' is incompatible with i386:x86-64 output
collect2: error: ld returned 1 exit status

Seems like there are some leftovers from earlier runs?

[probonopd]

So, we can build static binaries but currently only for the host architecture. Any help appreciated.

[probonopd]

Possibly use https://dev.to/kristoff/zig-makes-go-cross-compilation-just-work-29ho Thanks @dominikh for the hint and @kristoff-it for the article.

Looks like it can be used with musl libc: https://github.com/ziglang/zig/issues/514#issuecomment-472451736

[kristoff-it]

You can use this approach to deal with the outer directory from the Zig tarball:

https://github.com/ziglang/www.ziglang.org/blob/master/.github/workflows/build.yml#L30

[probonopd]

Why am I getting

+ echo ARCH: x86_64
+ echo GOARCH: amd64
+ echo GOHOSTARCH:
+ echo BUILDARCH: amd64
+ echo GOGCCFLAGS:
+ echo CC: zig cc -target x86_64-linux-musl
+ which zig
+ zig env
+ CGO_ENABLED=1
+ go build --tags extended /home/runner/work/go-appimage/go-appimage/src/appimagetool
# runtime/cgo
ge: zig [command] [options]

Commands:

  build            Build project from build.zig
(...)
error: unknown command: -E
Error: Process completed with exit code 2.

[kristoff-it]

Which version of Go are you using?

[probonopd]

Updated from Go 1.17 to 1.18. Now getting

+ go build --tags extended /home/runner/work/go-appimage/go-appimage/src/appimagetool
# github.com/probonopd/go-appimage/src/appimagetool
runtime.gcdata: missing Go type information for global symbol .dynsym: size 72
net(.text): relocation target __errno_location not defined
net(.text): relocation target getaddrinfo not defined
net(.text): relocation target freeaddrinfo not defined
net(.text): relocation target gai_strerror not defined
os/user(.text): relocation target malloc not defined
os/user(.text): relocation target free not defined
os/user(.text): relocation target getgrgid_r not defined
os/user(.text): relocation target getgrnam_r not defined
os/user(.text): relocation target getpwnam_r not defined
os/user(.text): relocation target getpwuid_r not defined
os/user(.text): relocation target realloc not defined
os/user(.text): relocation target sysconf not defined
runtime/cgo(.text): relocation target stderr not defined
runtime/cgo(.text): relocation target fwrite not defined
runtime/cgo(.text): relocation target vfprintf not defined
runtime/cgo(.text): relocation target fputc not defined
runtime/cgo(.text): relocation target abort not defined
runtime/cgo(.text): relocation target pthread_create not defined
runtime/cgo(.text): relocation target nanosleep not defined
runtime/cgo(.text): relocation target pthread_detach not defined
/opt/hostedtoolcache/go/1.18.1/x64/pkg/tool/linux_amd64/link: too many errors

[kristoff-it]

It's a bug in the Go toolchain, add -Wl,--no-gc-sections to the zig cc invocation as a workaround. Related issue: https://github.com/golang/go/issues/52690

[kristoff-it]

Although I am not 100% sure what is supposed to happen here since you're linking statically with musl. Maybe instructing Go to use Zig as the linker is the right approach. Maybe @andrewrk knows more.

[probonopd]

@kristoff-it if this all works, it is pure genius, the best thing since sliced br... musl libc and Go :+1:

[kristoff-it]

Are you sure you need musleabi instead of just musl for aarch64?

[probonopd]

For this arch we error out.

CC: zig cc -target aarch64-linux-musleabi
+ zig env
/usr/local/bin/zig
{
 "zig_exe": "/usr/local/bin/zig",
 "lib_dir": "/usr/local/bin/lib",
 "std_dir": "/usr/local/bin/lib/std",
 "global_cache_dir": "/home/runner/.cache/zig",
 "version": "0.10.0-dev.2112+0df28f9d4"
}
+ CGO_ENABLED=1
+ go build -ldflags '-v -linkmode=external' --tags extended /home/runner/work/go-appimage/go-appimage/src/appimaged
# runtime/cgo
_cgo_export.c:3:10: fatal error: 'stdlib.h' file not found
#include <stdlib.h>
         ^~~~~~~~~~
1 error generated.
Error: Process completed with exit code 2.

[probonopd]

Are you sure you need musleabi instead of just musl for aarch64?

No, it was a wild guess.

[kristoff-it]

You probably do need to use a different musl version for 32bit arm though. I'm not familiar enough but IIRC you need to know a bit more about your target. @MasterQ32 where can one find a concise guide that explains how to select the correct ABI for arm?

[andrewrk]

zig targets | jq .libc:

[
  "aarch64_be-linux-gnu",
  "aarch64_be-linux-musl",
  "aarch64_be-windows-gnu",
  "aarch64-linux-gnu",
  "aarch64-linux-musl",
  "aarch64-windows-gnu",
  "aarch64-macos-gnu",
  "aarch64-macos-gnu",
  "armeb-linux-gnueabi",
  "armeb-linux-gnueabihf",
  "armeb-linux-musleabi",
  "armeb-linux-musleabihf",
  "armeb-windows-gnu",
  "arm-linux-gnueabi",
  "arm-linux-gnueabihf",
  "arm-linux-musleabi",
  "arm-linux-musleabihf",
  "thumb-linux-gnueabi",
  "thumb-linux-gnueabihf",
  "thumb-linux-musleabi",
  "thumb-linux-musleabihf",
  "arm-windows-gnu",
  "csky-linux-gnueabi",
  "csky-linux-gnueabihf",
  "i386-linux-gnu",
  "i386-linux-musl",
  "i386-windows-gnu",
  "m68k-linux-gnu",
  "m68k-linux-musl",
  "mips64el-linux-gnuabi64",
  "mips64el-linux-gnuabin32",
  "mips64el-linux-musl",
  "mips64-linux-gnuabi64",
  "mips64-linux-gnuabin32",
  "mips64-linux-musl",
  "mipsel-linux-gnueabi",
  "mipsel-linux-gnueabihf",
  "mipsel-linux-musl",
  "mips-linux-gnueabi",
  "mips-linux-gnueabihf",
  "mips-linux-musl",
  "powerpc64le-linux-gnu",
  "powerpc64le-linux-musl",
  "powerpc64-linux-gnu",
  "powerpc64-linux-musl",
  "powerpc-linux-gnueabi",
  "powerpc-linux-gnueabihf",
  "powerpc-linux-musl",
  "riscv64-linux-gnu",
  "riscv64-linux-musl",
  "s390x-linux-gnu",
  "s390x-linux-musl",
  "sparc-linux-gnu",
  "sparcv9-linux-gnu",
  "wasm32-freestanding-musl",
  "wasm32-wasi-musl",
  "x86_64-linux-gnu",
  "x86_64-linux-gnux32",
  "x86_64-linux-musl",
  "x86_64-windows-gnu",
  "x86_64-macos-gnu",
  "x86_64-macos-gnu",
  "x86_64-macos-gnu"
]

[probonopd]

It built something, which is good.

But the binaries are neither static nor stripped it seems:

unzip artifacts.zip
chmod +x *x86_64.AppImage
./appimagetool-694-x86_64.AppImage --appimage-extract
file squashfs-root/usr/bin/appimagetool
squashfs-root/usr/bin/appimagetool: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, Go BuildID=MdOg4ubY6Wqu6zBUkYlO/-tfmbJID3VCB37_YQMfW/DwYNCDuhb2uwRRKQqxeb/AwznReSfU21Oz_8xQP20, with debug_info, not stripped

[kristoff-it]

Can you run ldd on it?

[probonopd]

It just executes it, no ldd output. So it seems "semi-static"?

[kristoff-it]

ldd should say "not a dynamic executable" or list the dependencies, it shouldn't run anything. You probably ran ld (great names I know :^))

[kristoff-it]

note that those flags that you added in the last commit (--no-gc-sections) are only useful if you remove all the external linker stuff from the Go command. It's meant to fix an issue when using Go as the linker. If you don't plan gong that route, you don't want those, so either try them with the simplified Go build command, or don't add them.

[probonopd]

ed62251 behaves best so far.

FreeBSD% file squashfs-root/usr/bin/appimagetool
squashfs-root/usr/bin/appimagetool: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, Go BuildID=1e3usIBxKzQmAPw5WMiE/P9QU1M_apYHeqahwX4hP/DwYNCDuhb2uwRRKQqxeb/8V-Y2SxUJjeCL_h4GlnE, with debug_info, not stripped
FreeBSD% ldd squashfs-root/usr/bin/appimagetool
ldd: squashfs-root/usr/bin/appimagetool: not a dynamic ELF executable

But it is still not stripped...

[probonopd]

What the *

strip appimaged
strip: Unable to recognise the format of the input file `appimaged'

[probonopd]

What got built is looking good now. Next step: Functionality testing on a Linux client machine.

[probonopd]

First quick tests looking good! @kristoff-it @andrewrk thank you very, very much for your help.

[motiejus]

Re. stripping: what about passing -Wl,--strip-all to the CFLAGS?