Unable to build for target linux/amd64.

[Lwrless]

Today I was setting up a new building environment, and was unable to build for target linux/amd64.

At first I xgo gave me the following error message, and I thought that it was ldflags which broke the build:

Compiling for linux/amd64...
panic: path "--ldflags= -extldflags \"-static\"" not in error "invalid import path \"--ldflags= -extldflags \\\"-static\\\"\"" [recovered]
    panic: path "--ldflags= -extldflags \"-static\"" not in error "invalid import path \"--ldflags= -extldflags \\\"-static\\\"\""

goroutine 1 [running]:
cmd/go/internal/load.(*preload).flush(0xc000282050)
    /usr/local/go/src/cmd/go/internal/load/pkg.go:1086 +0x78
panic({0x97d420, 0xc0003f5430})
    /usr/local/go/src/runtime/panic.go:884 +0x212
cmd/go/internal/load.ImportErrorf({0x7ffe042a3c52, 0x20}, {0xa2f3a2?, 0x20?}, {0xc000135630?, 0xc000135b18?, 0x0?})
    /usr/local/go/src/cmd/go/internal/load/pkg.go:523 +0x199
cmd/go/internal/load.(*Package).load(0xc00068f700, {0xb2e690, 0xc000026160}, {0x0, 0x0, 0x0, 0x1, 0x0, 0x0}, {0x7ffe042a3c52, ...}, ...)
    /usr/local/go/src/cmd/go/internal/load/pkg.go:1894 +0x147d
cmd/go/internal/load.loadImport({0xb2e690, 0xc000026160}, {0x0, 0x0, 0x0, 0x1, 0x0, 0x0}, 0xc000282050, {0x7ffe042a3c52, ...}, ...)
    /usr/local/go/src/cmd/go/internal/load/pkg.go:744 +0x570
cmd/go/internal/load.PackagesAndErrors({0xb2e690?, 0xc000026160?}, {0x0, 0x0, 0x0, 0x1, 0x0, 0x0}, {0xc0000240d0, 0x9, ...})
    /usr/local/go/src/cmd/go/internal/load/pkg.go:2794 +0x87b
cmd/go/internal/work.runBuild({0xb2e690, 0xc000026160}, 0xc00002e4b0?, {0xc0000240d0, 0x9, 0x9})
    /usr/local/go/src/cmd/go/internal/work/build.go:408 +0xba
main.invoke(0xe298a0, {0xc0000240c0, 0xa, 0xa})
    /usr/local/go/src/cmd/go/main.go:225 +0x34e
main.main()
    /usr/local/go/src/cmd/go/main.go:179 +0x7d1

Then I proceeded to try building without ldflags flag, and xgo still failed:

Compiling for linux/amd64...
malformed import path "--tags": leading dash
package netgo is not in GOROOT (/usr/local/go/src/netgo)
malformed import path "--ldflags= ": leading dash
malformed import path "--gcflags=": leading dash
malformed import path "-o": leading dash

Here is my command for building the package:

xgo -tags 'netgo' -ldflags  '-extldflags "-static"' --targets=linux/amd64 .

It seems that pull request #180 broke some string concatenation stuff in docker/base/build.sh. And with an older xgo Docker image, the building process succeeded:

xgo -image techknowlogick/xgo:go-1.19.2 -tags 'netgo' -ldflags  '-extldflags "-static"' --targets=linux/amd64 .

[zeripath]

Yup I think there was a bug with #180 that wasn't detected on CI - I'm also suspicious that #148 added --ldflags to go get when it shouldn't have.

[zeripath]

Yup CI doesn't appear to be rebuilding and testing the base images.

---

I think it's because the images aren't being tagged locally during the build so instead of using the recently built "base" and "go-1.19.3" for "latest" it's just downloading the old one.

---

I'm gonna try adding the setting outputs: type=image,push=false to the docker-push command to see if that can make the test-pr system use the newly built base tag.

---

ah looks like it should be load: true instead

---

Nope I don't know what to do!

[Lwrless]

Tested build.sh from PR #184 and problem is gone, this issue could be closed once #184 gets merged.

[xoxys]

This is still broken for me somehow:

root@ad6c2d348f54:/source# make xgo
go run src.techknowlogick.com/xgo@latest -go go-1.19.x -v -tags 'netgo' -ldflags '-extldflags "-static"' -targets 'linux/amd64,linux/arm-6,linux/arm-7,linux/arm64' -out drone-s3-sync --pkg cmd/drone-s3-sync .
Cross compiling ....
Building locally ....
Compiling for linux/amd64...
flag provided but not defined: -extldflags "-static"
usage: go build [-o output] [build flags] [packages]
Run 'go help build' for details.
2022/11/23 09:24:33 Failed to cross compile package: exit status 2.
exit status 1
Makefile:79: recipe for target 'xgo' failed
make: *** [xgo] Error 1

I have nothing changed on my build setup but the latest techknowlogick/xgo starts failing.

[xoxys]

Switching back to an older digest of the image techknowlogick/xgo:go-1.19.x@sha256:b3441e1e1d2e5da281bfa8398cf70d2450d4b82464ee61eea8f0480c96edeab9 for now and the build works again https://github.com/thegeeklab/drone-s3-sync/pull/51/files. That's of course just a workaround. Should I create a new issue?

[xoxys]

@zeripath Seems to be related to the change in Line 196 of https://github.com/techknowlogick/xgo/pull/184/files#diff-08ee30052ae3fc42f1e871d9cec2bd4c861efff636342532ef2cc43a9386daf7R196-R197

The resulting --ldflags= is missing quotes and the parts of ${LD[@]} were passed to go build as dedicated flags, resulting in flag provided but not defined. What works for me (not sure if that's the correct approach) is LDF=(--ldflags="$(printf "%s " "${LD[@]}")") to get a proper --ldflags parameter.

[xoxys]

Removing -v from go run src.techknowlogick.com/xgo@latest -go go-1.19.x -v -ldflags ... also avoids this problem, as in this case LD is an array with only one entry. In any other case where LD results in an array with more than one entry, the build script will fail due to incorrect construction of the --ldflags parameter.

[zeripath]

Damn! Sorry about this!

I have another PR that should fix that.

[zeripath]

diff --git a/docker/base/build.sh b/docker/base/build.sh
index 835d293..c134bfd 100644
--- a/docker/base/build.sh
+++ b/docker/base/build.sh
@@ -182,7 +182,7 @@ if [ "$FLAG_X" == "true" ];    then X=-x; fi
 if [ "$FLAG_RACE" == "true" ]; then R=-race; fi
 if [ "$FLAG_TAGS" != "" ];     then T=(--tags "$FLAG_TAGS"); fi
 if [ "$FLAG_LDFLAGS" != "" ];  then LD=("${LD[@]}" "${FLAG_LDFLAGS[@]}"); fi
-if [ "$FLAG_GCFLAGS" != "" ];  then GC=(--gcflags="$FLAG_GCFLAGS"); fi
+if [ "$FLAG_GCFLAGS" != "" ];  then GC=(--gcflags=\""$FLAG_GCFLAGS"\"); fi

 if [ "$FLAG_BUILDMODE" != "" ] && [ "$FLAG_BUILDMODE" != "default" ]; then BM=(--buildmode="${FLAG_BUILDMODE[@]}"); fi
 if [ "$FLAG_TRIMPATH" == "true" ]; then TP=-trimpath; fi
@@ -193,7 +193,7 @@ if [ "$TARGETS" == "" ]; then
   TARGETS="./."
 fi

-if [ "${#LD[@]}" -gt 0 ]; then LDF=(--ldflags="${LD[@]}"); fi
+if [ "${#LD[@]}" -gt 0 ]; then LDF=(--ldflags=\""${LD[@]}"\"); fi

 # Build for each requested platform individually
 for TARGET in $TARGETS; do
@@ -439,7 +439,7 @@ for TARGET in $TARGETS; do
       LDS=("-s" "${LDS[@]}")
     fi
     if [ ${#LDS[@]} -gt 0 ]; then
-      LDFS=(--ldflags="${LDS[@]}")
+      LDFS=(--ldflags=\""${LDS[@]}"\")
     fi
     # Build the requested darwin binaries
     if [ "$XGOARCH" == "." ] || [ "$XGOARCH" == "amd64" ]; then

[xoxys]

No need to apologize. Does that work for you? I tried simple double quotes + escaping as well without success but maybe I made a mistake.

[zeripath]

Could you give me a quick test case? I checked with echo and it looked right.

---

yup it's still wrong.

[xoxys]

user@superion xgo on  main [!] via 🐹 v1.19 via 🐍 v3.10.8 took 2s 
❯ docker pull techknowlogick/xgo:latest
latest: Pulling from techknowlogick/xgo
Digest: sha256:cb40d894cefb9f946d9fc0bec87f974a02ce29f1de62a80266d4ddd6a67d24d0
Status: Image is up to date for techknowlogick/xgo:latest
docker.io/techknowlogick/xgo:latest
user@superion xgo on  main [!] via 🐹 v1.19 via 🐍 v3.10.8 
❯ docker run -v $(pwd):/source --workdir /source --entrypoint bash -it techknowlogick/xgo:latest
root@ff3189e9d83d:/source# apt update && apt install vim -y
[..]
root@ff3189e9d83d:/source# vim /build.sh 
root@ff3189e9d83d:/source# grep -r "\-\-ldflags" -C 2 /build.sh 
fi

if [ "${#LD[@]}" -gt 0 ]; then LDF=(--ldflags=\""${LD[@]}"\"); fi

# Build for each requested platform individually
--
    fi
    if [ ${#LDS[@]} -gt 0 ]; then
      LDFS=(--ldflags="${LDS[@]}")
    fi
    # Build the requested darwin binaries
root@ff3189e9d83d:/source# go run xgo.go -v -ldflags '-extldflags "-static"' .            
Cross compiling ....
Building locally ....
Compiling for linux/amd64...
invalid value "\"-v" for flag -ldflags: missing =<value> in <pattern>=<value>
usage: go build [-o output] [build flags] [packages]
Run 'go help build' for details.
2022/11/23 15:44:47 Failed to cross compile package: exit status 2.
exit status 1
root@ff3189e9d83d:/source# rm -rf /deps/
root@ff3189e9d83d:/source# go run xgo.go -ldflags '-extldflags "-static"' .
Cross compiling ....
Building locally ....
Compiling for linux/amd64...
invalid value "\"-extldflags \"-static\"\"" for flag -ldflags: missing =<value> in <pattern>=<value>
usage: go build [-o output] [build flags] [packages]
Run 'go help build' for details.
2022/11/23 15:45:07 Failed to cross compile package: exit status 2.
exit status 1
root@ff3189e9d83d:/source# 

This way, it fails in both cases for me.

[zeripath]

I just saw your comment above using the printf which I think is the way to go so I've pushed that up to the PR

[zeripath]

just rebuilding base and go1.19.3 - will retag locally and retest with gitea

[zeripath]

It's really a shame that we're having to go down the printf route - there should be a way of using bash arrays a bit more cleverly - but my bashfoo is failing me.

[xoxys]

Agree still ugly but I gave up after an hour as well trying to find another way.

[zeripath]

OK the printf route is confirmed to work here. I might use my powers to merge.

[zeripath]

I have a feeling we should restructure the dockers - the base should actually be split into (at least) two - one let's say toolchain that has the OS and toolchains on (this should only depend on the Dockerfile) - and bootstrap which contains the bootstrap and build files.

[zeripath]

Thanks @techknowlogick !

[xoxys]

Thank you both.