Docker platform discrepancies

[maoueh]

Today, if you run docker run --platform linux/arm64 ... and docker run --platform linux/amd64 ... the set of system include paths is not consistent on disk:

docker run --platform=linux/amd64 --rm -it --entrypoint bash goreleaser/goreleaser-cross:v1.20.5 -c "ls /usr | grep -E '(aarch64-linux-gnu|x86_64-linux-gnu)'"
aarch64-linux-gnu

docker run --platform=linux/arm64 --rm -it --entrypoint bash goreleaser/goreleaser-cross:v1.20.5 -c "ls /usr | grep -E '(aarch64-linux-gnu|x86_64-linux-gnu)'"
x86_64-linux-gnu

This means that if you run the --platform=linux/arm64 version, path /usr/aarch64-linux-gnu does not exist as it's directly /usr and the opposite for --platform=linux/amd64 (path /usr/x86_64-linux-gnu does not exist).

I think we can make image consistent across platform by having 3 symlinks to /usr/{include,lib,bin}. If you agree, I would like to propose a PR for this.

[troian]

thanks for reporting, mind showing use-case when this is causing issues

[maoueh]

It's unclear to me yet if it's actually a real problem :) I was adding a new build for linux/arm64 and for linux/amd64 I had defined:

  - id: linux-amd64
    main: ./cmd/{{ .global.Binary }}
    binary: {{ .global.Binary }}
    goos:
      - linux
    goarch:
      - amd64
    env:
      - CGO_ENABLED=1
      - CC=x86_64-linux-gnu-gcc
      - CXX=x86_64-linux-gnu-g++
      - C_INCLUDE_PATH=/usr/x86_64-linux-gnu/include
      - LIBRARY_PATH=/usr/x86_64-linux-gnu/lib
    flags:
      - -trimpath
      - -mod=readonly
    ldflags:
      - -s -w -X main.version={{`{{.Version}}`}}

So I was looking for the exact include path replacement for /usr/x86_64-linux-gnu/{include,lib} which I expected should have been at /usr/aarch64-linux-gnu. But I'm running the linux/arm64 version of the Docker image so the path I was looking for is actually /usr/{include,lib}.

I'm not 100% sure it's a real problem, I think maybe the /usr/include will be checked anyway so it's gonna work.

I'm trying to build this linux/arm64 version as we speak, so I'll let you know.

[troian]

i see, check out here how we do that

[maoueh]

I removed C_INCLUDE_PATH and LIBRARY_PATH and my build still passes, with the new added linux/arm64. Interestingly, I'm pretty sure I added that at some point because I had some compilation issues related to CGO. But now I'm unsure what was it for this case.

Something I do however is to wrap goreleaser-cross with added native (read C) pre-compiled library + include for a build dependency and I place them in /usr/<triplet>/{include,lib}. So I feel there is still worth to have a consistent paths between Docker platform image.

However, it seems for now it's not a big deal, if you prefer to not have such PR, I'm good also and we can close the issue.

[troian]

yeah, lets close it, feel free to reopen if issue appears again, i'll give it a closer look

[maoueh]

Ok here a case that just bite me about the discrepancies of the include paths. I'm building a container out of goreleaser directly, adding some runtime dependencies within the image so that the build process can go smoothly. I'm downloading pre-compiled lib/include with this in my Dockerbuild file:

RUN cd /root &&\
    mkdir wasmedge-ubuntu-20.04-x86-64 &&\
    cd wasmedge-ubuntu-20.04-x86-64 &&\
    wget -O wasmedge.tar.gz https://github.com/WasmEdge/WasmEdge/releases/download/0.11.2/WasmEdge-0.11.2-ubuntu20.04_x86_64.tar.gz &&\
    tar -xzvf wasmedge.tar.gz &&\
    mkdir -p /usr/x86_64-linux-gnu/lib &&\
    cp -R WasmEdge-0.11.2-Linux/include/* /usr/x86_64-linux-gnu/include &&\
    cp -R WasmEdge-0.11.2-Linux/lib/* /usr/x86_64-linux-gnu/lib &&\
    rm -rf wasmedge-ubuntu-20.04-x86-64

RUN cd /root &&\
    mkdir wasmedge-ubuntu-20.04-aarch64 &&\
    cd wasmedge-ubuntu-20.04-aarch64 &&\
    wget -O wasmedge.tar.gz https://github.com/WasmEdge/WasmEdge/releases/download/0.11.2/WasmEdge-0.11.2-manylinux2014_aarch64.tar.gz &&\
    tar -xzvf wasmedge.tar.gz &&\
    mkdir -p /usr/lib &&\
    cp -R WasmEdge-0.11.2-Linux/include/* /usr/include &&\
    cp -R WasmEdge-0.11.2-Linux/lib/* /usr/lib &&\
    rm -rf wasmedge-ubuntu-20.04-aarch64

In this first RUN, I copy to /usr/x86_64-linux-gnu/include but this path doesn't exist if platform used for goreleaser is linux/amd64 (I'm running OSX M1 so it automatically pick linux/arm64 normally).

So I'm left to do conditional if to determine where to put the files exactly. Do you have workarounds that could be applicable?

[troian]

I think you're looking for this