milindl
commented
6 months ago This is a known issue, we're working on a fix. Thanks for the report!
The script that imports librdkafka static bundle into this repository has a bug, and is inconsistent across versions with regards to whether cyrus-sasl stuff is included or not. We plan to fix this in the next release while importing librdkafka, we're changing the script to always exclude it. More details here, https://github.com/confluentinc/confluent-kafka-go/issues/981.
maxime-roo
Description
Building a statically linked binary with the vendored library doesn't work consistently for arm64 across versions of the library.
I followed the steps in Static builds on Linux.
I understand that with the vendored library, I should be able to build a static binary:
Note that the issue is for arm64, following the steps in the readme for amd64 with the vendored library works. Building https://github.com/confluentinc/librdkafka from source can work too for both arm64 and amd64. I haven't tested with other architectures as only amd64 and arm64 vendored libraries are provided.
How to reproduce
Dockerfile
```dockerfile FROM --platform=linux/$TARGETARCH golang:1.22.1-alpine as build ARG LIB_VERSION RUN apk --no-cache add git gcc musl-dev RUN go mod init test-app && go get github.com/confluentinc/confluent-kafka-go/v2@$LIB_VERSION RUN echo $'package main\nimport "github.com/confluentinc/confluent-kafka-go/v2/kafka"\nfunc main() { kafka.NewProducer(&kafka.ConfigMap{}) }' > main.go RUN go build -o cmd -tags musl --ldflags '-linkmode external -extldflags "-static -s"' . FROM --platform=linux/$TARGETARCH gcr.io/distroless/static-debian12 as release COPY --from=build /go/cmd /cmd CMD ["/cmd"] ```Succeeds
Fails
output for arm64 and library version 2.3.0
Adding cyrus-sasl
Adding cyrus-sasl-static doesn't fix the linker issue either.
Dockerfile
```Dockerfile FROM --platform=linux/$TARGETARCH golang:1.22.1-alpine as build RUN apk --no-cache add git gcc musl-dev RUN apk --no-cache add cyrus-sasl-static cyrus-sasl-dev ARG LIB_VERSION RUN go mod init test-app && go get github.com/confluentinc/confluent-kafka-go/v2@$LIB_VERSION RUN echo $'package main\nimport "github.com/confluentinc/confluent-kafka-go/v2/kafka"\nfunc main() { kafka.NewProducer(&kafka.ConfigMap{}) }' > main.go RUN CGO_LDFLAGS="-lsasl2" go build -o cmd -tags musl --ldflags '-linkmode external -extldflags "-static -s"' . FROM --platform=linux/$TARGETARCH gcr.io/distroless/static-debian12 as release COPY --from=build /go/cmd /cmd CMD ["/cmd"] ```Output
Manualy building the library
It is possible to build a static binary when following the steps of #471
Dockerfile
```Dockerfile FROM --platform=linux/$TARGETARCH golang:1.22.1-alpine3.19 as build RUN apk --no-cache add git gcc musl-dev ARG LIB_VERSION RUN go mod init test-app && go get github.com/confluentinc/confluent-kafka-go/v2@$LIB_VERSION RUN apk add --no-cache \ bash \ build-base \ coreutils \ gcc \ git \ make \ musl-dev \ openssl-dev \ openssl-libs-static \ rpm \ lz4-dev \ lz4-static \ zlib-dev \ zlib-static \ zstd-dev \ zstd-static \ wget && \ # download libsasl to a temp location and build # enable/disable features as needed with configure cd $(mktemp -d) && \ wget -nv -O cyrus-sasl-2.1.28.tar.gz https://github.com/cyrusimap/cyrus-sasl/releases/download/cyrus-sasl-2.1.28/cyrus-sasl-2.1.28.tar.gz && \ tar -xz --strip-components=1 -f cyrus-sasl-2.1.28.tar.gz && \ rm -f cyrus-sasl-2.1.28.tar.gz && \ ./configure --prefix=/usr --disable-sample --disable-obsolete_cram_attr --disable-obsolete_digest_attr --enable-static --disable-shared \ --disable-checkapop --disable-cram --disable-digest --enable-scram --disable-otp --disable-gssapi --with-dblib=none --with-pic && \ make -j && \ make install && \ cd $(mktemp -d) && \ wget -nv -O v2.3.0.tar.gz https://github.com/confluentinc/librdkafka/archive/v2.3.0.tar.gz && \ tar -xz --strip-components=1 -f v2.3.0.tar.gz && \ rm -f v2.3.0.tar.gz && \ ./configure --prefix=/usr \ --enable-zlib \ --enable-zstd \ --enable-ssl \ --enable-gssapi \ --enable-sasl \ --enable-lz4-ext && \ make -j && \ make install RUN echo $'package main\nimport "github.com/confluentinc/confluent-kafka-go/v2/kafka"\nfunc main() { kafka.NewProducer(&kafka.ConfigMap{}) }' > main.go RUN CGO_LDFLAGS="-lsasl2" go build -o cmd -tags musl --ldflags '-linkmode external -extldflags "-static -s"' . FROM --platform=linux/$TARGETARCH gcr.io/distroless/static-debian12 as release COPY --from=build /go/cmd /cmd CMD ["/cmd"] ```Building librdkafka is not strictly necessary, building cyrus-sasl is:
Dockerfile
```Dockerfile FROM --platform=linux/$TARGETARCH golang:1.22.1-alpine3.19 as build RUN apk --no-cache add git gcc musl-dev ARG LIB_VERSION RUN go mod init test-app && go get github.com/confluentinc/confluent-kafka-go/v2@$LIB_VERSION RUN apk add --no-cache \ bash \ build-base \ coreutils \ gcc \ git \ make \ musl-dev \ openssl-dev \ openssl-libs-static \ rpm \ lz4-dev \ lz4-static \ zlib-dev \ zlib-static \ zstd-dev \ zstd-static \ wget && \ # download libsasl to a temp location and build # enable/disable features as needed with configure cd $(mktemp -d) && \ wget -nv -O cyrus-sasl-2.1.28.tar.gz https://github.com/cyrusimap/cyrus-sasl/releases/download/cyrus-sasl-2.1.28/cyrus-sasl-2.1.28.tar.gz && \ tar -xz --strip-components=1 -f cyrus-sasl-2.1.28.tar.gz && \ rm -f cyrus-sasl-2.1.28.tar.gz && \ ./configure --prefix=/usr --disable-sample --disable-obsolete_cram_attr --disable-obsolete_digest_attr --enable-static --disable-shared \ --disable-checkapop --disable-cram --disable-digest --enable-scram --disable-otp --enable-gssapi --with-dblib=none --with-pic && \ make -j && \ make install RUN echo $'package main\nimport "github.com/confluentinc/confluent-kafka-go/v2/kafka"\nfunc main() { kafka.NewProducer(&kafka.ConfigMap{}) }' > main.go RUN CGO_LDFLAGS="-lsasl2" go build -o cmd -tags musl --ldflags '-linkmode external -extldflags "-static -s"' . FROM --platform=linux/$TARGETARCH gcr.io/distroless/static-debian12 as release COPY --from=build /go/cmd /cmd CMD ["/cmd"] ```Checklist
Please provide the following information:
LibraryVersion()):ConfigMap{...}"debug": ".."as necessary)