Build not stable in Apple Silicon

[bugflux]

I understand this may not be a very informative way to put it, I'm happy to provide more info if necessary.

I've noticed that wurstmeister/kafka:2.13-2.8.1 is not very stable under Apple Silicon chips (M1 Pro 32GB of RAM).

Compared with this magical build:

• The CPU spikes quite a lot
  • It doesn't seem to be an issue with memory, it's the same with Xmx256m or 512m
• It takes longer to start
• It seems to hang occasionally

Any ideas what might be the cause, or what I can do or provide to find the causes?

[bugflux]

It seems to run fine if building FROM bellsoft/liberica-openjdk-alpine:11.

[rubensayshi]

I think we're seeing similar issues, or issues when the macbook comes out of a hibernate ...

[OACDesigns]

I'm also encountering strange issues with Kafka running in docker on Apple Silicon (also M1 Pro 32GB RAM). I'm using wurstmeister/kafka:2.13-2.8.1 and getting strange crashes here and there. I tell the cause of them, and don't know how to reproduce. But it's happened several times now.

The latest one resulted in a dump to screen which I've saved to this gist: https://gist.github.com/OACDesigns/580e0f816a96b4bd292cdca5ebddeb4e

Maybe someone is better at understanding the logs than I am.

[jimbogithub]

I had similar issues trying the wurstmeister/kafka:2.13-2.7.1. Looks like although it claims to be an arm64 image:

$ docker image inspect wurstmeister/kafka:2.13-2.7.1

[
    {
        "Id": "sha256:e0c8865c03ee43e3247476d85459c6d3009da6395b951752af7d750ca5a976ac",
        "RepoTags": [
            "wurstmeister/kafka:2.13-2.7.1"
        ],
...
        "Architecture": "arm64",
        "Os": "linux",
...

it's actually amd64 inside:

bash-5.1# uname -a
Linux 29e98d5a58d0 5.10.76-linuxkit #1 SMP PREEMPT Mon Nov 8 11:22:26 UTC 2021 x86_64 Linux
bash-5.1# java -version
openjdk version "1.8.0_292"
OpenJDK Runtime Environment (Zulu 8.54.0.21-CA-linux-musl-x64) (build 1.8.0_292-b10)
OpenJDK 64-Bit Server VM (Zulu 8.54.0.21-CA-linux-musl-x64) (build 25.292-b10, mixed mode)

and gives the same slow CPU burning results as forcing docker to use the amd64 image.

azul/zulu-openjdk-alpine doesn't have an arm64 version. I have a fork switching back to the openjdk which works fine:

-FROM azul/zulu-openjdk-alpine:8u292-8.54.0.21
+FROM openjdk:8u212-jre-alpine

Haven't tried the latest 8u312-jre yet.

[jimbogithub]

So +1 for @bugflux

It seems to run fine if building FROM bellsoft/liberica-openjdk-alpine:11.

if that's the best available alpine JRE 8 or 11.

[elja]

can you please share your docker file? the documentation is outdated, things changed a lot. I was only able to start it up from rosseta.

[bugflux]

Literally just replaced the base image in the Dockerfile, everything else unchanged

[jimbogithub]

So +1 for @bugflux

It seems to run fine if building FROM bellsoft/liberica-openjdk-alpine:11.

if that's the best available alpine JRE 8 or 11.

Found that one didn't work for me for amd64/x86: /usr/lib/jvm/jdk-11.0.14-bellsoft-x86_64/bin/java: error while loading shared libraries: libz.so.1: cannot open shared object file: No such file or directory

So stuck with openjdk:8u212-jre-alpine for now.

[joshbranham]

Has anyone found a base layer that will work for a cross-arch build with docker buildx? The alternative base images suggested seem to only be for arm64.

[jimbogithub]

Has anyone found a base layer that will work for a cross-arch build with docker buildx? The alternative base images suggested seem to only be for arm64.

openjdk:8u212-jre-alpine does work for both arm64 and amd64 but clearly is getting old and possibly insecure. Haven't found anything else based on alpine that will do the job. How hard would it be to move to a different linux distro?

[joshbranham]

Has anyone found a base layer that will work for a cross-arch build with docker buildx? The alternative base images suggested seem to only be for arm64.

openjdk:8u212-jre-alpine does work for both arm64 and amd64 but clearly is getting old and possibly insecure. Haven't found anything else based on alpine that will do the job. How hard would it be to move to a different linux distro?

Yeah you are right, mistake on my end. Do we know why the newer openjdk images aren't multi-arch?

[jimbogithub]

openjdk:11.0.14-jre is multi-arch on Debian. Here's my take on converting the Docker file:

Patch.zip

Notes:

• Installs libc-bin via apt rather than the wget/apk GLIBC on alpine. I just guessed that's what was needed - maybe someone can confirm it's the equivalent. Hence no longer needs/uses the GLIBC related args as it will just get the OS compatible package.
• Also had to add net-tools for netstat which was not on Debian by default. I know netstat is now out of favour but I didn't want to go making more changes to see where/how it was used.
• Uses a /tmp2 dir rather than /tmp as Debian keeps some non-deletable files in /tmp. Using /tmp2 makes it easy to just delete it when done.
• 'source' is not available on Debian, hence the change to use '.' in the download-kafka.sh which was also changed to use the /tmp2 dir.

That's it. I've built for amd64 and arm64 and done some testing on arm64. Seems to be OK.

Maybe someone can give a second opinion or tighten up anything that's dodgy? I'm not a Linux expert.

[jimbogithub]

Looks like the packages can be trimmed down to:

apt-get install -y --no-install-recommends jq net-tools ; \

[joshbranham]

openjdk:11.0.14-jre is multi-arch on Debian. Here's my take on converting the Docker file:

Patch.zip

Notes:

• Installs libc-bin via apt rather than the wget/apk GLIBC on alpine. I just guessed that's what was needed - maybe someone can confirm it's the equivalent. Hence no longer needs/uses the GLIBC related args as it will just get the OS compatible package.
• Also had to add net-tools for netstat which was not on Debian by default. I know netstat is now out of favour but I didn't want to go making more changes to see where/how it was used.
• Uses a /tmp2 dir rather than /tmp as Debian keeps some non-deletable files in /tmp. Using /tmp2 makes it easy to just delete it when done.
• 'source' is not available on Debian, hence the change to use '.' in the download-kafka.sh which was also changed to use the /tmp2 dir.

That's it. I've built for amd64 and arm64 and done some testing on arm64. Seems to be OK.

Maybe someone can give a second opinion or tighten up anything that's dodgy? I'm not a Linux expert.

Can you link a PR from a fork with the changes?

[andershermansen]

I had problems running the wurstmeister/kafka:2.13-2.8.1 image on my Apple Silicon machine, it crashed with som JVM error. I tried to build an image from the #703 PR. This is working perfectly and as expected with my application.

[mikael-carlstedt]

What is stopping this from getting merged?

[jimbogithub]

@mikael-carlstedt #703 has been merged so @wurstmeister can probably close this PR as obsolete.

[TBragi]

Just a heads up, this change from Apline to Debian gave me an error: I am using a setup where the hostname of my kafka cluster nodes is something in the likes of:

server01-domain1.com
server02-domain1.com
server03-domain1.com

I had previously used environment variable BROKER_ID_COMMAND: 'hostname | grep -oE "(\d+)" | head -1' which resulted in broker ID's: 01, 02, 03.

After this merge the grep would catch the d in domain instead. This has been fixed in two ways:

• BROKER_ID_COMMAND: 'hostname | grep -oE "[0-9]+" | head -1' this uses "correct" Extended regular expression
• BROKER_ID_COMMAND: 'hostname | grep -oP "(\d+)" | head -1' this uses Perl Regular expression

It seems like the previous grep had the Perl RegEx as part of the Extended RegEx, but that is no longer the case?