arm64 Docker images openly available

igorvpcleao commented 3 years ago

Description

apache/druid repository on Dockerhub stores amd64 images only. I'd like to suggest generating official arm64 images.

Motivation

arm64 workloads are becoming more and more popular. If possible, Docker images should also be available for this architecture.

anuragagarwal561994 commented 2 years ago

@igorvpcleao

https://github.com/arm64-compat/apache-druid

I have implemented a workaround for the same, if you are using Mac with M1 chip processor, these images will greatly speed up your processes (from my experience). Note that images are only for development purpose.

Not to be used in production.

I haven't throughly tested the same, I would love to hear your comments.

2bethere commented 2 years ago

Woho~.

Did you have do to anything special to get the arm build up and running?

anuragagarwal561994 commented 2 years ago

Nothing special as such, to summarise the changes I guess only 2 things are mainly required:

change the amd64/busybox to just busybox it will pick arm or amd as per the availabile arch
change the base image from distroless, distroless java 8 is on amd only. So I used adoptopenjdk

But in my repo I did a little bit more changes, I shifted the mvn build step from image to ci to make it utilise the maven build cache next time.

And I did some related changes.

Although the image that I pushed is not currently working I tested after I posted the comment, I repeated the same steps on my local and it was working fine. I will be able to figure out the issue and correct over the next 2 days.

The UI is not loading and giving 404 instead to me it seems like the console didn't compile properly in the ci environment and nothing was reported while building, this is my initial guess, will drill down.

2bethere commented 2 years ago

OK, thanks for trying this out. I think it'll be awesome to get a arm build to run well on travis.

anuragagarwal561994 commented 2 years ago

@2bethere https://github.com/arm64-compat/apache-druid/issues/1 I identified the issue and fixed the same.

If you want you can give this image a try as well.

I can also contribute to the apache/druid repo, the fixes in the docker file. I saw that the .travis build in the check already tests for ARM.

Just we might have to decide on a base image.

2bethere commented 2 years ago

Yeah, on travis the console is skipped to improve build speed right now. We should probably try to push https://github.com/apache/druid/pull/11109 forward to get an arm build on docker hub.

Double checking this is what you are asking for?

anuragagarwal561994 commented 2 years ago

Yes I may have mixed the commands while building on travis right now it is fixed in my image, I was asking if you would want to test out my image and see if everything works correctly or is there anyway I can contribute to the project to server arm compatible docker image. For me now I haven't seen any issues.

FrankChen021 commented 2 years ago

ARM64 image now can be built on the master branch from both Linux and Mac M1/M2. But I don't know when the official ARM64 image will be provided on Docker Hub.

m17kea commented 1 year ago

Any update on this?

dudo commented 9 months ago

Let's add 2024 to the mix.

Any update on this? I'm pulling apache/druid:28.0.1 and still not seeing ARM support in docker hub.

I see the docs show how to manually build, but is it that much of a hassle to host them? (legitimately curious)

anuragagarwal561994 commented 9 months ago

You can pull onofficial arm64 compatible image from https://github.com/arm64-compat/apache-druid

let me know if you want me to upgrade the version for the same or help contribute

dudo commented 9 months ago

Thanks, @anuragagarwal561994! It looks like there is "official support" now, you just have to build it yourself.

Unfortunately I wasn't able to compile main on my M1 last night. This is the error I'm getting, if anyone is more familiar with Java.


-------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:3.1.0:exec (generate-binary-license) on project distribution: Command execution failed.: Process exited with an error: 1 (Exit value: 1) -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:exec-maven-plugin:3.1.0:exec (generate-binary-license) on project distribution: Command execution failed.

anuragagarwal561994 commented 9 months ago

@dudo actually since it is more or less java, it is actually compiable multi platform. I will try to build it myself once, if there is more to look into, I will try to invest some time this week. Can you also tell me which version or tag you are trying to build

anuragagarwal561994 commented 9 months ago

@dudo I have built the latest version 28.0.1

https://github.com/arm64-compat/apache-druid/pkgs/container/apache%2Fdruid/177604533?tag=28.0.1

you can try and use it for your local / staging setup. Please refrain from using it in production as I am just a maintainer of this repo / oreganization but I can't actively test these images and their features myself, I hope I am able to make your life easier :)

dmitry-livchak-qco commented 9 months ago

I second this. Would be very handy if we could run an "out-of-box" Druid image on AWS Graviton. Makes a lot of sense both for small and large scale setups, especially if sharing a K8s cluster with other ARM workloads.