opensearch-project / data-prepper

OpenSearch Data Prepper is a component of the OpenSearch project that accepts, filters, transforms, enriches, and routes data at scale.
https://opensearch.org/docs/latest/clients/data-prepper/index/
Apache License 2.0
262 stars 202 forks source link

Docker image for ARM architectures #640

Open dlvenable opened 2 years ago

dlvenable commented 2 years ago

Is your feature request related to a problem? Please describe.

The Data Prepper Docker image is only available for x86 architectures presently. It is written in Java and should be able to run on ARM.

Describe the solution you'd like

Provide an ARM architecture Docker image. This should be done as a Docker multiarch image so that users can follow the same instructions regardless of their architecture.

Tasks

daixba commented 2 years ago

Since there are a lot of benefits of using Graviton instances in AWS, in both EC2 or EKS. It would be Nice to support running this container image in ARM architectures.

JustinasKO commented 2 years ago

Need this also to run on our EKS clusters (Graviton nodes) As AWS so much promoting they Graviton instances strange that DP still has no ARM image ready

dlvenable commented 2 years ago

It appears that the Gradle plugin that Data Prepper uses to build Docker images supports multiarch through buildx.

https://github.com/palantir/gradle-docker/pull/528

JustinasKO commented 2 years ago

Great finding @dlvenable! But then need to update info about image so it would be clear. ETC: https://hub.docker.com/r/opensearchproject/data-prepper/tags - shows that only supported arch is: linux/amd64

JustinasKO commented 2 years ago

@dlvenable Seems like I miss understood so there is possibility to build multi arch but only x64 is beeing built. So if this is small change maybe it could happen soon?

dlvenable commented 1 month ago

The difficulty we had is that we build on GitHub Actions which did not support ARM. However, it does appear that GitHub has added ARM workers recently.

https://github.blog/news-insights/product-news/arm64-on-github-actions-powering-faster-more-efficient-build-systems/