search

solo-io / gloo

The Feature-rich, Kubernetes-native, Next-Generation API Gateway Built on Envoy
https://docs.solo.io/
Apache License 2.0
4.06k stars 432 forks source link

Build Official ARM64 Images #3486

Open lgadban opened 4 years ago

lgadban commented 4 years ago

Is your feature request related to a problem? Please describe. Build official ARM images for the various gloo components

Describe the solution you'd like ARM compatible images

Describe alternatives you've considered N/A

Additional context Envoy now provides multi-arch images with arm64 support AWS ARM instances

Related Issues:

┆Issue is synchronized with this Asana task by Unito

CelsoSantos commented 3 years ago

This should be re-opened as my PR did not solve this issue (it tracks official releases, which still has some work to be done)

Sodman commented 3 years ago

Agreed, reopening! We have the makefile changes required to build ARM64 images of Gloo, but we don't have a CI/CD pipeline setup with the ARM worker machines required to build it yet. Our current pipeline is in GKE / Github Actions. We may need to integrate with AWS for this.

Further, for envoy-gloo there is at least one upstream dependency (glibc) which has no official arm64 support.

CelsoSantos commented 3 years ago

Indeed. I had to create the glibc image and the frovlad/alpine-glibc images for arm myself: https://github.com/CelsoSantos/alpine-glibc

I recommend Solo forks or recreates this repo in order to handle that

murphye commented 3 years ago

Is this on roadmap? Thanks!

christian-posta commented 3 years ago

Yah depends on ask, we have the builds for Istio so should be able to get this theoretically if someone needs it.

murphye commented 3 years ago

Would be great for Graviton :-)

ionutz89 commented 3 years ago

Any news about it?

WyriHaximus commented 3 years ago

Is there anyway we can help out to move this forward?

lgadban commented 2 years ago

@chrisgaun FYI

pranaypratyush commented 2 years ago

Need this bad.

phenixblue commented 2 years ago

It should be possible to use Docker's buildx feature to build multi-arch images without needing access to runners of different architectures.

Example doing this in GitHub Actions: https://github.com/tmobile/magtape/blob/b78170b5b020dd2f0a336d514e48fec5980227cb/.github/workflows/ci-image-build.yaml#L62

chrisgaun commented 2 years ago

We have done this with Istio btw and found that there is much high CPU usage for TLS handshakes in ARM vs Intel/AMD. So for those looking to save money using Graviton it would depend on the number of TLS connections.

CelsoSantos commented 2 years ago

@chrisgaun would you have some more information to share? Could you eventually do a comparison for the same number of connections how that affect performance? (CPU/RAM/Network, etc). I'd be very interested in learning more about it

phenixblue commented 2 years ago

I know a lot of our use case behind this is more for local development so performance differences aren't as much a concern (still a concern if there's a drastic difference). A lot of our development teams use Apple laptops and those will be moving to arm64 as an only option before too long. We like to have a full k8s/gloo deployment in a local environment on the laptops and that's going to be impossible/difficult without upstream support for arm64

chrisgaun commented 2 years ago

Ah yes. We have the same use case internally actually as many of us are on the M1s. We have good internal docs on this that we should add to our docs.

WyriHaximus commented 2 years ago

To add to the use case list: I want to use Gloo on my home lab k8s cluster as the ingress service. Doubt I'll see many TLS connections so that's not a major issue for me.

chrisgaun commented 2 years ago

Understood understood.

ac427 commented 2 years ago

@chrisgaun , any new update?

scags9876 commented 2 years ago

I'm going to second what @phenixblue stated. Exact same use case for us. Any updates on this issue being addressed?

abebars commented 1 year ago

any updates on this issue?

chrisgaun commented 1 year ago

Let me check with engineering. Most run on M1 macs.

chrisgaun commented 1 year ago

Everyone. This seems to be a limitation on using Google cloud. They did not have ARM machines until very recently and even now it is very limited supply.

For Gloo Mesh we use an (expensive) self hosted AWS EKS cluster as a runner to build ARM images. Google says the ARM machines will be GA in "early 2023." Until then we can do one offs for customers potentially but more general pipeline still is a bit out.

WyriHaximus commented 1 year ago

You don't need ARM machines to build ARM images tho. All you need is docker with qemu to do a build or buildx to create the images. (It is slow to build...)

soloio-bot commented 1 year ago

Zendesk ticket #1138 has been linked to this issue.

alex-berger commented 9 months ago

Any progress/plans on this? Currently, gloo-edge is the last amd64 workload in our clusters and (for cost optimization purposes) I wold love to have arm64 only.

From what I understand this would imply

Maybe I can help with a PR if this is not too complex :-), any pointers and hints are highly appreciated.

github-actions[bot] commented 3 months ago

This issue has been marked as stale because of no activity in the last 180 days. It will be closed in the next 180 days unless it is tagged "no stalebot" or other activity occurs.

alex-berger commented 3 months ago

Not stale, still relevant.

BeauSelisker28 commented 2 months ago

Another customer cited seeking Graviton based images for Gloo Edge.

nrjpoddar commented 1 month ago

@nfuden to look at this issue for scope/estimation.

Work with @rpunia1 on it.