bitnami / charts

Bitnami Helm Charts
https://bitnami.com
Other
9.01k stars 9.22k forks source link

Support for ARM64 architecture in Bitnami container images #7305

Closed carrodher closed 1 year ago

carrodher commented 3 years ago

Currently, the Bitnami container images do not support the ARM64 architecture.

We are aware of the growing interest in this architecture and there are ongoing internal plans to release the Bitnami Community Catalog for ARM64 in the future, definitely, it is something we would like to support, but we need to find the bandwidth to do it in a proper way maintaining our quality standards.

At the moment, only our base image bitnami/minideb has support for ARM64 thanks to the community contribution in this PR: bitnami/minideb#90

Although there are some initiatives and Engineering work going on, we cannot guarantee a specific ETA for this topic. We will update this issue with more information.

Thanks for using Bitnami Containers and Helm Charts!


Update 24th February: ARM containers are already available in DockerHub, see https://github.com/bitnami/charts/issues/7305#issuecomment-1443187343

ZCube commented 2 years ago

@ZCube Could you please allow the issues on your repo so that we don't pollute this thread with new image requests :)?

OK

turowicz commented 2 years ago

@ZCube if it builds, it works lol

tormi commented 2 years ago

bitnami/elasticsearch

@ZCube, thanks for the efforts! Do I understand it correctly that you're not using Bitnami`s setup at all here: https://github.com/ZCube/bitnami-compat/blob/main/docker/elasticsearch/7.16.3-4/Dockerfile.install ?

ZCube commented 2 years ago

This isn't the goal of not using Bitnami's setup. It is to replace the binary installation method to make Bitnami's chart work on ARM64. Reference : https://www.elastic.co/guide/en/elasticsearch/reference/current/targz.html

tormi commented 2 years ago

Okay, thanks. The side effect seems to be the loss of Bitnami's envvars setup for Elasticsearch.

ZCube commented 2 years ago

@tormi It supports some environment variables, but the contents of the repository are not the subject of this issue. I do not want to pollute this thread. If you have any questions, ask them as an issue in the repository.

RedstoneWizard08 commented 2 years ago

I'm also working on armhf support here: https://github.com/RedstoneWizard08/minideb

RedstoneWizard08 commented 2 years ago

https://github.com/bitnami/minideb/pull/119

talesam commented 2 years ago

I compiled the speech + postgres + redis for ARM64, but I didn't succeed in connecting the bank to the speech, it probably needs more things to change. Has anyone come to test the speech for ARM64? I have some servers at Oracle on ARM64 and I would like to run your Docker Image Speech there, but it's complicated, I've been trying for two days, could anyone help me?

anuragagarwal561994 commented 2 years ago

I have access to ARM based system, how can I contribute to the project for supporting ARM based images?

yankay commented 2 years ago

We have the same problem like @anuragagarwal561994 . How can we contribute bitnami to support ARM. Is there any sample ?

FarhanSajid1 commented 2 years ago

Any updates on redis-cluster m1 support?

rafariossaa commented 2 years ago

Hi,

I'm afraid it is still in our backlog. We've done some feasibility studies but I still cannot provide a date on when this will be available.

anuragagarwal561994 commented 2 years ago

What do you think are the issues and how can the community help?

nodesocket commented 2 years ago

Would absolutely ❤️ arm64 containers / helm chart support for the following:

For example, a project I am trying to deploy Gitea uses Bitnami for PostgreSQL and Memcached dependencies, but I cannot run it because running on a Raspberry Pi (arm64) Kubernetes cluster.

ddelange commented 2 years ago

@nodesocket you could give ZCube/bitnami-compat a spin ref https://github.com/bitnami/charts/issues/7305#issuecomment-1025062796

nodesocket commented 2 years ago

@ddelange from a security standpoint, not sure we can in good faith use the ZCube builds unfortunately. We need a trusted source of truth.

jokeyrhyme commented 2 years ago

Just to double-check: are we blocked here because ARM64 build agents are not available (e.g. paid for)? Or are we blocked because there's potentially non-trivial work to adopt ARM64 build agents?

t1 commented 2 years ago

You don't need special build agents. E.g. https://github.com/t1/rdohna-wildfly-docker-image builds just fine on standard GitHub build infrastructure, both arm and amd.

ddelange commented 2 years ago

Just to double-check: are we blocked here because ARM64 build agents are not available (e.g. paid for)? Or are we blocked because there's potentially non-trivial work to adopt ARM64 build agents?

I think the bottleneck is not so much the computational resources but rather the human resources to get this done in the internal Jenkins pipeline (https://github.com/bitnami/charts/issues/7305#issuecomment-906480229 and https://github.com/bitnami/charts/issues/7305#issuecomment-957326180). If I understand correctly, in the end the ETA will come down to the priorities in the Tanzu roadmap (https://github.com/bitnami/charts/issues/7305#issuecomment-908347702), as it is not something the open source community can support with.

caiofbpa commented 2 years ago

Is there a tracker anywhere marking which images are already supporting arm64 and which ones aren't? It would be really nice to follow the progress over time here.

carrodher commented 2 years ago

It's not progressive work, if finally arm64 is supported, all the container images will be available at the same time, no little by little. Technically, the key point is the underlying test & release infrastructure, not the images themselves.

sobrito commented 2 years ago

Also already added support, or is there still prediction for arm64 for postgresql-ha?, in addition to raspberry pi 4, we also have a complete cluster in oracle with this "Ampere A1" architecture with OCPU ARM, it would be amazing to be able to run the kubernetes cluster too!

nunopato commented 2 years ago

Technically, the key point is the underlying test & release infrastructure, not the images themselves.

@carrodher Is there anything we at https://nhost.io could help with to make this faster? We are currently evaluating moving all our databases to k8s and the bitnami postgres chart covers all our requirements. We are talking about a considerable deployment of the chart for 2k+ databases.

SpootyMcSpoot commented 2 years ago

Any update on this work? We like using bitnami upstream, but without arm-64 image support we need to start looking elsewhere.

arthurfiorette commented 2 years ago

Hello, can we have this issue prioritized? A lot of our containers uses bitnami images...

joaquinjsb commented 2 years ago

any news on this issue? some images are running fine with arm64 now.

anton-artemyev commented 2 years ago

@joaquinjsb do you know which images are running for you? looks like latest is failing, but it was working for me last week on M1

sanskritkm commented 2 years ago

When could you make it possible to support aws arm? m waiting for

nightswimmings commented 2 years ago

This:

It's not progressive work, if finally arm64 is supported, all the container images will be available at the same time, no little by little. Technically, the key point is the underlying test & release infrastructure, not the images themselves.

and this:

any news on this issue? some images are running fine with arm64 now.

are contradictory.

Do we have some non-official support for native ARM on some images already? If so, how can we track which ones are ready?

ddelange commented 2 years ago

Do we have some non-official support for native ARM on some images already? If so, how can we track which ones are ready?

from higher up in this thread: https://github.com/ZCube/bitnami-compat

nightswimmings commented 2 years ago

"These images are NOT intended for production use."

Jonatthu commented 2 years ago

Is this means that I can not use these charts if I am on a macbook pro m1 max?

shanduur commented 2 years ago

Is there any progress on OpenLDAP image for ARM64?

TobiasBales commented 2 years ago

Is this means that I can not use these charts if I am on a macbook pro m1 max?

MacOS supports running them in x64 emulation generally, but e.g. raspberry pis won't work

Jonatthu commented 2 years ago

https://github.com/ZCube/bitnami-compat

@ddelange

Is it possible to make a pull request out of this? Or bitnami is not interested into support the arm64 architecture?

ddelange commented 2 years ago

Hi @Jonatthu,

I'm afraid it's a hack that replaces binaries that are built by internal bitnami pipelines with ones available via official channels like apt-get (which are generally built for multiple cpu architectures, whereas bitnami pipelines currently only build x86).

See also my earlier comment https://github.com/bitnami/charts/issues/7305#issuecomment-1091102891

Jonatthu commented 2 years ago

@ddelange Is there any workaround to run these images on MacOs m1 pro max with rosseta? After all usually when we deploy these charts they go to an EKS or AKS cluster running in Linux or Windows

I just want to be able to test these charts locally.

ddelange commented 2 years ago

Possible, but will be generally very slow and/or buggy

sfxworks commented 2 years ago

Can bitnami helm charts be scheduled to be on kubernetes.io/arch: amd64 in the meantime? It's always annoying to run into random issues only for it to boil down to exec format error

[root@office-server-1 ~]# kubectl get pods -n kubeapps -o wide
NAME                                                          READY   STATUS             RESTARTS         AGE     IP           NODE              NOMINATED NODE   READINESS GATES
apprepo-kubeapps-cleanup-bitnami-dzzgt-7slgs                  0/1     Error              0                96s     10.0.3.1     pinewall-2        <none>           <none>
apprepo-kubeapps-cleanup-bitnami-dzzgt-gn6pn                  0/1     Error              0                4m30s   10.0.4.70    pinewall-1        <none>           <none>
apprepo-kubeapps-cleanup-bitnami-dzzgt-lsfrx                  0/1     Error              0                4m14s   10.0.4.95    pinewall-1        <none>           <none>
apprepo-kubeapps-cleanup-bitnami-dzzgt-qlrhr                  0/1     Error              0                3m49s   10.0.4.234   pinewall-1        <none>           <none>
apprepo-kubeapps-cleanup-bitnami-dzzgt-v97xv                  0/1     Error              0                3m2s    10.0.9.37    pinewall-4        <none>           <none>
apprepo-kubeapps-cleanup-bitnami-dzzgt-zntgw                  0/1     Error              0                4m36s   10.0.4.153   pinewall-1        <none>           <none>
apprepo-kubeapps-sync-mcsh-27671740-zndtx                     0/1     CrashLoopBackOff   5 (2m13s ago)    5m14s   10.0.7.26    pinewall-3        <none>           <none>
apprepo-kubeapps-sync-mcsh-79kzv-5krb2                        0/1     CrashLoopBackOff   5 (87s ago)      4m22s   10.0.4.188   pinewall-1        <none>           <none>
apprepo-kubeapps-sync-mcsh-7jrgz-6jj5z                        0/1     CrashLoopBackOff   5 (30s ago)      3m27s   10.0.7.212   pinewall-3        <none>           <none>
apprepo-kubeapps-sync-mcsh-9qvvz-f7jrj                        0/1     CrashLoopBackOff   4 (58s ago)      2m34s   10.0.7.234   pinewall-3        <none>           <none>
apprepo-kubeapps-sync-mcsh-gsrm4-wxmbx                        0/1     CrashLoopBackOff   4 (57s ago)      2m41s   10.0.9.110   pinewall-4        <none>           <none>
apprepo-kubeapps-sync-mcsh-jht9h-449vl                        0/1     CrashLoopBackOff   5 (2m2s ago)     5m2s    10.0.3.81    pinewall-2        <none>           <none>
apprepo-kubeapps-sync-mcsh-k878h-hxbmf                        0/1     Error              5 (89s ago)      3m6s    10.0.3.243   pinewall-2        <none>           <none>
apprepo-kubeapps-sync-mcsh-mb4vq-22khd                        0/1     CrashLoopBackOff   5 (115s ago)     5m7s    10.0.9.35    pinewall-4        <none>           <none>
apprepo-kubeapps-sync-mcsh-nhshj-ztp77                        0/1     CrashLoopBackOff   5 (78s ago)      4m33s   10.0.9.122   pinewall-4        <none>           <none>
apprepo-kubeapps-sync-mcsh-qqzwj-8wnh2                        0/1     CrashLoopBackOff   4 (63s ago)      2m45s   10.0.4.155   pinewall-1        <none>           <none>
apprepo-kubeapps-sync-mcsh-vp7wj-khl22                        0/1     Error              5 (95s ago)      3m8s    10.0.4.22    pinewall-1        <none>           <none>
kubeapps-56fdc4565d-5gtzl                                     2/2     Running            19 (3d10h ago)   29d     10.0.0.227   office-server-1   <none>           <none>
kubeapps-56fdc4565d-w25dr                                     2/2     Running            20 (3d10h ago)   29d     10.0.0.42    office-server-1   <none>           <none>
kubeapps-internal-apprepository-controller-6f96498df7-97tcg   1/1     Running            3                29d     10.0.0.180   office-server-1   <none>           <none>
kubeapps-internal-dashboard-55694d5b9d-5tnk9                  1/1     Running            3                29d     10.0.0.123   office-server-1   <none>           <none>
kubeapps-internal-dashboard-55694d5b9d-p4j8z                  1/1     Running            3                29d     10.0.0.250   office-server-1   <none>           <none>
kubeapps-internal-kubeappsapis-6c6c54976f-fdskd               1/1     Running            3                29d     10.0.0.126   office-server-1   <none>           <none>
kubeapps-internal-kubeappsapis-6c6c54976f-rzdzz               1/1     Running            3                29d     10.0.0.172   office-server-1   <none>           <none>
kubeapps-internal-kubeops-77ddc5fddc-ct5sn                    1/1     Running            3                29d     10.0.0.232   office-server-1   <none>           <none>
kubeapps-internal-kubeops-77ddc5fddc-k8lzf                    1/1     Running            3                29d     10.0.0.52    office-server-1   <none>           <none>
kubeapps-postgresql-0                                         1/1     Running            3                29d     10.0.0.114   office-server-1   <none>           <none>
blowfishpro commented 2 years ago

Can bitnami helm charts be scheduled to be on kubernetes.io/arch: amd64

It's not zero work but it looks like most of the charts accept nodeAffinity somewhere in their values. If you're running a mixed cluster that might be the best option for now.

ScOut3R commented 2 years ago

We taint our arm64 instances so we can set the toleration on our own charts for services that can be deployed there. This way the "default" is amd64, which is backward compatible with Bitnami and other third party charts.

ddelange commented 2 years ago

I found this a useful read on multi-arch clusters: https://cablespaghetti.dev/2021/02/20/managing-multi-arch-kubernetes-clusters/#taints-and-tolerations

alf1e commented 2 years ago

Why is this on hold?

dfquaresma commented 2 years ago

Hi! Is Solr going to have support for ARM64 too? Can you guys share what workaround (if there's any) could we execute to update the current image and use it with ARM64? Thanks in advance!

huangjc7 commented 2 years ago

When I use docker.io/bitnami/minideb:buster to make a docker image on arm64 architecture, it prompts "E: The repository 'http://security.debian.org buster/updates InRelease' is not signed." Is there any way to solve it?

hameno commented 2 years ago

Soo...this issue has it's first birthday a few weeks ago, what's the plan here?

lefte commented 2 years ago

Soo...this issue has it's first birthday a few weeks ago, what's the plan here?

I found a workaround (specifically for Lando, but the Docker containers referenced should work on standard Docker, too) that I published here: https://github.com/lando/lando/issues/2688#issuecomment-1229130266 I hope it helps someone else out there! -- Good luck, M1 frens

nischi commented 2 years ago

Hi @carrodher Is there any plan, roadmap for this topic? To be honest, it feels a little bit like you not really care. The most important apps has already ARM images, but bitnami can't deliver.

Would be nice that you are give us at least some plan.

Thanks

lusson-luo commented 2 years ago

mysql, redis and redis-cluster to support arm64