Helm Chart does not work on ARM

pspoerri commented 1 year ago

Steps to reproduce the problem

Steps taken to reproduce this issue:

Install the helm chart on a Kubernetes Cluster Raspberry Pi (k3s in my case)
```
helm install --namespace mastodon --create-namespace mastodon myvalues.yml
```

Helm installation fails

Error: INSTALLATION FAILED: failed post-install: timed out waiting for the condition

Dependencies fail to start:

kubectl get pods
NAME                                  READY   STATUS             RESTARTS         AGE
mastodon-assets-precompile-sk7dp      0/1     Completed          0                3h16m
mastodon-db-migrate-bv9pr             0/1     Error              0                3h15m
mastodon-db-migrate-545rk             0/1     Error              0                3h13m
mastodon-db-migrate-cnkhz             0/1     Error              0                3h11m
mastodon-db-migrate-k5mmk             0/1     Error              0                3h8m
mastodon-db-migrate-h8vmb             0/1     Error              0                3h6m
mastodon-db-migrate-fts5r             0/1     Error              0                3h3m
mastodon-db-migrate-66kpr             0/1     Error              0                3h1m
mastodon-sidekiq-f469fd769-46g86      0/1     CrashLoopBackOff   29 (3m18s ago)   3h16m
mastodon-redis-master-0               0/1     CrashLoopBackOff   43 (85s ago)     3h16m
mastodon-streaming-75b9bc7659-dpktc   0/1     CrashLoopBackOff   69 (72s ago)     3h16m
mastodon-postgresql-0                 0/1     CrashLoopBackOff   43 (77s ago)     3h16m
mastodon-web-d6459dff8-9v44t          0/1     Running            54 (30s ago)     3h16m

Error message for postgres: exec /opt/bitnami/scripts/postgresql/entrypoint.sh: exec format error

kubectl describe indicates that the AMD64 docker images are loaded:

Containers:
postgresql:
Container ID:   containerd://eec9162ed1fc16460455845cd21bc90fcc3c9872b12d0a6bac5992ea4b8b40ab
Image:          docker.io/bitnami/postgresql:14.2.0-debian-10-r14
Image ID:       docker.io/bitnami/postgresql@sha256:d0e50b7d78623f2da85420d9dfc1a31c43fe09215edff9edf2a417c3edacee1c

Expected behaviour

The mastodon helm chart works on ARM64 (e.g. Raspberry Pi) and AMD64.

Actual behaviour

Images fail to start due to exec format error failures.

Detailed description

The Helm Chart depends on bitnami images which do not support ARM64 at the moment.

The respective Bitnami Issue: https://github.com/bitnami/charts/issues/7305

Specifications

Mastodon: 290d78cea4850982a2843dc1a2954f0d66fe58d8

Target: Raspberry Pi with K3S (v1.25.3+k3s1)

ineffyble commented 1 year ago

Until Bitnami provide support, one option is to disable the Helm dependencies for postgres/redis/etc (via the appropriate values) and deploy separate, non-Bitnami instances of those components instead (and configure values with the appropriate config for Mastodon to use them).

digitalbuddha commented 1 year ago

Just a note, seeing same thing on m1 with minikube

 failed post-install: timed out waiting for the condition
INSTALLATION FAILED
main.newInstallCmd.func2
    helm.sh/helm/v3/cmd/helm/install.go:141
github.com/spf13/cobra.(*Command).execute
    github.com/spf13/cobra@v1.5.0/command.go:872
github.com/spf13/cobra.(*Command).ExecuteC
    github.com/spf13/cobra@v1.5.0/command.go:990
github.com/spf13/cobra.(*Command).Execute
    github.com/spf13/cobra@v1.5.0/command.go:918
main.main
    helm.sh/helm/v3/cmd/helm/helm.go:83
runtime.main
    runtime/proc.go:250
runtime.goexit
    runtime/asm_arm64.s:1172

magsol commented 1 year ago

@pspoerri I've been working the last couple weeks to deploy Mastodon on my own 5x Raspberry Pi k3s cluster. I've got it working, with these high-level changes:

disable all bitnami dependencies (postgres, redis, and elasticsearch) in both the values.yaml (enabled: false where appropriate) and Chart.yaml (make sure each of the three has a condition: <postgres|redis|elasticsearch>.enabled line under it).
for postgres and redis, I could still use the bitnami charts (since you now have to install these yourself, as you've disabled them in the Mastodon helm chart), with the added argument of --set image.repository="postgres|redis" to have it pull from the official image repository (and hence arm64-compatible).
for elasticsearch... stay tuned. Still haven't quite got that working.

Fortunately, the first and second points are all that's required to get Mastodon working. I have proof with my own instance right now :)

wesleyks commented 1 year ago

fyi i've been using images from ZCube/bitnami-compat which provides arm versions of bitnami's images. might be worth a peek

magsol commented 1 year ago

@wesleyks PERFECT, thank you! To be fair, I was able to get things working with zero issues for redis and postgres, but elasticsearch has been challenging because it uses a sub-chart for the bitnami-shell, which I can't seem to disable manually. It looks like ZCube's images do support bitnami-shell so I'll give that a try, thank you!!

BloodBladeW commented 1 year ago

fyi i've been using images from ZCube/bitnami-compat which provides arm versions of bitnami's images. might be worth a peek

Hey, can you tell me how I should use the images with this Chart? I just recently started using k8s and I don't fully understand the usage of these things yet.

magsol commented 1 year ago

Hey, can you tell me how I should use the images with this Chart? I just recently started using k8s and I don't fully understand the usage of these things yet.

Same! It took me a few weeks of pretty intense effort to get my own Mastodon instance up and running on my k3s cluster (same as the OP). I had to make a lot of modifications to the Helm chart--particularly around the dependencies (redis, postgres, and elasticsearch)--to get things working.

I'd be happy to try and answer your questions offline (since I think our discussion would be getting a bit off-topic from the OP) :)

BloodBladeW commented 1 year ago

Hey, can you tell me how I should use the images with this Chart? I just recently started using k8s and I don't fully understand the usage of these things yet.

Same! It took me a few weeks of pretty intense effort to get my own Mastodon instance up and running on my k3s cluster (same as the OP). I had to make a lot of modifications to the Helm chart--particularly around the dependencies (redis, postgres, and elasticsearch)--to get things working.

I'd be happy to try and answer your questions offline (since I think our discussion would be getting a bit off-topic from the OP) :)

OK, I created a Discussions, maybe we can continue over there.

ZCube commented 1 year ago

I also wanted to personally distribute mastodon, so I also tried deploying it.

Replace Bitnami dependency images. deps.yaml

postgresql:
!include https://raw.githubusercontent.com/ZCube/bitnami-compat/main/values/values_bitnami_compat_postgresql.yaml

redis:
!include https://raw.githubusercontent.com/ZCube/bitnami-compat/main/values/values_bitnami_compat_redis.yaml

elasticsearch:
!include https://raw.githubusercontent.com/ZCube/bitnami-compat/main/values/values_bitnami_compat_elasticsearch.yaml

The part marked with !include must be filled in with the contents of the file.

...
elasticsearch:
image:
digest: ''
registry: ghcr.io
repository: zcube/bitnami-compat/elasticsearch
tag: 7.17.8-debian-11-r47
kibana: null
metrics:
image:
digest: ''
...

Deploy.

helm dep update
helm install --namespace mastodon --create-namespace my-mastodon ./ -f dev-values.yaml -f deps.yaml

Result.

$ kubectl get pods -n mastodon

NAME                                             READY   STATUS    RESTARTS      AGE
my-mastodon-postgresql-0                         1/1     Running   0             15m
my-mastodon-redis-master-0                       1/1     Running   0             15m
my-mastodon-redis-replicas-0                     1/1     Running   0             15m
my-mastodon-streaming-7464c4cb84-x5t7b           1/1     Running   1 (14m ago)   15m
my-mastodon-elasticsearch-ingest-0               1/1     Running   0             15m
my-mastodon-elasticsearch-coordinating-0         1/1     Running   0             15m
my-mastodon-elasticsearch-master-0               1/1     Running   0             15m
my-mastodon-elasticsearch-data-0                 1/1     Running   0             15m
my-mastodon-web-546cfb7cb-bgl5j                  1/1     Running   2 (12m ago)   15m
my-mastodon-sidekiq-all-queues-777bb64cf-n5pmw   1/1     Running   3 (12m ago)   15m

Tested on mac mini m1 with k3d.

renchap commented 1 year ago

Duplicate of #25

See my latest comment there for more details on our plan about this!

carrodher commented 1 year ago

Hi all, we are more than happy to announce that from now on the Bitnami container catalog is available as multi-arch in Docker Hub 🎉 🎉

This means you don’t need to specify anything when pulling the container images from Docker Hub, Docker (or any other software) will automatically pull the container image matching the host platform from which the pull command was issued.

Here you can find more info about this announcement.

As usual, please create a new GitHub issue if you would like to report any bug or problem or directly contribute by creating a PR. Here you can find the contributing guidelines.

gefloh commented 1 year ago

The only thing is: That alone doesn't help when e.g. the referenced Postgresql Helm Chart is so old, that it uses an image that supports only amd64.

rtsmith commented 4 months ago

I got the latest (as of writing) version of this chart to work with Postgres 15.4, which is contained in the Bitnami chart 12.12.10. Also the latest Redis version/chart seems to work too. So for ARM support, change the deps in Chart.yaml to

  - name: postgresql
    version: 12.12.10
    repository: https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami
    condition: postgresql.enabled
  - name: redis
    version: 18.18.0
    repository: https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami
    condition: redis.enabled

mastodon / chart