search

mailcow / mailcow-dockerized

mailcow: dockerized - šŸ® + šŸ‹ = šŸ’•
https://mailcow.email
GNU General Public License v3.0
8.58k stars 1.15k forks source link

ARM64 Support #879

Closed extremeshok closed 7 months ago

extremeshok commented 6 years ago

I have extensive experience with docker on arm, so before I get to work on this, has anyone got mailcow-dockerized ported or running on arm ?

Did a quick search through the repo and I do not see any references to arm.

Does the repo owner have any issues against arm support being added ?

Thanks

Peddaahh commented 2 years ago

@Peddaahh What are you getting when you run docker-compose exec netfilter-mailcow iptables -L on the Oracle ARM VM? I keep getting: iptables v1.8.7 (legacy): can't initialize iptables table `filter': iptables who? (do you need to insmod?) Works outside the docker container obviously. I tried building netfilter for arm64v8 instead of the qemu route, iptable_filter is def loaded in the kernel image. šŸ¤” Edit: Scratch that, building netfilter for arm64v8/alpine did work, I just had the image set incorrectly in the compose file. iptables works and the container isn't restarting anymore. Zero issues with the setup since this one is fixed, surprisingly! Activesync, sogo, and everything is performing well.

This issue seems to be happen randomly.. some friends of mine complained about that too, some didn't have the issue like me... but yes building this single Container for arm seems to evade the issue... happens perhaps due to the emulation with the amd64 image..

And it is possible in more detail? What to delete? What to create? go into ./data/Dockerfiles/netfilter/ then run docker build -t mynetfilterImagename . after the build is done replace the image in your docker-compose file with the correct name of your manually built image

unixfox commented 2 years ago

If anyone is interested, I'm maintaining up-to-date docker images of mailcow that are compatible on ARM64/ARMv8, I basically automated the build of https://github.com/mailcow/mailcow-dockerized/tree/master/data/Dockerfiles using GitHub actions: https://github.com/unixfox/periodic-build-with-github-actions/blob/master/.github/workflows/docker-build-mailcow.yml

Inside the /opt/mailcow-dockerized directory, you just need to paste this content inside the file docker-compose.override.yml:

version: '2.4'
services:
  unbound-mailcow:
    image: quay.io/mailcowarm64/unbound
  clamd-mailcow:
    platform: linux/amd64
  rspamd-mailcow:
    #image: quay.io/mailcowarm64/rspamd #image patched from official dockerfile, uncomment if you want
    platform: linux/amd64
  php-fpm-mailcow:
    image: quay.io/mailcowarm64/phpfpm
  sogo-mailcow:
    platform: linux/amd64
  dovecot-mailcow:
    platform: linux/amd64
  postfix-mailcow:
    image: quay.io/mailcowarm64/postfix
  acme-mailcow:
    image: quay.io/mailcowarm64/acme
  netfilter-mailcow:
    image: quay.io/mailcowarm64/netfilter
  watchdog-mailcow:
    image: quay.io/mailcowarm64/watchdog
  dockerapi-mailcow:
    image: quay.io/mailcowarm64/dockerapi
  solr-mailcow:
    image: quay.io/mailcowarm64/solr
  olefy-mailcow:
    image: quay.io/mailcowarm64/olefy

All the docker images are uploaded there: https://quay.io/organization/mailcowarm64

Do note that you need at least QEMU 6, on Debian bullseye (tested) you can get it with Debian backports then install it using apt-get install -t bullseye-backports qemu binfmt-support qemu-user-static or use Ubuntu 22.04 (haven't tested). And at least or higher version of docker-compose 1.21, on Debian bullseye: apt install --no-install-recommends -t bullseye-backports docker-compose.

This way, this shouldn't break the update.sh mechanism of mailcow. I would highly not recommend doing that for production environment, at least with native Docker images for ARM64 you will have way less performance issue/slowdowns.

Peddaahh commented 2 years ago

If anyone is interested, I'm maintaining up-to-date docker images of mailcow that are compatible on ARM64/ARMv8, I basically automated the build of https://github.com/mailcow/mailcow-dockerized/tree/master/data/Dockerfiles using GitHub actions: https://github.com/unixfox/periodic-build-with-github-actions/blob/master/.github/workflows/docker-build-mailcow.yml

Inside the /opt/mailcow-dockerized directory, you just need to paste this content inside the file docker-compose.override.yml:

version: '2.4'
services:
  unbound-mailcow:
    image: quay.io/mailcowarm64/unbound
  clamd-mailcow:
    platform: linux/amd64
  rspamd-mailcow:
    platform: linux/amd64
  php-fpm-mailcow:
    image: quay.io/mailcowarm64/phpfpm
  sogo-mailcow:
    platform: linux/amd64
  dovecot-mailcow:
    platform: linux/amd64
  postfix-mailcow:
    image: quay.io/mailcowarm64/postfix
  acme-mailcow:
    image: quay.io/mailcowarm64/acme
  netfilter-mailcow:
    image: quay.io/mailcowarm64/netfilter
  watchdog-mailcow:
    image: quay.io/mailcowarm64/watchdog
  dockerapi-mailcow:
    image: quay.io/mailcowarm64/dockerapi
  solr-mailcow:
    image: quay.io/mailcowarm64/solr
  olefy-mailcow:
    image: quay.io/mailcowarm64/olefy

All the docker images are uploaded there: https://quay.io/organization/mailcowarm64

Do note that you need at least QEMU 6, on Debian bullseye (tested) you can get it with Debian backports then install it using apt-get install -t bullseye-backports qemu binfmt-support qemu-user-static or use Ubuntu 22.04 (haven't tested). And at least or higher version of docker-compose 1.21, on Debian bullseye: apt install --no-install-recommends -t bullseye-backports docker-compose.

This way, this shouldn't break the update.sh mechanism of mailcow. I would highly not recommend doing that for production environment, at least with native Docker images for ARM64 you will have way less performance issue/slowdowns.

I was too lazy for that but... perfect, easy and convenient solution

DerLinkman commented 1 year ago

Reopening this as i wanted to plan a possible mailcow: ARM Edition exclusively for ARM Devices (with the needed System Requirements of course).

Please let me know if anyone had a good idea so far (which i read here).

unixfox commented 1 year ago

Reopening this as i wanted to plan a possible mailcow: ARM Edition exclusively for ARM Devices (with the needed System Requirements of course).

Please let me know if anyone had a good idea so far (which i read here).

Great! I think the major step forward would be to publish the Docker images that already work for ARM64 (see https://quay.io/organization/mailcowarm64 for the list except rspamd). An update of the current github workflow is needed for that: https://github.com/mailcow/mailcow-dockerized/blob/master/.github/workflows/image_builds.yml. You can build multi architecture images through docker buildx: https://www.docker.com/blog/multi-arch-build-and-images-the-simple-way/

The remaining Docker images that won't build out of the box on ARM64 are:

Some of these images could be easily be ported to ARM64 compatibility by using Ubuntu repositories, for some components like rspamd Ubuntu provides already built binaries for ARM64: https://packages.ubuntu.com/jammy/rspamd#pdownload

DerLinkman commented 1 year ago

ClamAV, Dovecot and Rspamd are available for Alpine in ARM Versions.

So we can use them.

In general i would like to switch to Alpine based Images for ARM, as they are (at least what i expect) lightweight and more performant then the Debian/Ubuntu packages are.

SOGo does not. As this ARM Version of mailcow won't be a 1by1 copy of the mainline Project more a light version so to say. We could use Roundcube instead as a webmailer.

Unfortunately Roundcube does not have a DAV Part build in like SOGo did. So maybe we have to implement a seperate thing or even write a own thing for this.

bolet commented 1 year ago

In general i would like to switch to Alpine based Images for ARM, as they are (at least what i expect) lightweight and more performant then the Debian/Ubuntu packages are.

Totally agree. I'd rather see most images migrating to Alpine, as this would factorise the effort of testing/maintaining to a single image per service, and it would not hurt the x86 platform neither.

Unfortunately Roundcube does not have a DAV Part build in like SOGo did. So maybe we have to implement a seperate thing or even write a own thing for this.

Roundcube has a handful of extensions we could investigate. CardDav works pretty well for instance, I use this setup daily. But here again, we should consider the difficulty of maintaining an additional image (Roundcube + SoGo).

What about creating our own multiplatform SoGo images from sources? It is mostly GNUStep/Obj-C which compile on all platforms. As a matter of fact I explored this path long ago, but had no time to finish. Resource wise it was not so bad, compared to Roundcube which needs a webserver and php.

c-des commented 1 year ago 
platform: linux/amd64

I reverted back to the emulated AMD64 package due to frequent: 2023-01-22 15:58:39 #48(controller) rspamd_crash_sig_handler: caught fatal signal 11(Segmentation fault), pid: 48, trace: 2023-01-22 15:58:39 #48(controller) rspamd_print_crash: 0: 0000FFFFB5A7FD2C: 2023-01-22 15:58:39 #48(controller) rspamd_print_crash: 1: 0000000000000032:

vinogradovnet commented 1 year ago

Thank you, @unixfox !

I ran mailcow on the Oracle Cloud ARM Free Tier and it works, but sometimes .sh scripts running by ofelia on the dovecot container freeze and consume all CPU.

As a quick solution, I added timeout 10 before each script, but maybe there is a better solution?

       ofelia.job-exec.dovecot_imapsync_runner.command: "/bin/bash -c \"[[ $${MASTER} == y ]] && /usr/bin/timeout 20 /usr/local/bin/gosu nobody /usr/local/bin/imapsync_runner.pl || exit 0\""
        ofelia.job-exec.dovecot_trim_logs.command: "/bin/bash -c \"[[ $${MASTER} == y ]] && /usr/bin/timeout 20 /usr/local/bin/gosu vmail /usr/local/bin/trim_logs.sh || exit 0\""
        ofelia.job-exec.dovecot_clean_q_aged.command: "/bin/bash -c \"[[ $${MASTER} == y ]] && /usr/bin/timeout 20 /usr/local/bin/gosu vmail /usr/local/bin/clean_q_aged.sh || exit 0\""
        ofelia.job-exec.dovecot_maildir_gc.command: "/bin/bash -c \"source /source_env.sh ; /usr/bin/timeout 20 /usr/local/bin/gosu vmail /usr/local/bin/maildir_gc.sh\""
        ofelia.job-exec.dovecot_repl_health.command: "/bin/bash -c \"/usr/bin/timeout 20 /usr/local/bin/gosu vmail /usr/local/bin/repl_health.sh\""
        ofelia.job-exec.dovecot_quarantine.command: "/bin/bash -c \"[[ $${MASTER} == y ]] && /usr/bin/timeout 20 /usr/local/bin/gosu vmail /usr/local/bin/quarantine_notify.py || exit 0\""
c-des commented 1 year ago

Thank you, @unixfox !

I ran mailcow on the Oracle Cloud ARM Free Tier and it works, but sometimes .sh scripts running by ofelia on the dovecot container freeze and consume all CPU.

As a quick solution, I added timeout 10 before each script, but maybe there is a better solution?

       ofelia.job-exec.dovecot_imapsync_runner.command: "/bin/bash -c \"[[ $${MASTER} == y ]] && /usr/bin/timeout 20 /usr/local/bin/gosu nobody /usr/local/bin/imapsync_runner.pl || exit 0\""
        ofelia.job-exec.dovecot_trim_logs.command: "/bin/bash -c \"[[ $${MASTER} == y ]] && /usr/bin/timeout 20 /usr/local/bin/gosu vmail /usr/local/bin/trim_logs.sh || exit 0\""
        ofelia.job-exec.dovecot_clean_q_aged.command: "/bin/bash -c \"[[ $${MASTER} == y ]] && /usr/bin/timeout 20 /usr/local/bin/gosu vmail /usr/local/bin/clean_q_aged.sh || exit 0\""
        ofelia.job-exec.dovecot_maildir_gc.command: "/bin/bash -c \"source /source_env.sh ; /usr/bin/timeout 20 /usr/local/bin/gosu vmail /usr/local/bin/maildir_gc.sh\""
        ofelia.job-exec.dovecot_repl_health.command: "/bin/bash -c \"/usr/bin/timeout 20 /usr/local/bin/gosu vmail /usr/local/bin/repl_health.sh\""
        ofelia.job-exec.dovecot_quarantine.command: "/bin/bash -c \"[[ $${MASTER} == y ]] && /usr/bin/timeout 20 /usr/local/bin/gosu vmail /usr/local/bin/quarantine_notify.py || exit 0\""

Can confirm the same issue on OCI!

shoopea commented 1 year ago

Thank you, @unixfox ! I ran mailcow on the Oracle Cloud ARM Free Tier and it works, but sometimes .sh scripts running by ofelia on the dovecot container freeze and consume all CPU. As a quick solution, I added timeout 10 before each script, but maybe there is a better solution?

       ofelia.job-exec.dovecot_imapsync_runner.command: "/bin/bash -c \"[[ $${MASTER} == y ]] && /usr/bin/timeout 20 /usr/local/bin/gosu nobody /usr/local/bin/imapsync_runner.pl || exit 0\""
        ofelia.job-exec.dovecot_trim_logs.command: "/bin/bash -c \"[[ $${MASTER} == y ]] && /usr/bin/timeout 20 /usr/local/bin/gosu vmail /usr/local/bin/trim_logs.sh || exit 0\""
        ofelia.job-exec.dovecot_clean_q_aged.command: "/bin/bash -c \"[[ $${MASTER} == y ]] && /usr/bin/timeout 20 /usr/local/bin/gosu vmail /usr/local/bin/clean_q_aged.sh || exit 0\""
        ofelia.job-exec.dovecot_maildir_gc.command: "/bin/bash -c \"source /source_env.sh ; /usr/bin/timeout 20 /usr/local/bin/gosu vmail /usr/local/bin/maildir_gc.sh\""
        ofelia.job-exec.dovecot_repl_health.command: "/bin/bash -c \"/usr/bin/timeout 20 /usr/local/bin/gosu vmail /usr/local/bin/repl_health.sh\""
        ofelia.job-exec.dovecot_quarantine.command: "/bin/bash -c \"[[ $${MASTER} == y ]] && /usr/bin/timeout 20 /usr/local/bin/gosu vmail /usr/local/bin/quarantine_notify.py || exit 0\""

Can confirm the same issue on OCI!

I confirm aswell, my current fix was to roll back to dovecot 2.3.19.1 via mailcow/dovecot:1.21 which still works fine. edit : Sogo is deactivated on my setup, so as mentionned by @vinogradovnet below, use mailcow/sogo:1.114 instead of mailcow/sogo:1.115

vinogradovnet commented 1 year ago

Thank you, @shoopea!

    dovecot-mailcow:
      platform: linux/amd64
      image: mailcow/dovecot:1.21

This works for me also.

Another reason for official arm images ;)

vinogradovnet commented 1 year ago

Same problem after last upgrade. Right now with mailcow/sogo:1.115 image. Downgrade to mailcow/sogo:1.114 resolved problem.

    sogo-mailcow:
      platform: linux/amd64
      image: mailcow/sogo:1.114

Both new dovecot and sogo images use GOSU_VERSION 1.16, but working images use GOSU_VERSION 1.14

Maybe this is issue.

c-des commented 1 year ago

I downgraded to 1.21/1.114 after getting too many of those:

Out of memory: Killed process 2722 (ofelia) total-vm:12899740kB, anon-rss:11018228kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:21808kB oom_score_adj:0

jsschmid commented 1 year ago

SOGo

The issue with SoGo is a none issue according to my "intensive research"

ubuntu@vps-********:/opt/mailcow-dockerized# apt-cache search sogo-common
sogo-common - Scalable groupware server - common files
ubuntu@vps-********:/opt/mailcow-dockerized# uname -a
Linux vps-******** 5.15.0-1030-oracle #36-Ubuntu SMP Wed Feb 15 05:57:14 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux
marfeljoergsen commented 1 year ago

For ARM-cpu on OCI: I have many of the issues described here and tried some of the solutions, e.g. downgrading Sogo and sogo-mailcow. But 4 of my containers keep restarting:

6235f150e0a5   mailcow/rspamd:1.92              "/docker-entrypoint.ā€¦"   2 minutes ago   Restarting (1) 17 seconds ago    mailcowdockerized-rspamd-mailcow-1
9006e32e36c0   mailcow/dovecot:1.21             "/docker-entrypoint.ā€¦"   2 minutes ago   Restarting (1) 14 seconds ago    mailcowdockerized-dovecot-mailcow-1
81b6ea77a80b   mailcow/clamd:1.61               "/sbin/tini -g -- /cā€¦"   2 minutes ago   Restarting (1) 21 seconds ago    mailcowdockerized-clamd-mailcow-1
269b4760a4ce   mailcow/sogo:1.114               "/docker-entrypoint.ā€¦"   2 minutes ago   Restarting (1) 25 seconds ago    mailcowdockerized-sogo-mailcow-1

Furthermore, before downgrading I also got the frequent _"rspamd_crash_sighandler: caught fatal signal 11(Segmentation fault), pid: 4"-problems. If I go to the https://(mailserver-IP)/SOGo/ I receive: "What is happening? Please do not stop the stack while we are initializing the database or do other preparations. What is happening? - Nginx cannot connect to an upstream server or other services are not ready yet. This is fine if mailcow was just installed or updated and can take a few minutes to complete."...

A bit from my logs:

mailcowdockerized-ofelia-mailcow-1  | 2023-03-26T15:05:04.02+02:00  common.go:121 ā–¶ ERROR [Job "sogo_ealarms" (29c29659d8cb)] Finished in "19.00723ms", failed: true, skipped: false, error: error creating exec: API error (409): Container 269b4760a4ced7e353768970611553d20a4b0f4977ff59dfee0889a7e2a10bcc is restarting, wait until the container is running
mailcowdockerized-ofelia-mailcow-1  | 2023-03-26T15:05:04.031+02:00  common.go:121 ā–¶ ERROR [Job "dovecot_imapsync_runner" (74ed07014ae2)] Finished in "25.214146ms", failed: true, skipped: false, error: error creating exec: API error (409): Container 9006e32e36c087f11476446ca876eee7ebdbd84d6198ce0ea0b4b04a2d7e0fc4 is restarting, wait until the container is running
mailcowdockerized-ofelia-mailcow-1  | 2023-03-26T15:05:04.032+02:00  common.go:121 ā–¶ ERROR [Job "sogo_sessions" (387248c521d9)] Finished in "30.107493ms", failed: true, skipped: false, error: error creating exec: API error (409): Container 269b4760a4ced7e353768970611553d20a4b0f4977ff59dfee0889a7e2a10bcc is restarting, wait until the container is running
mailcowdockerized-ofelia-mailcow-1  | 2023-03-26T15:05:04.033+02:00  common.go:121 ā–¶ ERROR [Job "dovecot_trim_logs" (9bdf2e558745)] Finished in "27.720881ms", failed: true, skipped: false, error: error creating exec: API error (409): Container 9006e32e36c087f11476446ca876eee7ebdbd84d6198ce0ea0b4b04a2d7e0fc4 is restarting, wait until the container is running
...
mailcowdockerized-rspamd-mailcow-1  | exec /docker-entrypoint.sh: exec format error
...
mailcowdockerized-clamd-mailcow-1      | exec /sbin/tini: exec format error
...
mailcowdockerized-sogo-mailcow-1       | exec /docker-entrypoint.sh: exec format error
...
mailcowdockerized-watchdog-mailcow-1   | CRITICAL - Socket timeout
...
mailcowdockerized-acme-mailcow-1       | ping: bad address 'dovecot'
...
mailcowdockerized-dovecot-mailcow-1    | exec /docker-entrypoint.sh: exec format error

I might have made mistakes, but don't feel this is really stable enough for running on ARM with OCI... I'm not sure what I'll do, but googled for an alternative and found e.g. https://github.com/manfromdownunder/mailserver-arm which I might try - also I've only tried master branch instead of nightly, so there a few options, but would prefer ARM-support worked more "straight-out-of-the-box"...

jebabin commented 1 year ago

marfeljoergsen have you done apt-get install qemu binfmt-support qemu-user-static ?

marfeljoergsen commented 1 year ago

@jebabin Oh, thanks a lot, I missed that part! It seems I had to do: apt install qemu-system-arm binfmt-support qemu-user-static - This works much better, now my containers aren't restarting the whole time and the amount of error messages is much lower.

In my docker-compose.override.yml I've specified to use mailcow/sogo:1.114 and mailcow/dovecot:1.21. I've setup domain and added email, but if I try to access http://(mydomain.com)/SOGo I get "What is happening? Please do not stop the stack while we are initializing the database or do other preparations" - this might however be because I'm only testing so I haven't made any DNS-changes, hence mail.(mydomain).com doesn't point to my IP... hmm, I'll play more with it tomorrow and probably I should redirect my noob-questions to the forum at https://community.mailcow.email/ - thanks for now, this looks a lot better!

dragoangel commented 1 year ago

for some components like rspamd Ubuntu provides already built binaries for ARM64: https://packages.ubuntu.com/jammy/rspamd#pdownload

@unifox it should not be taken as a base, as the Rspamd package version that usual OS repos provide is critically old, and this is not only about Rspamd. In such cases need to build them from the source to desired architecture and pack them inside of the docker image. Same can be said about SOGo, but about clamav and dovecot - it really can be taken from OS repos. General problem with OS repos that it's not getting updates as it should sometimes, and because of it generally anyway it will end up in building anything from code which resource and time consuming.

andryyy commented 1 year ago

What @dragoangel says. :)

Vsevo does also advises against it, these are non supported builds. You might want to ask him for support in his repo. I don't think he is working on it atm.

NLion74 commented 1 year ago

+1 for arm64 support. Been thinking about migrating my mailserver to Hetzner's new arm64 servers, as it costs nearly half of what I pay now.

KGBist2000 commented 1 year ago

+1, please introduce an official mailcow Package with arm64 Support.

matelush commented 1 year ago

+1, official support for arm64 would be like cheesecake without raisins

aleksasiriski commented 1 year ago

Hetzner introduced CAX servers (Ampere Altra which is ARM64) which now go for cheaper than their Intel and Amd servers and come with double the RAM. ARM64 support is a must in today's world.

rflrkn commented 1 year ago

+1 - thought about migrating to a Hetzner arm vps, and mailcow is basically the only thing holding me back.

DerLinkman commented 1 year ago

I'll start on testing/building mailcow for ARM probably the next day on a hetzner ARM Server.

funkyfilip commented 1 year ago

+1

DerLinkman commented 1 year ago

RSPAMD + CLAMAV have been ported to ARM with minimal changes to the main Dockerfile so that we can keep it inside the main repo.

Regarding Dovecot is a problem right now... we actually have to compile it for ARM64 to make it usable. This works but i'm not sure if i feel happy about it.

SOGo the same. But i've opened up a ticket at Alinto (which you guys could upvote, would be nice btw: https://bugs.sogo.nu/view.php?id=5754) maybe they'll add a native ARM64 precompiled package in their repo if we make enough "noise" you know what i mean. However if not, we'll probably continue with the compiling route of those subcontainers which are not natively ARM64 compatible.

DerLinkman commented 1 year ago

Status Update (2nd May 2023)

mailcow has been sucessfully ported to ARM64 without modifiying the general code base (except the Dockerfiles, but those work on x86_64 the same as they do on ARM64).

As a small bonus i've added a little architecture information to the status page which will be live with the first release of May 2023. (Screenshot below) grafik

Now begins the testing phase.

Which i'll invite anyone to which if interested. I'm not sure if i want to integrate this into nightly already. If i've decided on how to continue i'll write it here.

Anyways: Hopefully i'll get the first testing versions out this week.

jebabin commented 1 year ago

Thanks @DerLinkman, I tried your changes for arm support and I got everything to work fine except rspamd. Rspamd crash with a segment fault during startup

2023-05-03 16:16:03 #60(main) <6uyjnm>; cfg; rspamd_config_is_module_enabled: lua module rspamd_update is disabled in the configuration
2023-05-03 16:16:03 #60(main) <oxhd99>; map; rspamd_map_add: added map http://nginx:8081/settings.php
2023-05-03 16:16:03 #60(main) <6uyjnm>; cfg; rspamd_init_lua_filters: init lua module settings from /usr/share/rspamd/plugins/settings.lua; digest: 094d643b66
2023-05-03 16:16:03 #60(main) <6uyjnm>; lua; spamassassin.lua:1714: loading SA rules from /etc/rspamd/custom/sa-rules
Segmentation fault (core dumped)

I'm running on Oracle Cloud infrastructure and did not made any change to the rspamd conf in the data directory If anyone else is running on OCI, can you tell if you encounter the same issue ?

DerLinkman commented 1 year ago

Did you've build the images by yourself? Which you have as I didn't published ARM images on docker hub.

jebabin commented 1 year ago

Did you've build the images by yourself? Which you have as I didn't published ARM images on docker hub.

Yes i rebuilt all images from feat/arm64-cow Edit: Thanks to @unixfox, I did not notice you did not update the Dockerfile for rspamd, after replacing [arch=amd64] by [arch=arm64] in the Dockefile, it's working well

unixfox commented 1 year ago

Like my comment in https://github.com/mailcow/mailcow-dockerized/issues/879#issuecomment-1037241696, I just published the test docker images from the PR https://github.com/mailcow/mailcow-dockerized/pull/5219.

The images are available under the docker tag arm64-cow-latest and arm64-cow-COMMITID. Automatically updated when a new commit is out. The images will be available for 3 months, probably enough to have the official support before that. Link to the docker images: https://quay.io/organization/mailcowarm64 It's targeted towards enthusiasts that want to do some early testing.

rspamd doesn't seem to have an ARM64 docker image for it though, no new dockerfile images are available in the PR https://github.com/mailcow/mailcow-dockerized/pull/5219/files.

jebabin commented 1 year ago

rspamd doesn't seem to have an ARM64 docker image for it though, no new dockerfile images are available in the PR https://github.com/mailcow/mailcow-dockerized/pull/5219/files.

You can remplace [arch=amd64] by [arch=arm64] in the Dockerfile and you will get rspamd working

BrianHenryIE commented 1 year ago

I'm trying this out now. It's up and running on Raspberry Pi 4 8GB, alongside NextCloud, and I'm configuring my first domain.

sudo su
umask
# 0022 # <- Verify it is 0022
cd /opt
git clone https://github.com/mailcow/mailcow-dockerized
cd mailcow-dockerized
git checkout feat/arm64-cow
nano generate_config.sh // Commented out some git commands so it doesn't change away from this branch, maybe unnecessary
nano data/Dockerfiles/rspamd/Dockerfile //  Replace [arch=amd64] with [arch=arm64] 
nano docker-compose.override.yml // Paste in contents of #879
./generate_config.sh
nano mailcow.conf // Change ports etc.
docker compose pull
docker compose up -d

Thanks for all the work ye put in. I just bought a Pi this week to host a mailserver, and very luckily the ARM work was done moments before I started reading the documentation!

DerLinkman commented 1 year ago

Thanks @DerLinkman, I tried your changes for arm support and I got everything to work fine except rspamd. Rspamd crash with a segment fault during startup

2023-05-03 16:16:03 #60(main) <6uyjnm>; cfg; rspamd_config_is_module_enabled: lua module rspamd_update is disabled in the configuration
2023-05-03 16:16:03 #60(main) <oxhd99>; map; rspamd_map_add: added map http://nginx:8081/settings.php
2023-05-03 16:16:03 #60(main) <6uyjnm>; cfg; rspamd_init_lua_filters: init lua module settings from /usr/share/rspamd/plugins/settings.lua; digest: 094d643b66
2023-05-03 16:16:03 #60(main) <6uyjnm>; lua; spamassassin.lua:1714: loading SA rules from /etc/rspamd/custom/sa-rules
Segmentation fault (core dumped)

I'm running on Oracle Cloud infrastructure and did not made any change to the rspamd conf in the data directory If anyone else is running on OCI, can you tell if you encounter the same issue ?

Hi,

thanks to my colleague i was able to confirm this strange behaviour on Oracles Cloud Infrastructure.

I've personally build the images (and tested them) on a Hetzner ARM64 Server which worked flawlessly so far.

Can you give us more informations about your ARM64 VPS? Like CPU Cores, CPU Model (can be seen with lscpu), RAM Amount, Virtualisation Platform (can be seen with systemd-detect-virt)

DerLinkman commented 1 year ago

OMG i'm so dumb sorry.... i missed the AMD64 Tag inside the Rspamd Image.... sorry guys fix is deployed...

DerLinkman commented 1 year ago

Hey everyone i've published the official test images to docker hub.

As this is a testing phase i've also remapped the image versions inside the docker-compose.yml to arm64-dev instead of a version.

This change is made with the latest commit.

You can now try it out if you want. And yes if someone asks: RSPAMD is also running on ARM64 now without a change needed (thanks for the reminder guys).

qy117121 commented 1 year ago

Hey everyone i've published the official test images to docker hub.

As this is a testing phase i've also remapped the image versions inside the docker-compose.yml to arm64-dev instead of a version.

This change is made with the latest commit.

You can now try it out if you want. And yes if someone asks: RSPAMD is also running on ARM64 now without a change needed (thanks for the reminder guys).

https://github.com/mailcow/mailcow-dockerized/blob/a2693c3b94a6c0ffe97286a21060e4413b06efc2/docker-compose.yml#LL531C3-L531C3

image: mailcow/solr:dev
missing arm64-

KeldorDE commented 1 year ago

Same here! Branch: feat/arm64-cow

docker compose pull

[+] Running 19/19
āœ˜ redis-mailcow Error                                                                                                            1.2s
āœ˜ watchdog-mailcow Error                                                                                                         1.2s
āœ˜ netfilter-mailcow Error                                                                                                        1.2s
āœ˜ ofelia-mailcow Error                                                                                                           1.2s
āœ˜ php-fpm-mailcow Error                                                                                                          1.2s
āœ˜ mysql-mailcow Error                                                                                                            1.2s
āœ˜ dovecot-mailcow Error                                                                                                          1.2s
āœ˜ ipv6nat-mailcow Error                                                                                                          1.2s
āœ˜ dockerapi-mailcow Error                                                                                                        1.2s
āœ˜ nginx-mailcow Error                                                                                                            1.2s
āœ˜ solr-mailcow Error                                                                                                             1.2s
āœ˜ olefy-mailcow Error                                                                                                            1.2s
āœ˜ postfix-mailcow Error                                                                                                          1.2s
āœ˜ rspamd-mailcow Error                                                                                                           1.2s
āœ˜ unbound-mailcow Error                                                                                                          1.2s
āœ˜ clamd-mailcow Error                                                                                                            1.2s
āœ˜ memcached-mailcow Error                                                                                                        1.2s
āœ˜ sogo-mailcow Error                                                                                                             1.2s
āœ˜ acme-mailcow Error                                                                                                             1.2s
Error response from daemon: manifest for mailcow/solr:dev not found: manifest unknown: manifest unknown
KeldorDE commented 1 year ago

It looks like in the docker-compose.yml the tag of Solr has to be set to arm64-dev. After that you can pull the images without problems.

KeldorDE commented 1 year ago

Another error running the restore script. Seems like the backup image has no arm64 build yet.

Unable to find image 'mailcow/backup:latest' locally
latest: Pulling from mailcow/backup
docker: no matching manifest for linux/arm64/v8 in the manifest list entries.
See 'docker run --help'.
ef03e7eb35ed
0f6198c5e897
Unable to find image 'mailcow/backup:latest' locally
latest: Pulling from mailcow/backup
docker: no matching manifest for linux/arm64/v8 in the manifest list entries.
See 'docker run --help'.
0f6198c5e897
DerLinkman commented 1 year ago

Another error running the restore script. Seems like the backup image has no arm64 build yet.

Unable to find image 'mailcow/backup:latest' locally
latest: Pulling from mailcow/backup
docker: no matching manifest for linux/arm64/v8 in the manifest list entries.
See 'docker run --help'.
ef03e7eb35ed
0f6198c5e897
Unable to find image 'mailcow/backup:latest' locally
latest: Pulling from mailcow/backup
docker: no matching manifest for linux/arm64/v8 in the manifest list entries.
See 'docker run --help'.
0f6198c5e897

Yeah the backup and restore functionality has not been tested yet so that didn't work natively yet.

You can build that image by yourself though and use it that way.

DerLinkman commented 1 year ago

Hey everyone i've published the official test images to docker hub. As this is a testing phase i've also remapped the image versions inside the docker-compose.yml to arm64-dev instead of a version. This change is made with the latest commit. You can now try it out if you want. And yes if someone asks: RSPAMD is also running on ARM64 now without a change needed (thanks for the reminder guys).

https://github.com/mailcow/mailcow-dockerized/blob/a2693c3b94a6c0ffe97286a21060e4413b06efc2/docker-compose.yml#LL531C3-L531C3

image: mailcow/solr:dev missing arm64-

Ah damn... we have to much containers... :)

Should be fixed now!

KeldorDE commented 1 year ago

I have now started mailcow with docker compose up, however something does not seem to be working properly.

The dockerapi container is constantly restarting:

mailcowdockerized-dockerapi-mailcow-1  | Traceback (most recent call last):
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/api/client.py", line 214, in _retrieve_server_version
mailcowdockerized-dockerapi-mailcow-1  |     return self.version(api_version=False)["ApiVersion"]
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/api/daemon.py", line 181, in version
mailcowdockerized-dockerapi-mailcow-1  |     return self._result(self._get(url), json=True)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/utils/decorators.py", line 46, in inner
mailcowdockerized-dockerapi-mailcow-1  |     return f(self, *args, **kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/api/client.py", line 237, in _get
mailcowdockerized-dockerapi-mailcow-1  |     return self.get(url, **self._set_request_timeout(kwargs))
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/requests/sessions.py", line 600, in get
mailcowdockerized-dockerapi-mailcow-1  |     return self.request("GET", url, **kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
mailcowdockerized-dockerapi-mailcow-1  |     resp = self.send(prep, **send_kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
mailcowdockerized-dockerapi-mailcow-1  |     r = adapter.send(request, **kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/requests/adapters.py", line 486, in send
mailcowdockerized-dockerapi-mailcow-1  |     resp = conn.urlopen(
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/urllib3/connectionpool.py", line 790, in urlopen
mailcowdockerized-dockerapi-mailcow-1  |     response = self._make_request(
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/urllib3/connectionpool.py", line 496, in _make_request
mailcowdockerized-dockerapi-mailcow-1  |     conn.request(
mailcowdockerized-dockerapi-mailcow-1  | TypeError: HTTPConnection.request() got an unexpected keyword argument 'chunked'
mailcowdockerized-dockerapi-mailcow-1  |
mailcowdockerized-dockerapi-mailcow-1  | During handling of the above exception, another exception occurred:
mailcowdockerized-dockerapi-mailcow-1  |
mailcowdockerized-dockerapi-mailcow-1  | Traceback (most recent call last):
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/bin/uvicorn", line 8, in <module>
mailcowdockerized-dockerapi-mailcow-1  |     sys.exit(main())
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
mailcowdockerized-dockerapi-mailcow-1  |     return self.main(*args, **kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/click/core.py", line 1055, in main
mailcowdockerized-dockerapi-mailcow-1  |     rv = self.invoke(ctx)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
mailcowdockerized-dockerapi-mailcow-1  |     return ctx.invoke(self.callback, **ctx.params)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/click/core.py", line 760, in invoke
mailcowdockerized-dockerapi-mailcow-1  |     return __callback(*args, **kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/uvicorn/main.py", line 410, in main
mailcowdockerized-dockerapi-mailcow-1  |     run(
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/uvicorn/main.py", line 578, in run
mailcowdockerized-dockerapi-mailcow-1  |     server.run()
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/uvicorn/server.py", line 61, in run
mailcowdockerized-dockerapi-mailcow-1  |     return asyncio.run(self.serve(sockets=sockets))
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
mailcowdockerized-dockerapi-mailcow-1  |     return loop.run_until_complete(main)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
mailcowdockerized-dockerapi-mailcow-1  |     return future.result()
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/uvicorn/server.py", line 68, in serve
mailcowdockerized-dockerapi-mailcow-1  |     config.load()
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/uvicorn/config.py", line 473, in load
mailcowdockerized-dockerapi-mailcow-1  |     self.loaded_app = import_from_string(self.app)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/uvicorn/importer.py", line 21, in import_from_string
mailcowdockerized-dockerapi-mailcow-1  |     module = importlib.import_module(module_str)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
mailcowdockerized-dockerapi-mailcow-1  |     return _bootstrap._gcd_import(name[level:], package, level)
mailcowdockerized-dockerapi-mailcow-1  |   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
mailcowdockerized-dockerapi-mailcow-1  |   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
mailcowdockerized-dockerapi-mailcow-1  |   File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
mailcowdockerized-dockerapi-mailcow-1  |   File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
mailcowdockerized-dockerapi-mailcow-1  |   File "<frozen importlib._bootstrap_external>", line 883, in exec_module
mailcowdockerized-dockerapi-mailcow-1  |   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
mailcowdockerized-dockerapi-mailcow-1  |   File "/app/dockerapi.py", line 546, in <module>
mailcowdockerized-dockerapi-mailcow-1  |     sync_docker_client = docker.DockerClient(base_url='unix://var/run/docker.sock', version='auto')
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/client.py", line 45, in __init__
mailcowdockerized-dockerapi-mailcow-1  |     self.api = APIClient(*args, **kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/api/client.py", line 197, in __init__
mailcowdockerized-dockerapi-mailcow-1  |     self._version = self._retrieve_server_version()
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/api/client.py", line 221, in _retrieve_server_version
mailcowdockerized-dockerapi-mailcow-1  |     raise DockerException(
mailcowdockerized-dockerapi-mailcow-1  | docker.errors.DockerException: Error while fetching server API version: HTTPConnection.request() got an unexpected keyword argument 'chunked'
mailcowdockerized-dockerapi-mailcow-1 exited with code 1

The dovecot container also reports the following since 10 minutes ago: mailcowdockerized-dovecot-mailcow-1 | Waiting for versions table to be created...

And the Rspamd Container is reporting this: mailcowdockerized-rspamd-mailcow-1 | Waiting for PHP on port 9001...

Am I doing something wrong? I specially created a fresh ARM64 instance at Hetzner and did a new installation.

vh133 commented 1 year ago

I have now started mailcow with docker compose up, however something does not seem to be working properly.

The dockerapi container is constantly restarting:

mailcowdockerized-dockerapi-mailcow-1  | Traceback (most recent call last):
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/api/client.py", line 214, in _retrieve_server_version
mailcowdockerized-dockerapi-mailcow-1  |     return self.version(api_version=False)["ApiVersion"]
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/api/daemon.py", line 181, in version
mailcowdockerized-dockerapi-mailcow-1  |     return self._result(self._get(url), json=True)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/utils/decorators.py", line 46, in inner
mailcowdockerized-dockerapi-mailcow-1  |     return f(self, *args, **kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/api/client.py", line 237, in _get
mailcowdockerized-dockerapi-mailcow-1  |     return self.get(url, **self._set_request_timeout(kwargs))
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/requests/sessions.py", line 600, in get
mailcowdockerized-dockerapi-mailcow-1  |     return self.request("GET", url, **kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
mailcowdockerized-dockerapi-mailcow-1  |     resp = self.send(prep, **send_kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
mailcowdockerized-dockerapi-mailcow-1  |     r = adapter.send(request, **kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/requests/adapters.py", line 486, in send
mailcowdockerized-dockerapi-mailcow-1  |     resp = conn.urlopen(
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/urllib3/connectionpool.py", line 790, in urlopen
mailcowdockerized-dockerapi-mailcow-1  |     response = self._make_request(
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/urllib3/connectionpool.py", line 496, in _make_request
mailcowdockerized-dockerapi-mailcow-1  |     conn.request(
mailcowdockerized-dockerapi-mailcow-1  | TypeError: HTTPConnection.request() got an unexpected keyword argument 'chunked'
mailcowdockerized-dockerapi-mailcow-1  |
mailcowdockerized-dockerapi-mailcow-1  | During handling of the above exception, another exception occurred:
mailcowdockerized-dockerapi-mailcow-1  |
mailcowdockerized-dockerapi-mailcow-1  | Traceback (most recent call last):
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/bin/uvicorn", line 8, in <module>
mailcowdockerized-dockerapi-mailcow-1  |     sys.exit(main())
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
mailcowdockerized-dockerapi-mailcow-1  |     return self.main(*args, **kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/click/core.py", line 1055, in main
mailcowdockerized-dockerapi-mailcow-1  |     rv = self.invoke(ctx)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
mailcowdockerized-dockerapi-mailcow-1  |     return ctx.invoke(self.callback, **ctx.params)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/click/core.py", line 760, in invoke
mailcowdockerized-dockerapi-mailcow-1  |     return __callback(*args, **kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/uvicorn/main.py", line 410, in main
mailcowdockerized-dockerapi-mailcow-1  |     run(
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/uvicorn/main.py", line 578, in run
mailcowdockerized-dockerapi-mailcow-1  |     server.run()
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/uvicorn/server.py", line 61, in run
mailcowdockerized-dockerapi-mailcow-1  |     return asyncio.run(self.serve(sockets=sockets))
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
mailcowdockerized-dockerapi-mailcow-1  |     return loop.run_until_complete(main)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
mailcowdockerized-dockerapi-mailcow-1  |     return future.result()
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/uvicorn/server.py", line 68, in serve
mailcowdockerized-dockerapi-mailcow-1  |     config.load()
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/uvicorn/config.py", line 473, in load
mailcowdockerized-dockerapi-mailcow-1  |     self.loaded_app = import_from_string(self.app)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/uvicorn/importer.py", line 21, in import_from_string
mailcowdockerized-dockerapi-mailcow-1  |     module = importlib.import_module(module_str)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
mailcowdockerized-dockerapi-mailcow-1  |     return _bootstrap._gcd_import(name[level:], package, level)
mailcowdockerized-dockerapi-mailcow-1  |   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
mailcowdockerized-dockerapi-mailcow-1  |   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
mailcowdockerized-dockerapi-mailcow-1  |   File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
mailcowdockerized-dockerapi-mailcow-1  |   File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
mailcowdockerized-dockerapi-mailcow-1  |   File "<frozen importlib._bootstrap_external>", line 883, in exec_module
mailcowdockerized-dockerapi-mailcow-1  |   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
mailcowdockerized-dockerapi-mailcow-1  |   File "/app/dockerapi.py", line 546, in <module>
mailcowdockerized-dockerapi-mailcow-1  |     sync_docker_client = docker.DockerClient(base_url='unix://var/run/docker.sock', version='auto')
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/client.py", line 45, in __init__
mailcowdockerized-dockerapi-mailcow-1  |     self.api = APIClient(*args, **kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/api/client.py", line 197, in __init__
mailcowdockerized-dockerapi-mailcow-1  |     self._version = self._retrieve_server_version()
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/api/client.py", line 221, in _retrieve_server_version
mailcowdockerized-dockerapi-mailcow-1  |     raise DockerException(
mailcowdockerized-dockerapi-mailcow-1  | docker.errors.DockerException: Error while fetching server API version: HTTPConnection.request() got an unexpected keyword argument 'chunked'
mailcowdockerized-dockerapi-mailcow-1 exited with code 1

The dovecot container also reports the following since 10 minutes ago: mailcowdockerized-dovecot-mailcow-1 | Waiting for versions table to be created...

And the Rspamd Container is reporting this: mailcowdockerized-rspamd-mailcow-1 | Waiting for PHP on port 9001...

Am I doing something wrong? I specially created a fresh ARM64 instance at Hetzner and did a new installation.

I have exactly the same problem. (Hetzner Cloud, arm64, Ubuntu 22.04) I think the error is somewhere in the initialization of the database because (at least for me) there are no tables in the mailcow DB. But so far I could not determine exactly where the error happens.

derolidev commented 1 year ago

Just tried to install Mailcow on my arm64 VPS in Hetzner's Cloud and I am facing the same problems as mentioned above with dockerapi. Container keeps constantly restarting. In addition to dockerapi, container sogo seems to have some trouble, too.

mailcowdockerized-sogo-mailcow-1       | Waiting for schema update...
mailcowdockerized-sogo-mailcow-1       | ERROR 1146 (42S02) at line 1: Table 'mailcow.versions' doesn't exist
mailcowdockerized-dockerapi-mailcow-1  | -----
mailcowdockerized-dockerapi-mailcow-1  | Traceback (most recent call last):
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/api/client.py", line 214, in _retrieve_server_version
mailcowdockerized-dockerapi-mailcow-1  |     return self.version(api_version=False)["ApiVersion"]
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/api/daemon.py", line 181, in version
mailcowdockerized-dockerapi-mailcow-1  |     return self._result(self._get(url), json=True)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/utils/decorators.py", line 46, in inner
mailcowdockerized-dockerapi-mailcow-1  |     return f(self, *args, **kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/api/client.py", line 237, in _get
mailcowdockerized-dockerapi-mailcow-1  |     return self.get(url, **self._set_request_timeout(kwargs))
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/requests/sessions.py", line 600, in get
mailcowdockerized-dockerapi-mailcow-1  |     return self.request("GET", url, **kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
mailcowdockerized-dockerapi-mailcow-1  |     resp = self.send(prep, **send_kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
mailcowdockerized-dockerapi-mailcow-1  |     r = adapter.send(request, **kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/requests/adapters.py", line 486, in send
mailcowdockerized-dockerapi-mailcow-1  |     resp = conn.urlopen(
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/urllib3/connectionpool.py", line 790, in urlopen
mailcowdockerized-dockerapi-mailcow-1  |     response = self._make_request(
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/urllib3/connectionpool.py", line 496, in _make_request
mailcowdockerized-dockerapi-mailcow-1  |     conn.request(
mailcowdockerized-dockerapi-mailcow-1  | TypeError: HTTPConnection.request() got an unexpected keyword argument 'chunked'
mailcowdockerized-dockerapi-mailcow-1  | 
mailcowdockerized-dockerapi-mailcow-1  | During handling of the above exception, another exception occurred:
mailcowdockerized-dockerapi-mailcow-1  | 
mailcowdockerized-dockerapi-mailcow-1  | Traceback (most recent call last):
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/bin/uvicorn", line 8, in <module>
mailcowdockerized-dockerapi-mailcow-1  |     sys.exit(main())
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
mailcowdockerized-dockerapi-mailcow-1  |     return self.main(*args, **kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/click/core.py", line 1055, in main
mailcowdockerized-dockerapi-mailcow-1  |     rv = self.invoke(ctx)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
mailcowdockerized-dockerapi-mailcow-1  |     return ctx.invoke(self.callback, **ctx.params)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/click/core.py", line 760, in invoke
mailcowdockerized-dockerapi-mailcow-1  |     return __callback(*args, **kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/uvicorn/main.py", line 410, in main
mailcowdockerized-dockerapi-mailcow-1  |     run(
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/uvicorn/main.py", line 578, in run
mailcowdockerized-dockerapi-mailcow-1  |     server.run()
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/uvicorn/server.py", line 61, in run
mailcowdockerized-dockerapi-mailcow-1  |     return asyncio.run(self.serve(sockets=sockets))
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
mailcowdockerized-dockerapi-mailcow-1  |     return loop.run_until_complete(main)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
mailcowdockerized-dockerapi-mailcow-1  |     return future.result()
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/uvicorn/server.py", line 68, in serve
mailcowdockerized-dockerapi-mailcow-1  |     config.load()
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/uvicorn/config.py", line 473, in load
mailcowdockerized-dockerapi-mailcow-1  |     self.loaded_app = import_from_string(self.app)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/uvicorn/importer.py", line 21, in import_from_string
mailcowdockerized-dockerapi-mailcow-1  |     module = importlib.import_module(module_str)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
mailcowdockerized-dockerapi-mailcow-1  |     return _bootstrap._gcd_import(name[level:], package, level)
mailcowdockerized-dockerapi-mailcow-1  |   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
mailcowdockerized-dockerapi-mailcow-1  |   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
mailcowdockerized-dockerapi-mailcow-1  |   File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
mailcowdockerized-dockerapi-mailcow-1  |   File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
mailcowdockerized-dockerapi-mailcow-1  |   File "<frozen importlib._bootstrap_external>", line 883, in exec_module
mailcowdockerized-dockerapi-mailcow-1  |   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
mailcowdockerized-dockerapi-mailcow-1  |   File "/app/dockerapi.py", line 546, in <module>
mailcowdockerized-dockerapi-mailcow-1  |     sync_docker_client = docker.DockerClient(base_url='unix://var/run/docker.sock', version='auto')
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/client.py", line 45, in __init__
mailcowdockerized-dockerapi-mailcow-1  |     self.api = APIClient(*args, **kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/api/client.py", line 197, in __init__
mailcowdockerized-dockerapi-mailcow-1  |     self._version = self._retrieve_server_version()
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/api/client.py", line 221, in _retrieve_server_version
mailcowdockerized-dockerapi-mailcow-1  |     raise DockerException(
mailcowdockerized-dockerapi-mailcow-1  | docker.errors.DockerException: Error while fetching server API version: HTTPConnection.request() got an unexpected keyword argument 'chunked'
mailcowdockerized-dockerapi-mailcow-1 exited with code 1
dasn4pp3l commented 1 year ago

Yeah the backup and restore functionality has not been tested yet so that didn't work natively yet.

You can build that image by yourself though and use it that way.

That seems to be working fine afterwards, I just restored my small private mail server with 2 domains and ~15 mailboxes to an OCI Instance without issues.

BrianHenryIE commented 1 year ago

Currently testing feat/arm64-cow https://github.com/mailcow/mailcow-dockerized/pull/5219/ at commit 57ba8036. At home on a Pi 4 8GB.

docker-compose.override.yml

version: '2.4'
services:
  ipv6nat-mailcow:
    image: bash:latest
    restart: "no"
    entrypoint: ["echo", "ipv6nat disabled in docker-compose.override.yml"]

  clamd-mailcow:
    image: bash:latest
    restart: "no"
    entrypoint: ["echo", "clamd disabled in docker-compose.override.yml"]

  dockerapi-mailcow:
    image: quay.io/mailcowarm64/dockerapi

mailcow.conf

...
SKIP_CLAMD=y
...

IPv6 is disabled because I don't have an IPv6 address. The other two changes were troubleshooting to get things up and running. I was getting a ~"please wait while we get started" screen that never changed (can't remember the exact wording). I disabled clamd because I saw it mentioned in the logs. Then I started swapping out images one by one with unixfox 's work until it booted properly.

Edit: turns out my ISP (Comcast) blocks port 25 by default, so I've hit a wall for now in testing. I have sending working via AWS SES. There is currently no indication of any problems. I'm optimistic I'll have this fully working soon. Solved: I signed up for dynu.com to forward emails to another port.

justiceserv commented 1 year ago

I have now started mailcow with docker compose up, however something does not seem to be working properly.

The dockerapi container is constantly restarting:

mailcowdockerized-dockerapi-mailcow-1  | Traceback (most recent call last):
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/api/client.py", line 214, in _retrieve_server_version
mailcowdockerized-dockerapi-mailcow-1  |     return self.version(api_version=False)["ApiVersion"]
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/api/daemon.py", line 181, in version
mailcowdockerized-dockerapi-mailcow-1  |     return self._result(self._get(url), json=True)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/utils/decorators.py", line 46, in inner
mailcowdockerized-dockerapi-mailcow-1  |     return f(self, *args, **kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/api/client.py", line 237, in _get
mailcowdockerized-dockerapi-mailcow-1  |     return self.get(url, **self._set_request_timeout(kwargs))
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/requests/sessions.py", line 600, in get
mailcowdockerized-dockerapi-mailcow-1  |     return self.request("GET", url, **kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
mailcowdockerized-dockerapi-mailcow-1  |     resp = self.send(prep, **send_kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
mailcowdockerized-dockerapi-mailcow-1  |     r = adapter.send(request, **kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/requests/adapters.py", line 486, in send
mailcowdockerized-dockerapi-mailcow-1  |     resp = conn.urlopen(
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/urllib3/connectionpool.py", line 790, in urlopen
mailcowdockerized-dockerapi-mailcow-1  |     response = self._make_request(
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/urllib3/connectionpool.py", line 496, in _make_request
mailcowdockerized-dockerapi-mailcow-1  |     conn.request(
mailcowdockerized-dockerapi-mailcow-1  | TypeError: HTTPConnection.request() got an unexpected keyword argument 'chunked'
mailcowdockerized-dockerapi-mailcow-1  |
mailcowdockerized-dockerapi-mailcow-1  | During handling of the above exception, another exception occurred:
mailcowdockerized-dockerapi-mailcow-1  |
mailcowdockerized-dockerapi-mailcow-1  | Traceback (most recent call last):
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/bin/uvicorn", line 8, in <module>
mailcowdockerized-dockerapi-mailcow-1  |     sys.exit(main())
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
mailcowdockerized-dockerapi-mailcow-1  |     return self.main(*args, **kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/click/core.py", line 1055, in main
mailcowdockerized-dockerapi-mailcow-1  |     rv = self.invoke(ctx)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
mailcowdockerized-dockerapi-mailcow-1  |     return ctx.invoke(self.callback, **ctx.params)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/click/core.py", line 760, in invoke
mailcowdockerized-dockerapi-mailcow-1  |     return __callback(*args, **kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/uvicorn/main.py", line 410, in main
mailcowdockerized-dockerapi-mailcow-1  |     run(
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/uvicorn/main.py", line 578, in run
mailcowdockerized-dockerapi-mailcow-1  |     server.run()
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/uvicorn/server.py", line 61, in run
mailcowdockerized-dockerapi-mailcow-1  |     return asyncio.run(self.serve(sockets=sockets))
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
mailcowdockerized-dockerapi-mailcow-1  |     return loop.run_until_complete(main)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
mailcowdockerized-dockerapi-mailcow-1  |     return future.result()
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/uvicorn/server.py", line 68, in serve
mailcowdockerized-dockerapi-mailcow-1  |     config.load()
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/uvicorn/config.py", line 473, in load
mailcowdockerized-dockerapi-mailcow-1  |     self.loaded_app = import_from_string(self.app)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/uvicorn/importer.py", line 21, in import_from_string
mailcowdockerized-dockerapi-mailcow-1  |     module = importlib.import_module(module_str)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
mailcowdockerized-dockerapi-mailcow-1  |     return _bootstrap._gcd_import(name[level:], package, level)
mailcowdockerized-dockerapi-mailcow-1  |   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
mailcowdockerized-dockerapi-mailcow-1  |   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
mailcowdockerized-dockerapi-mailcow-1  |   File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
mailcowdockerized-dockerapi-mailcow-1  |   File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
mailcowdockerized-dockerapi-mailcow-1  |   File "<frozen importlib._bootstrap_external>", line 883, in exec_module
mailcowdockerized-dockerapi-mailcow-1  |   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
mailcowdockerized-dockerapi-mailcow-1  |   File "/app/dockerapi.py", line 546, in <module>
mailcowdockerized-dockerapi-mailcow-1  |     sync_docker_client = docker.DockerClient(base_url='unix://var/run/docker.sock', version='auto')
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/client.py", line 45, in __init__
mailcowdockerized-dockerapi-mailcow-1  |     self.api = APIClient(*args, **kwargs)
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/api/client.py", line 197, in __init__
mailcowdockerized-dockerapi-mailcow-1  |     self._version = self._retrieve_server_version()
mailcowdockerized-dockerapi-mailcow-1  |   File "/usr/lib/python3.10/site-packages/docker/api/client.py", line 221, in _retrieve_server_version
mailcowdockerized-dockerapi-mailcow-1  |     raise DockerException(
mailcowdockerized-dockerapi-mailcow-1  | docker.errors.DockerException: Error while fetching server API version: HTTPConnection.request() got an unexpected keyword argument 'chunked'
mailcowdockerized-dockerapi-mailcow-1 exited with code 1

The dovecot container also reports the following since 10 minutes ago: mailcowdockerized-dovecot-mailcow-1 | Waiting for versions table to be created...

And the Rspamd Container is reporting this: mailcowdockerized-rspamd-mailcow-1 | Waiting for PHP on port 9001...

Am I doing something wrong? I specially created a fresh ARM64 instance at Hetzner and did a new installation.

May not be a proper fix, but I fixed it by changing docker-compose.yml file's dockerapi-mailcow's image value to quay.io/mailcowarm64/dockerapi. I am running Ubuntu 22.04 Minimal Image on OCI, Ampere Instance.