Kiwix serve docker image doesn't work on Apple M1 mac system

[natamox]

The container can run, but when I enter the address in the address bar to access, an error will be reported, and the container will stop running

[natamox]

qemu: uncaught target signal 11 (Segmentation fault) - core dumped Segmentation fault Here is the content of /data: /data/.DS_Store /data/wikipedia.zim /data/wiktionary.zim

[kelson42]

@rgaudin Do you know how to help? Should such a scenario work at all using the arm64 container image? Do we have a native binary, released, which works on M1 with macOS? I guess the problem is unrelated to the container?

[rgaudin]

It should work but as we cannot test, I can't be certain.

We do have a kiwix tools nightly for macos arm64 which should also work.

[kelson42]

@rgaudin @mgautierfr Even on nightly we don't have a native build for macOS M1... or I'm blindhttps://download.kiwix.org/nightly/2023-07-08/

[rgaudin]

Ah! Should be there ; as you can see we have libkiwix. Or is this ticket still open?

[kelson42]

@rgaudin @mgautierfr I had to reopen it! https://github.com/kiwix/kiwix-build/issues/592#issuecomment-1627433974

[natamox]

For testing, I installed parallels desktop virtual machine on my mac and then installed ubuntu arm64, but this time it seems to be worse than before. I don't know if it's because of parallels desktop, but I think it's more likely because of M1...

[kelson42]

@natamox This last one looks more like a weakness in the container, can you share the full log in text mode (from the point you download the container image) please.

[natamox]

@kelson42 Sorry for the late arrival, my internet seems to be a bit slow. Is this the log you want?

Sorry, it seems that this content cannot be copied directly in the virtual machine, I will find a way later

[kelson42]

@natamox what is the content of your docker compose yml file?

[natamox]

@kelson42

version: '3'
services:
  kiwix-serve:
    image: ghcr.io/kiwix/kiwix-serve
    volumes:
      - ./data:/data
    ports:
      - 8080:8080
    command: ['*.zim']

This is my demo file directory

[rgaudin]

Thank you @natamox ; could you run the following commands both from macOS and from parallels. Copy paste the command and the output please.

docker run -it ghcr.io/kiwix/kiwix-tools:3.5.0 sh -c "uname -a && apk add file && file /usr/local/bin/kiwix-serve && kiwix-serve --help"

This assumes you have a wikipedia.zim file in ./data:

docker run -it -p 8080:80 -v $PWD/data:/data ghcr.io/kiwix/kiwix-tools:3.5.0 wikipedia.zim

Try to access it (http://localhost:8080) if it doesn't crash.

Docker on macOS also uses an aarch64 Linux VM so results should be similar (and working! 🙃) but as I have no Apple chip machine, I can't be sure. Thanks for your help.

[natamox]

@rgaudin Thank you for your reply; of course, here is the result of running the first command on my mac

Linux 2546e467b361 5.15.49-linuxkit-pr #1 SMP PREEMPT Thu May 25 07:27:39 UTC 2023 aarch64 Linux
fetch https://dl-cdn.alpinelinux.org/alpine/v3.16/main/aarch64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.16/community/aarch64/APKINDEX.tar.gz
(1/2) Installing libmagic (5.41-r0)
(2/2) Installing file (5.41-r0)
Executing busybox-1.35.0-r17.trigger
OK: 13 MiB in 17 packages
/usr/local/bin/kiwix-serve: ELF 32-bit LSB executable, ARM, EABI5 version 1 (GNU/Linux), statically linked, for GNU/Linux 3.2.0, stripped
Usage:
kiwix-serve [OPTIONS] ZIM_PATH+
kiwix-serve --library [OPTIONS] LIBRARY_PATH

Purpose:
Deliver ZIM file(s) articles via HTTP

Mandatory arguments:
LIBRARY_PATHXML library file path listing ZIM file to serve. To be used only with the --library argument.
ZIM_PATHZIM file path(s)

Optional arguments:

-h, --helpPrint this help

-a, --attachToProcessWxit if given process id is not running anymore
-d, --daemonDetach the HTTP server daemon from the main process
-i, --addressListen only on this ip address, all available ones otherwise
-M, --monitorLibraryMonitor the XML library file and reload it automatically
-m, --nolibrarybuttonDon't print the builtin home button in the builtin top bar overlay
-n, --nosearchbarDon't print the builtin bar overlay on the top of each served page
-b, --blockexternalPrevent users from directly accessing external links
-p, --portTCP port on which to listen to HTTP requests (default: 80)
-r, --urlRootLocationURL prefix on which the content should be made available (default: /)
-s, --searchLimitMaximun number of zim in a fulltext multizim search (default: No limit)
-t, --threadsNumber of threads to run in parallel (default: 4)
-v, --verbosePrint debug log to STDOUT
-V, --versionPrint software version
-z, --nodatealiasesCreate URL aliases for each content by removing the date
-c, --customIndexAdd path to custom index.html for welcome page
-L, --ipConnectionLimitMax number of (concurrent) connections per IP (default: infinite, recommended: >= 6)

Documentation:
Source codehttps://github.com/kiwix/kiwix-tools
More infohttps://wiki.kiwix.org/wiki/Kiwix-serve

But when I run the second command, something strange happens, kiwix-tools can’t run, but serve can run, but http://localhost:8080/ still can’t be accessed

And the docker container doesn't seem to crash either. . . . .😵😵😵

[natamox]

This is the result of executing it in the virtual machine the first command

Unable to find image 'ghcr.io/kiwix/kiwix-tools:3.5.0' locally
3.5.0: Pulling from kiwix/kiwix-tools
547446be3368: Already exists 
3450538a2f7d: Already exists 
Digest: sha256:d5888abeaf44dc2de2c521f1e7ed9a7e6eacf2baad35d39c47450595bc291a6b
Status: Downloaded newer image for ghcr.io/kiwix/kiwix-tools:3.5.0
Linux 041807d4f0cc 5.15.0-41-generic #44-Ubuntu SMP Thu Jun 23 11:20:13 UTC 2022 aarch64 Linux
fetch https://dl-cdn.alpinelinux.org/alpine/v3.16/main/aarch64/APKINDEX.tar.gz
ERROR: https://dl-cdn.alpinelinux.org/alpine/v3.16/main: temporary error (try again later)
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.16/main: No such file or directory
fetch https://dl-cdn.alpinelinux.org/alpine/v3.16/community/aarch64/APKINDEX.tar.gz
ERROR: https://dl-cdn.alpinelinux.org/alpine/v3.16/community: temporary error (try again later)
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.16/community: No such file or directory
ERROR: unable to select packages:
  file (no such package):
    required by: world[file]

the second

[rgaudin]

But when I run the second command, something strange happens, kiwix-tools can’t run, but serve can run, but http://localhost:8080/ still can’t be accessed

My bad, kiwix-tools doesn't support passing just filenames. Please use this command:

docker run -it -p 8080:80 -v $PWD/data:/data ghcr.io/kiwix/kiwix-tools:3.5.0 kiwix-serve /data/wikipedia.zim

But the output above shows an issue already: the kiwix-serve binary is 32b ARM instead of 64b. It's been fixed but we haven't released kiwix-tools since so you'll have to wait for the next release.

In the mean time, you can try with this other repo that uses nightly versions

docker run -it -p 8080:80 -v $PWD/data:/data ghcr.io/rgaudin/kiwix-tools:nightly sh -c "uname -a && apk add file && file /usr/local/bin/kiwix-serve && kiwix-serve --help && kiwix-serve /data/wikipedia.zim

[natamox]

But when I run the second command, something strange happens, kiwix-tools can’t run, but serve can run, but http://localhost:8080/ still can’t be accessed

My bad, kiwix-tools doesn't support passing just filenames. Please use this command:

docker run -it -p 8080:80 -v $PWD/data:/data ghcr.io/kiwix/kiwix-tools:3.5.0 kiwix-serve /data/wikipedia.zim

But the output above shows an issue already: the kiwix-serve binary is 32b ARM instead of 64b. It's been fixed but we haven't released kiwix-tools since so you'll have to wait for the next release.

In the mean time, you can try with this other repo that uses nightly versions

docker run -it -p 8080:80 -v $PWD/data:/data ghcr.io/rgaudin/kiwix-tools:nightly sh -c "uname -a && apk add file && file /usr/local/bin/kiwix-serve && kiwix-serve --help && kiwix-serve /data/wikipedia.zim"

Thanks a lot, it works now😀😀😀

[natamox]

@rgaudin But it still seems to be a problem on ubuntu

Linux 963865e3afa0 5.15.0-41-generic #44-Ubuntu SMP Thu Jun 23 11:20:13 UTC 2022 aarch64 Linux
fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/main/aarch64/APKINDEX.tar.gz
WARNING: updating and opening https://dl-cdn.alpinelinux.org/alpine/v3.18/main: temporary error (try again later)
fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/community/aarch64/APKINDEX.tar.gz
WARNING: updating and opening https://dl-cdn.alpinelinux.org/alpine/v3.18/community: temporary error (try again later)
ERROR: unable to select packages:
  file (no such package):
    required by: world[file]

[rgaudin]

Well this seems to be unrelated to kiwix-tools. apk add (from alpine base image) fails with what looks like a network error.

[natamox]

Well this seems to be unrelated to kiwix-tools. apk add (from alpine base image) fails with what looks like a network error.

Thanks, I'll try again later

[kelson42]

@natamox We also publish now binaries for M1, see https://download.kiwix.org/release/kiwix-tools/kiwix-tools_macos-arm64-3.6.0.tar.gz