search

multiarch / qemu-user-static

:earth_africa: `/usr/bin/qemu-*-static`
https://hub.docker.com/r/multiarch/qemu-user-static/
MIT License
2.46k stars 229 forks source link

iptables-nft not supported in qemu-aarch64-static #191

Open AndyEWang opened 1 year ago

AndyEWang commented 1 year ago

Is this a bug report, feature (enhancement) request or question? (leave only one on its own line) /kind enhancement

Description: Running cmd iptables-nft shows "nft: Protocol not supported" in the container using arm64 alpine with qemu-aarch64-static.

/ # /sbin/iptables-nft --version
iptables: Failed to initialize nft: Protocol not supported
/ # ls -l /sbin/iptables-nft
lrwxrwxrwx    1 root     root            17 Apr 23 05:39 /sbin/iptables-nft -> xtables-nft-multi

Steps to reproduce the issue:

  1. using CentOS Linux release 7.8.2003 (Core)

  2. docker run --rm --privileged multiarch/qemu-user-static --reset

  3. docker run --rm -it -v "/usr/bin/qemu-aarch64-static:/usr/bin/qemu-aarch64-static" alpine:3.17.3 sh

  4. apk add iptables

  5. /sbin/iptables-nft --version

Describe the results you received: / # /sbin/iptables-nft --version iptables: Failed to initialize nft: Protocol not supported

Describe the results you expected: Should be the same output as alpine linux/amd64. / # iptables-nft --version iptables v1.8.8 (nf_tables)

Environment: CentOS Linux release 7.8.2003 (Core)

Output of docker version

Client: Docker Engine - Community
 Version:           20.10.22
 API version:       1.41
 Go version:        go1.18.9
 Git commit:        3a2c30b
 Built:             Thu Dec 15 22:30:24 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.22
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.9
  Git commit:       42c8b31
  Built:            Thu Dec 15 22:28:33 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.14
  GitCommit:        9ba4b250366a5ddde94bb7c9d1def331423aa323
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
konjas commented 1 year ago

Same issue with RPM-based containers.

For example, fedora:36 docker image running on:

zandercodes commented 1 year ago

You can't use IpTable in qemu if you use a different architecture than the one from the host.

AndyEWang commented 1 year ago

@zandercodes Thanks for your reply. Does qemu-aarch64-static plan to support it?

zandercodes commented 1 year ago

@zandercodes Thanks for your reply. Does qemu-aarch64-static plan to support it?

You can try docker run --rm --privileged multiarch/qemu-user-static --reset -p yes and docker run --rm -it arm64v8/alpine:3.17.3 sh

root@ZanderCodes ~ # docker run --rm --cap-add=NET_ADMIN --cap-add=NET_RAW -it arm64v8/alpine:3.17.3 sh
WARNING: The requested image's platform (linux/arm64/v8) does not match the detected host platform (linux/amd64/v3) and no specific platform was requested
/ # apk add iptables
fetch https://dl-cdn.alpinelinux.org/alpine/v3.17/main/aarch64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.17/community/aarch64/APKINDEX.tar.gz
(1/3) Installing libmnl (1.0.5-r0)
(2/3) Installing libnftnl (1.2.4-r0)
(3/3) Installing iptables (1.8.8-r2)
Executing busybox-1.35.0-r29.trigger
OK: 15 MiB in 18 packages
/ # /sbin/iptables-nft -v
iptables: Failed to initialize nft: Protocol not supported
/ # /sbin/iptables -v
iptables v1.8.8 (legacy): no command specified
Try `iptables -h' or 'iptables --help' for more information.
/ #

With emulation not working. Without works.

root@ZanderCodes ~ # docker run --rm --cap-add=NET_ADMIN --cap-add=NET_RAW -it alpine:3.17.3 sh
Unable to find image 'alpine:3.17.3' locally
3.17.3: Pulling from library/alpine
f56be85fc22e: Already exists
Digest: sha256:124c7d2707904eea7431fffe91522a01e5a861a624ee31d03372cc1d138a3126
Status: Downloaded newer image for alpine:3.17.3
/ # apk add iptables
fetch https://dl-cdn.alpinelinux.org/alpine/v3.17/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.17/community/x86_64/APKINDEX.tar.gz
(1/3) Installing libmnl (1.0.5-r0)
(2/3) Installing libnftnl (1.2.4-r0)
(3/3) Installing iptables (1.8.8-r2)
Executing busybox-1.35.0-r29.trigger
OK: 9 MiB in 18 packages
/ # /sbin/iptables-nft -v
iptables v1.8.8 (nf_tables): no command specified
Try `iptables -h' or 'iptables --help' for more information.
/ #
AndyEWang commented 1 year ago

@zandercodes Actually, we wants your first case to work. So we cannot rely on qemu-aarch64-static to run arm64 alpine container on amd64 host, right? I mean qemu-aarch64-static cannot simulate iptables inside arm64 container on the amd64 host.

hasan4791 commented 1 year ago

Same here on M1 mac with toolbox running x86 container on qemu-user-static-x86

[root@toolbox ~]# iptables-nft -L
iptables: Failed to initialize nft: Protocol not supported
⬢[root@toolbox ~]#

Surprisingly everything works fine with rosetta.