Unable to start via k8s

devil-ming commented 1 month ago

Operating system

alinux3

Description

I was able to start with the original docker command, but not via k8s pod

Docker compose

my docker run command

docker run -it -v /home/original-git.qcow2:/boot.qcow2 -e DISK_SIZE=128G -e DISK_CACHE=directsync -p 8006:8006 --device=/dev/kvm --cap-add NET_ADMIN qemu-docker:0.1

my pod yaml

apiVersion: v1
kind: Pod
metadata:
  name: test
  namespace: test
spec:
  terminationGracePeriodSeconds: 120
  containers:
    - name: worker
      image: qemu-docker:0.1
      securityContext:
        capabilities:
          add:
            - NET_ADMIN
      volumeMounts:
        - mountPath: /boot.qcow2
          name: vm-disk
        - mountPath: /dev/kvm
          name: dev-kvm
      env:
        - name: DISK_SIZE
          value: "128G"
        - name: DISK_CACHE
          value: "directsync"
      ports:
        - containerPort: 8006
          hostPort: 8006
      tty: true
  volumes:
    - name: dev-kvm
      hostPath:
        path: /dev/kvm
    - name: vm-disk
      hostPath:
        path: /home/original-git.qcow2
        type: File

Docker log

❯ Starting QEMU for Docker v0.0...
❯ For support visit https://github.com/qemus/qemu-docker
❯ CPU: Intel Xeon Platinum 8269CY CPU | RAM: 739/755 GB | DISK: 291 GB (xfs) | HOST: 5.10.134-13.al8.x86_64...

iptables v1.8.9 (legacy): can't initialize iptables table `nat': Table does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.
❯ ERROR: The 'ip_tables' kernel module is not loaded. Try this command: sudo modprobe ip_tables iptable_nat
❯ Warning: falling back to usermode networking! Performance will be bad and port forwarding will not work.
❯ ERROR: KVM acceleration not available (no write access), this will cause a major loss of performance.
❯ ERROR: See the FAQ on how to diagnose the cause, or continue without KVM by setting KVM=N (not recommended).

Screenshots (optional)

No response

kroese commented 1 month ago

Your pod yaml looks nothing like https://github.com/qemus/qemu-docker/blob/master/kubernetes.yml You removed important parts like privileged flag.

devil-ming commented 1 month ago

I noticed this, but I have tried it and the result is the same。 I executed sudo modprobe ip_tables iptable_nat , now the error message is

❯ Starting QEMU for Docker v0.0...
❯ For support visit https://github.com/qemus/qemu-docker
❯ CPU: Intel Xeon Platinum 8269CY CPU | RAM: 739/755 GB | DISK: 489 GB (xfs) | HOST: 5.10.134-13.al8.x86_64...

❯ Creating a 128G growable disk image in raw format...
❯ ERROR: KVM acceleration not available (no write access), this will cause a major loss of performance.
❯ ERROR: See the FAQ on how to diagnose the cause, or continue without KVM by setting KVM=N (not recommended).

I don't know why the docker run can succeed, but it fails when it is replaced by pod.