search

spheenik / vfio-isolate

CPU and memory isolation for VFIO
MIT License
90 stars 8 forks source link

Cannot set CPU affinity on process: Invalid argument #6

Open fidelix opened 3 years ago

fidelix commented 3 years ago

My qemu hook:


RSET=/
HSET=/host.slice
MSET=/machine.slice

disable_isolation () {
    vfio-isolate \
        cpuset-delete $HSET \
        cpuset-delete $MSET
}

enable_isolation () {
    vfio-isolate -u /root/.vfioundo drop-caches compact-memory cpuset-create --cpus C0-3,10-13 $HSET cpuset-create --cpus C4-9,14-19 -nlb $MSET move-tasks / $HSET
}

case "$2" in
"prepare")
    enable_isolation
    ;;
"release")
    disable_isolation
    ;;
esac

Error I get when starting VM:

Error starting domain: cannot set CPU affinity on process 2492: Invalid argument

Am I missing some dependency? I'm using Arch, kernel 5.10.9. libvirt 7.0.0, vfio-isolate installed from git.

spheenik commented 3 years ago

Hmm, libvirt 7.0.0? You installed that from git too? Does the VM start without the isolation script? What exactly is process 2492?

fidelix commented 3 years ago

No, libvirt 7.0.0 was installed from the arch repo.

When I run sudo virsh start win it hangs for many minutes (30 min so far). ps aux shows

root        6722  0.0  0.1 1549980 33816 ?       Ssl  11:33   0:00 /usr/bin/libvirtd --timeout 120
root        6759  0.0  0.0  64972 16352 ?        Ss   11:33   0:00 /usr/bin/virtlogd
root        6847  0.0  0.0   4208  2924 ?        S    11:34   0:00 /bin/bash /etc/libvirt/hooks/qemu win prepare begin -
root         245  0.0  0.0      0     0 ?        I<   10:58   0:00 [vfio-irqfd-clea]
root        6848 99.9  0.0  18932 17036 ?        R    11:34  17:18 /usr/bin/python3 /usr/bin/vfio-isolate -u /root/.vfioundo drop-caches compact-memory cpuset-create --cpus C0-3,10-13 /host.slice cpuset-create --cpus C4-9,14-19 -nlb /machine.slice move-tasks / /host.slice

I'm not even getting to the part where I get that error anymore.

fidelix commented 3 years ago

After 10 more minutes it failed:

error: cannot set CPU affinity on process 14796: Invalid argument

A process with that PID did not exist when I checked /proc/14796/

spheenik commented 3 years ago

Ok, the update to libvirt 7 seems pretty new. I'll test here, if I see similar results.

You did not answer though: Does it work without the hook?

fidelix commented 3 years ago

Well, if I run vfio-isolate directly, without the hook, the problem is also there.

But if I don't use vfio-isolate, yes, libvirt is working just fine.