kernel bug at mmaped fuse file

[krha]

This is randomly reported when create overlay VM. It tends to be happen more frequently on low-end machine.

How to reproduce

• It happends when you access a FUSE file that is mmaped by QEMU process even after QEMU process is terminated.

  Follow up Test

  1. QEMU successfully munmap the FUSE file
  2. Make simple program that mmaps FUSE file as QEMU do and munmap successfully. And repeat mmap/munmap several time with FUSE file. This does not cause any problem.

     Possible causes

  3. Suspicious on QEMU process, but don't know yet.
  4. Avoid accessing FUSE file access after QEMU finishes.

[krha]

test

1. resume VM with pause state
   OK: access memory while KVM is alive
   OK: access memory after KVM finishes (it caused kernel failed in VM running status)
2. Accessing mmaped memory while running QEMU
   FAILED: kernel panic.
3. Test with simple dummy program that does mmaped to fuse memory and heavily accesses it. OK: no problem.

--> QEMU touches mmaped file in abnormal way?

[krha]

It is not happening at Yohsi's code, so the problem might be coming from FUSE related code. I'll revert my fuse code to original to remove all my modification.

[krha]

This is interesting bug. It only happens at certain environment where the system has around 4GB memory. It seems to be related to available cache size and cannot reproduce with my custom mmap test program.

Next Test

1. suspicious on KVM memory management -> disable KVM and test again
   • This case works without problem..means the way KVM touches memory causes this problem. This is why I couldn't reproduce this from my code.
2. What if I disable EPT? EPT is one difference from my machine and Yoshi's machine.

I disabled EPT by

$ sudo modprobe -r kvm_intel $ sudo modprobe kvm_intel "ept=0"

and verified disabled EPT by

$ cat /sys/module/kvm_intel/parameters/ept

Wow, it works well without EPT!!

[krha]

Conclusion

EPT makes problem at mmaped fuse memory.

Workaround

1. First, check you have EPT support or not.

   $ cat /proc/cpuinfo | grep " ept" If it shows something, you have EPT support.

2. Disable EPT if you have it by

   $ sudo modprobe -r kvm_intel $ sudo modprobe kvm_intel "ept=0"

3. Verify that you successfully disabled it by

   $ cat /sys/module/kvm_intel/parameters/ept N $

I'll keep working on this. Before having concrete solution, you can use this workaround if you have this problem.