Qemu runner is quite slow

Keno commented 6 years ago

The qemu runner feels quite slow, we should consider

Giving it more than one cpu
Enabling caching for the 9p mounts (though maybe not in interactive mode to make sure changes are reflected).

staticfloat commented 6 years ago

Enabling caching for the 9p mounts (though maybe not in interactive mode to make sure changes are reflected).

We could put a sync within the bash exit trap stuff to make sure.

staticfloat commented 6 years ago

I've been chipping away at this on sf/bb8:

Setting -cpu host seems to work nicely. Silences some warnings too.
Increasing the -smp number seems to trigger an internal race condition in QEMU; sometimes it just fails to start and spins idly. At least it tells you it's doing so (it complains about an I/O loop running for 1000 iterations or something like that). -smp 2 seems to work. but -smp 4 reliably fails for me.
Disk I/O on 9p filesystems is atrocious:

bash-4.4# time dd if=/dev/urandom of=foo bs=1M count=100
100+0 records in
100+0 records out

real    0m5.490s
user    0m0.000s
sys     0m0.488s
bash-4.4# time dd if=/dev/urandom of=/tmp/foo bs=1M count=100
100+0 records in
100+0 records out

real    0m0.230s
user    0m0.000s
sys     0m0.230s

This is in a situation where my cwd is a 9p mount. This is with loose caching enabled, which as far as I can tell, is supposed to be the fastest available. Right now, we have a nice debugging setup where if a build fails, the build tree state is kept on disk, because it was in a 9p mount. I may experiment with removing that and having the build environment be fully transitory; e.g. only mount $WORKSPACE/destdir on 9p, and have $WORKSPACE/srcdir be just part of the overlay and not a 9p mount. That will mean that it will be harder to inspect the intermediate state of the build (it will require a little more skullduggery to make --debug work properly, which I think is the "proper" way to debug these things anyway) but the performance advantages are swaying me to this side.

staticfloat commented 6 years ago

Okay I significantly sped this up by setting access=any, which stops linux from doing a back-and-forth with QEMU to figure out which user it should use when accessing a file, and instead just accesses as the same user every time. Also, the caching scheme fscache seems to be slightly faster than loose.

giordano commented 4 years ago

Can be closed? QEMU runner is not a thing anymore

JuliaPackaging / BinaryBuilder.jl

Qemu runner is quite slow #263