Bad qcow2 performance

[jacobat]

I'm trying to load some small mysql databases into a mysql instance in docker with a docker volume for storing the data.

With the old raw file format I can load my databases in 30 secs. With the new qcow2 format it takes 10 mins+.

This is on corectl 0.7.15.

Please let me know what information you need. Thanks :)

[AntonioMeireles]

@jacobat (sorry lag, first hollidays, and then getting work queue manageable :-) ) can you provide as a simple way to reproduce ? also, on 10.10 (yosemite) or Sierra ? using Fusion drives by any chance ?

[jacobat]

I'm on OSX 10.11.6. Corectl 0.7.15.

Repro steps

Run a mysql server in a Docker container like:

CID=$(docker run -v $(pwd):/data -d -p 3306:3306 -e MYSQL_PASS="password" -e ON_CREATE_DB="test_db" tutum/mysql)

In the current directory put this sql dump: https://gist.githubusercontent.com/jacobat/e085b9af8a893941cc8fdaf62751a1bd/raw/bf5efa9717048f028aa160ac053c459c9912ba46/test.dump.sql

Then load the dump into mysql:

docker exec -ti $CID sh -c "mysql < /data/test.dump.sql"

Results

When I run this on a raw filesystem it's instantaneous. When I run this on a qcow filesystem it takes around 7 seconds. This is a very small dump, not the real dump I'm trying to load but it does show the difference.

And welcome back - I hope you had a great holiday :)

[AntonioMeireles]

hi @jacobat

afawct this was an issue with recent hyperkit releases that arised when it was build against the wrong qcow-format scam lib. It should be fixed in the 0.7.17's blob here at github and i'm in the process of updating homebrew's to consume fix too (https://github.com/Homebrew/homebrew-core/pull/4885)

please confirm that this really fixes things to your use case :-) and once again thanks again for your patience

[jacobat]

Just tested this with 0.7.17. The issue remains I'm afraid. I even tried making entirely new disks for both qcow2 and raw formats.

[AntonioMeireles]

@jacobat can you test again please? (and sorry for the lag) In general terms I/O performance should be no better or worse than the one in docker's official macOS app, as we consume hyperkit as them. In the best, we may had not be building it (specially the OCAML bits) the most deterministic way, but hopefullt we've got this right by now... (if you want to build things fully manually - see in the Makefile the comments regading OCAML ...)

[jacobat]

Trying to run qcow-tool create --size=20GiB ./docker.img.qcow2 I get:

dyld: Library not loaded: /usr/local/opt/libev/lib/libev.4.dylib
  Referenced from: /usr/local/bin/qcow-tool
  Reason: image not found

[jacobat]

brew install libev fixed that.

[AntonioMeireles]

@jacobat many thanks for catching this one. Will propagate fix soonish. (&& have a great weekend!)

[jacobat]

Tested again. qcow2 performance appears to be on par with old style disk format now. Thanks - and a great weekend to you too :)

[AntonioMeireles]

WOOT! && thanks!