search

michaelmcandrew / civicrm-buildkit-docker

This is a read only copy. Please make PRs here: https://lab.civicrm.org/michaelmcandrew/civicrm-buildkit-docker
https://lab.civicrm.org/michaelmcandrew/civicrm-buildkit-docker
GNU Affero General Public License v3.0
40 stars 31 forks source link

Tests running slow #16

Closed michaelmcandrew closed 5 years ago

michaelmcandrew commented 6 years ago

@totten - I am opening up this issue to discuss slow running tests on buildkit (see https://github.com/michaelmcandrew/civicrm-buildkit-docker/issues/15#issuecomment-379359365 for the origin)

time env CIVICRM_UF=UnitTests phpunit4 tests/phpunit/CRM/Core/AllTests.php on my machine (Intel(R) Core(TM) i7-5500U CPU @ 2.40GHz dual core with 8GB ram) is 14m4.434s based on a sample size of one.

I do not recall getting it much below this even with bare metal on this machine. Not sure what that means but regardless, it would be great to speed things up.

michaelmcandrew commented 6 years ago

The civicrm image is based on php:7.0-apache-jessie. The php cli on this image uses the same php config as apache, which is PHP defaults apart from these lines https://github.com/michaelmcandrew/civicrm-buildkit-docker/blob/master/civicrm/buildkit.ini

One of which sets the memory limit to 1024M.

However, running phpunit with time env CIVICRM_UF=UnitTests phpunit4 -d memory_limit=-1 tests/phpunit/CRM/Core/AllTests.php had no effect on execution time

In other words, still a mystery.

totten commented 6 years ago

  1. @michaelmcandrew , were you running mysql with a ramdisk (tmpfs) or hdd/ssd (persistent volume)?

  2. I forgot to mention the hardware in the previous benchmarks. Reproducing below with hardware info.

  1. FWIW, just to get a ballpark to compare performance in different hardware, we could pull up the single-threaded scores fromcpubenchmark.net: i7-5500U gets 1546 and i7-4870HQ gets 2039 (~30% faster). If the software configurations were identical, we'd expect the 4870HQ to run ~30% faster (~10m). But the software stacks are different, and it's actually ~40% slower (~19m).

  2. IIRC, the biggest time sink in CiviUnitTestCase is resetting mysqld. Although I setup docker-compose.yml to put mysqld in tmpfs, and although I checked that this went live (via docker ... exec and mount/df), this kind of performance discrepancy still feels like a slow mysqld. A smart thing might be to try some synthetic benchmark on each of (a) php processes and (b) mysql i/o -- see if one or the other has a clear performance issue.

michaelmcandrew commented 6 years ago

re 1. nope re 4: yeah, does anything spring to mind? was also wondering if some xdebug profiling might be of use to point to a particular function that was slow.

we are using the default mysql docker image with out of the box settings.

/googles 'mysql docker performance'

michaelmcandrew commented 5 years ago

did some testing docker vs bare metal on comparable machines. test speed seems within range.