heroku / heroku-buildpack-php

Heroku's buildpack for PHP applications.
https://devcenter.heroku.com/categories/php
MIT License
808 stars 1.59k forks source link

Restructure tests to reduce duplication and improve runtimes #680

Closed dzuelke closed 10 months ago

dzuelke commented 10 months ago

tl;dr: fewer test lines, and tests now take about 5:30-6:30 minutes instead of 7:00-8:00 minutes.

Remove duplication mostly for SIGTERM related tests, and split some tests up into smaller/faster test files to improve parallel_tests' ability to balance test files around based on their logged runtimes.

Test file names are now more consistent (php_concurrency_shared.rb as the base, php-8.3_concurrency-nginx_spec.rb as an "instance" for PHP 8.3 and Nginx).

Sped up some tests, in particular the one for request_terminate_timeout, which is now ~25 seconds faster than before.

Runs will now automatically use ceil($numTestsForStack/2) (based on the runtime log) parallel processes to ensure that there are enough workers for the few really slow tests to run standalone, while at the same time not having faster tests pile up for longer than the slowest test.
This is better than a hard-coded process count, since it means 1) we have fewer parallel processes (and thus less Heroku load) for heroku-22 since there are fewer tests (for fewer supported PHP versions), and 2) we do not have to remember adjusting things downward when heroku-20 goes EOL (and its tests get removed), or 3) adjust things upwards when a future PHP version is added (which again increases the number of tests).

GUS-W-14875944