search

sebastianbergmann / phpunit

The PHP Unit Testing framework.
https://phpunit.de/
BSD 3-Clause "New" or "Revised" License
19.65k stars 2.2k forks source link

Increase in running time and memory usage after PHPUnit 10 upgrade #5369

Closed robjmills closed 1 year ago

robjmills commented 1 year ago
Q A
PHPUnit version 10.1.3
PHP version 8.2.5
Installation Method Composer

Summary

We have experienced a major increase in the running time of the test suite, and memory usage, after upgrading from PHPUnit 9 to PHPUnit 10

Current behavior

Timing on PHPUnit 10

Time: 00:31.536, Memory: 2.06 GB

OK, but there are issues!
Tests: 5316, Assertions: 22293, Warnings: 1, Skipped: 3.

Timing on PHPUnit 9

Time: 00:11.451, Memory: 527.00 MB

OK, but incomplete, skipped, or risky tests!
Tests: 5340, Assertions: 22334, Skipped: 3.

Composer show output

acmephp/core                                   1.2.0                          Raw implementation of...
acmephp/ssl                                    1.3.0                          PHP wrapper around Op...
assetic/framework                              v2.0.2                         Asset Management for PHP
aws/aws-crt-php                                v1.2.1                         AWS Common Runtime fo...
aws/aws-sdk-php                                3.269.13                       AWS SDK for PHP - Use...
beberlei/assert                                v2.1                           Thin assertion librar...
beberlei/doctrineextensions                    v1.3.0                         A set of extensions t...
brick/math                                     0.9.3                          Arbitrary-precision a...
brick/money                                    0.5.3                          Money and currency li...
browscap/browscap-php                          2.1.1                          Standalone replacemen...
bummzack/omnipay-postfinance                   dev-omnipay-upgrade a045a48    Postfinance Gateway f...
cebe/php-openapi                               1.7.0                          Read and write OpenAP...
clue/stream-filter                             v1.6.0                         A simple and modern a...
composer/ca-bundle                             1.3.5                          Lets you find a path ...
composer/package-versions-deprecated           1.11.99.5                      Composer plugin that ...
cronofy/cronofy                                v1.5.3                         SDK for Cronofy - the...
data-uri/data-uri                              dev-loosen-requirement b01c364 PHP DataURI component
dealerdirect/phpcodesniffer-composer-installer v0.7.2                         PHP_CodeSniffer Stand...
doctrine/annotations                           1.14.3                         Docblock Annotations ...
doctrine/cache                                 1.13.0                         PHP Doctrine Cache li...
doctrine/collections                           1.8.0                          PHP Doctrine Collecti...
doctrine/common                                3.4.3                          PHP Doctrine Common p...
doctrine/dbal                                  2.13.9                         Powerful PHP database...
doctrine/deprecations                          v1.0.0                         A small layer on top ...
doctrine/event-manager                         1.2.0                          The Doctrine Event Ma...
doctrine/inflector                             2.0.6                          PHP Doctrine Inflecto...
doctrine/instantiator                          1.5.0                          A small, lightweight ...
doctrine/lexer                                 1.2.3                          PHP Doctrine Lexer pa...
doctrine/orm                                   2.8.5                          Object-Relational-Map...
doctrine/persistence                           2.5.7                          The Doctrine Persiste...
donatj/phpuseragentparser                      v1.4.0                         Lightning fast, minim...
endroid/qrcode                                 1.9.3                          Endroid QR Code
ezyang/htmlpurifier                            v4.16.0                        Standards compliant H...
fakerphp/faker                                 v1.22.0                        Faker is a PHP librar...
firebase/php-jwt                               v6.5.0                         A simple library to e...
google/apiclient                               v1.1.9                         Client library for Go...
google/auth                                    v1.26.0                        Google Auth Library f...
google/cloud-core                              v1.51.2                        Google Cloud PHP shar...
google/cloud-storage                           v1.5.2                         Cloud Storage Client ...
google/cloud-web-risk                          v1.2.7                         Google Cloud Web Risk...
google/common-protos                           v3.2.0                         Google API Common Pro...
google/gax                                     v1.19.1                        Google API Core for PHP
google/grpc-gcp                                v0.2.1                         gRPC GCP library for ...
google/longrunning                             v0.2.6                         Google LongRunning Cl...
google/protobuf                                v3.23.0                        proto library for PHP
google/recaptcha                               1.1.3                          Client library for re...
grpc/grpc                                      1.52.0                         gRPC library for PHP
gumlet/php-image-resize                        2.0.3                          PHP class to re-size ...
guzzlehttp/guzzle                              6.5.8                          Guzzle is a PHP HTTP ...
guzzlehttp/oauth-subscriber                    0.3.0                          Guzzle OAuth 1.0 subs...
guzzlehttp/promises                            1.5.2                          Guzzle promises library
guzzlehttp/psr7                                1.9.1                          PSR-7 message impleme...
hamcrest/hamcrest-php                          dev-master d34238d             This is the PHP port ...
http-interop/http-factory-guzzle               1.2.0                          An HTTP Factory using...
intercom/intercom-php                          v3.2.0                         Intercom API client b...
intervention/image                             2.7.2                          Image handling and ma...
ircmaxell/random-lib                           v1.2.0                         A Library For Generat...
ircmaxell/security-lib                         v1.1.0                         A Base Security Library
jeremykendall/php-domain-parser                5.7.2                          Public Suffix List ba...
jms/twig-js                                    dev-master 3a59ab9             twig.js, flexible, se...
justinrainbow/json-schema                      5.2.12                         A library to validate...
kevinrob/guzzle-cache-middleware               v4.0.2                         A HTTP/1.1 Cache for ...
klarna/kco_rest                                v4.2.3                         Official Klarna REST ...
league/color-extractor                         dev-master 1aff19c             Extract colors from a...
league/omnipay                                 v3.0.2                         Omnipay payment proce...
league/openapi-psr7-validator                  0.18                           Validate PSR-7 messag...
league/uri                                     6.8.0                          URI manipulation library
league/uri-components                          2.4.1                          URI components manipu...
league/uri-interfaces                          2.3.0                          Common interface for ...
mattwright/urlresolver                         2.0                            PHP class that attemp...
mercadopago/sdk                                v0.5.6                         MercadoPago SDK modul...
mikey179/vfsstream                             v1.6.11                        Virtual file system t...
mixpanel/mixpanel-php                          2.6.2                          The Official PHP libr...
mobiledetect/mobiledetectlib                   2.8.41                         Mobile_Detect is a li...
mockery/mockery                                1.4.4                          Mockery is a simple y...
mollie/mollie-api-php                          1.3.3                          Mollie API client lib...
moneyphp/money                                 v4.1.1                         PHP implementation of...
monolog/monolog                                1.27.1                         Sends your logs to fi...
mtdowling/jmespath.php                         2.6.1                          Declaratively specify...
mustache/mustache                              v2.14.2                        A Mustache implementa...
myclabs/deep-copy                              1.11.1                         Create deep copies (c...
nelmio/security-bundle                         v2.12.0                        Extra security-relate...
nette/php-generator                            v3.2.3                         🐘 Nette PHP Genera...
nette/utils                                    v3.0.4                         🛠 Nette Utils: lig...
nikic/php-parser                               v4.15.5                        A PHP parser written ...
omnipay/2checkout                              v3.0.0                         2Checkout driver for ...
omnipay/common                                 v3.2.0                         Common components for...
omnipay/payfast                                v3.1.0                         PayFast driver for th...
openpayu/openpayu                              2.2.13                         OpenPayU PHP Library
paragonie/constant_time_encoding               v2.6.3                         Constant-time Impleme...
paragonie/random_compat                        v2.0.21                        PHP 5.x polyfill for ...
paypal/sdk-core-php                            v3.2.4                         PayPal Core SDK for PHP
paytrail/paytrail-php-sdk                      1.1.0                          PHP Software Developm...
pdepend/pdepend                                2.13.0                         Official version of p...
pelago/emogrifier                              v6.0.0                         Converts CSS styles i...
phar-io/manifest                               2.0.3                          Component for reading...
phar-io/version                                3.2.1                          Library for handling ...
php-http/discovery                             1.18.1                         Finds and installs PS...
php-http/guzzle6-adapter                       v2.0.2                         Guzzle 6 HTTP Adapter
php-http/httplug                               2.4.0                          HTTPlug, the HTTP cli...
php-http/message                               1.16.0                         HTTP Message related ...
php-http/message-factory                       1.1.0                          Factory interfaces fo...
php-http/promise                               1.1.0                          Promise used for asyn...
php-mock/php-mock                              2.4.0                          PHP-Mock can mock bui...
php-mock/php-mock-integration                  2.2.1                          Integration package f...
php-mock/php-mock-phpunit                      2.7.1                          Mock built-in PHP fun...
phpmd/phpmd                                    2.6.1                          PHPMD is a spin-off p...
phpseclib/phpseclib                            3.0.19                         PHP Secure Communicat...
phpstan/extension-installer                    1.3.0                          Composer plugin for a...
phpstan/phpdoc-parser                          1.20.4                         PHPDoc parser with su...
phpstan/phpstan                                1.10.15                        PHPStan - PHP Static ...
phpstan/phpstan-doctrine                       1.3.40                         Doctrine extensions f...
phpstan/phpstan-mockery                        1.1.1                          PHPStan Mockery exten...
phpunit/php-code-coverage                      10.1.1                         Library that provides...
phpunit/php-file-iterator                      4.0.2                          FilterIterator implem...
phpunit/php-invoker                            4.0.0                          Invoke callables with...
phpunit/php-text-template                      3.0.0                          Simple template engine.
phpunit/php-timer                              6.0.0                          Utility class for timing
phpunit/phpunit                                10.1.3                         The PHP Unit Testing ...
pimple/pimple                                  v3.5.0                         Pimple, a simple Depe...
psr/cache                                      3.0.0                          Common interface for ...
psr/container                                  1.1.2                          Common Container Inte...
psr/event-dispatcher                           1.0.0                          Standard interfaces f...
psr/http-client                                1.0.2                          Common interface for ...
psr/http-factory                               1.0.2                          Common interfaces for...
psr/http-message                               1.1                            Common interface for ...
psr/http-server-handler                        1.0.2                          Common interface for ...
psr/http-server-middleware                     1.0.2                          Common interface for ...
psr/log                                        1.1.4                          Common interface for ...
psr/simple-cache                               1.0.1                          Common interfaces for...
ralouphie/getallheaders                        3.0.3                          A polyfill for getall...
ramsey/uuid                                    3.7.3                          Formerly rhumsaa/uuid...
rector/rector                                  0.15.25                        Instant Upgrade and A...
respect/stringifier                            0.2.0                          Converts any value to...
respect/validation                             2.2.4                          The most awesome vali...
riverline/multipart-parser                     2.1.1                          One class library to ...
rize/uri-template                              0.3.5                          PHP URI Template (RFC...
roave/security-advisories                      dev-latest d1af114             Prevents installation...
sabberworm/php-css-parser                      8.4.0                          Parser for CSS Files ...
scssphp/scssphp                                v1.11.0                        scssphp is a compiler...
sebastian/cli-parser                           2.0.0                          Library for parsing C...
sebastian/code-unit                            2.0.0                          Collection of value o...
sebastian/code-unit-reverse-lookup             3.0.0                          Looks up which functi...
sebastian/comparator                           5.0.0                          Provides the function...
sebastian/complexity                           3.0.0                          Library for calculati...
sebastian/diff                                 5.0.3                          Diff implementation
sebastian/environment                          6.0.1                          Provides functionalit...
sebastian/exporter                             5.0.0                          Provides the function...
sebastian/global-state                         6.0.0                          Snapshotting of globa...
sebastian/lines-of-code                        2.0.0                          Library for counting ...
sebastian/object-enumerator                    5.0.0                          Traverses array struc...
sebastian/object-reflector                     3.0.0                          Allows reflection of ...
sebastian/recursion-context                    5.0.0                          Provides functionalit...
sebastian/type                                 4.0.0                          Collection of value o...
sebastian/version                              4.0.1                          Library that helps wi...
shardj/zf1-future                              1.20.2                         Zend Framework 1. The...
silexandfriends/silex                          dev-master dc310a0             The PHP micro-framewo...
simshaun/recurr                                v4.0.5                         PHP library for worki...
slevomat/coding-standard                       7.2.1                          Slevomat Coding Stand...
spatie/enum                                    2.3.8                          PHP Enums
spatie/icalendar-generator                     1.0.6                          Build calendars in th...
squizlabs/php_codesniffer                      3.7.2                          PHP_CodeSniffer token...
stripe/stripe-php                              v7.128.0                       Stripe PHP Library
swiftmailer/swiftmailer                        v5.4.12                        Swiftmailer, free fea...
symfony-cmf/routing                            2.3.4                          Extends the Symfony r...
symfony/asset                                  v5.4.21                        Manages URL generatio...
symfony/cache                                  v6.2.10                        Provides extended PSR...
symfony/cache-contracts                        v3.2.1                         Generic abstractions ...
symfony/config                                 v5.4.21                        Helps you find, load,...
symfony/console                                v5.4.23                        Eases the creation of...
symfony/css-selector                           v5.4.21                        Converts CSS selector...
symfony/dependency-injection                   v5.4.23                        Allows you to standar...
symfony/deprecation-contracts                  v2.5.2                         A generic function an...
symfony/error-handler                          v6.2.10                        Provides tools to man...
symfony/event-dispatcher                       v5.4.22                        Provides tools that a...
symfony/event-dispatcher-contracts             v3.2.1                         Generic abstractions ...
symfony/expression-language                    v5.4.21                        Provides an engine th...
symfony/filesystem                             v5.4.23                        Provides basic utilit...
symfony/finder                                 v5.4.21                        Finds files and direc...
symfony/framework-bundle                       v5.4.22                        Provides a tight inte...
symfony/http-foundation                        v5.4.23                        Defines an object-ori...
symfony/http-kernel                            v5.4.23                        Provides a structured...
symfony/intl                                   v5.4.23                        Provides a PHP replac...
symfony/mime                                   v6.2.10                        Allows manipulating M...
symfony/monolog-bridge                         v5.4.22                        Provides integration ...
symfony/options-resolver                       v3.4.47                        Symfony OptionsResolv...
symfony/password-hasher                        v6.2.7                         Provides password has...
symfony/polyfill-ctype                         v1.27.0                        Symfony polyfill for ...
symfony/polyfill-intl-grapheme                 v1.27.0                        Symfony polyfill for ...
symfony/polyfill-intl-idn                      v1.27.0                        Symfony polyfill for ...
symfony/polyfill-intl-normalizer               v1.27.0                        Symfony polyfill for ...
symfony/polyfill-mbstring                      v1.27.0                        Symfony polyfill for ...
symfony/polyfill-php72                         v1.27.0                        Symfony polyfill back...
symfony/polyfill-php73                         v1.27.0                        Symfony polyfill back...
symfony/polyfill-php80                         v1.27.0                        Symfony polyfill back...
symfony/polyfill-php81                         v1.27.0                        Symfony polyfill back...
symfony/process                                v5.4.23                        Executes commands in ...
symfony/property-access                        v5.4.22                        Provides functions to...
symfony/property-info                          v6.2.10                        Extracts information ...
symfony/routing                                v5.4.22                        Maps an HTTP request ...
symfony/security-bundle                        v5.4.22                        Provides a tight inte...
symfony/security-core                          v5.4.22                        Symfony Security Comp...
symfony/security-csrf                          v6.2.7                         Symfony Security Comp...
symfony/security-guard                         v5.4.22                        Symfony Security Comp...
symfony/security-http                          v5.4.23                        Symfony Security Comp...
symfony/service-contracts                      v2.5.2                         Generic abstractions ...
symfony/string                                 v6.2.8                         Provides an object-or...
symfony/translation                            v5.4.22                        Provides tools to int...
symfony/translation-contracts                  v2.5.2                         Generic abstractions ...
symfony/twig-bridge                            v6.1.11                        Provides integration ...
symfony/twig-bundle                            v5.4.21                        Provides a tight inte...
symfony/validator                              v5.4.23                        Provides tools to val...
symfony/var-dumper                             v5.4.23                        Provides mechanisms f...
symfony/var-exporter                           v6.2.10                        Allows exporting any ...
symfony/yaml                                   v5.4.23                        Loads and dumps YAML ...
symplify/easy-coding-standard                  11.3.4                         Use Coding Standard w...
theseer/tokenizer                              1.2.1                          A small library for c...
twig/extensions                                v1.3.0                         Common additional fea...
twig/twig                                      v2.15.3                        Twig, the flexible, f...
ua-parser/uap-php                              v3.9.14                        A multi-language port...
vstelmakh/url-highlight                        v3.0.2                         Library to parse urls...
wallee/sdk                                     3.2.0                          wallee SDK for PHP
webmozart/assert                               1.11.0                         Assertions to validat...
yabacon/paystack-php                           v2.2.1                         Helps make your Payst...

How to reproduce

Running our test suite, this has been tested on MacOS and Linux, and both exhibit the same behaviour

Expected behavior

Similar timing to PHPUnit 9

sebastianbergmann commented 1 year ago

If you can provide a minimal, self-contained, reproducing test case for this then I (or somebody else) will be able to investigate this issue.

However, issues such as the one you describe can rarely be reduced. If your test suite is not Open Source then I will, unfortunately, not be able to help you. Not because I do not want to, but because I cannot investigate it without access to your source code and test suite.

sebastianbergmann commented 1 year ago

No feedback, closing.