Each test separately gets sorted alone, thus sorting doesn't work.

Q	A
PHPUnit version	9.5.2
PHP version	7.4.15
Installation Method	Composer

asm89/stack-cors                     v2.0.2             Cross-origin resource sharing library and stack middleware
aws/aws-sdk-php                      3.173.20           AWS SDK for PHP - Use Amazon Web Services in your PHP project
barryvdh/laravel-ide-helper          v2.8.2             Laravel IDE Helper, generates correct PHPDocs for all Facade classes, to improve auto-completion.
barryvdh/reflection-docblock         v2.0.6            
beyondcode/laravel-dump-server       1.6.0              Symfony Var-Dump Server for Laravel
brick/math                           0.9.1              Arbitrary-precision arithmetic library
brick/money                          0.5.0              Money and currency library
cakephp/chronos                      2.0.6              A simple API extension for DateTime.
clue/stream-filter                   v1.5.0             A simple and modern approach to stream filtering in PHP
composer/ca-bundle                   1.2.8              Lets you find a path to the system CA bundle, and includes a fallback to the Mozilla CA bundle.
composer/composer                    2.0.7              Composer helps you declare, manage and install dependencies of PHP projects. It ensures you have the right stack everywhere.
composer/package-versions-deprecated 1.11.99.1          Composer plugin that provides efficient querying for installed package versions (no runtime IO)
composer/semver                      3.2.4              Semver library that offers utilities, version constraint parsing and validation.
composer/spdx-licenses               1.5.4              SPDX licenses list and validation library.
composer/xdebug-handler              1.4.5              Restarts a process without Xdebug.
darkaonline/l5-swagger               dev-master d1b2f77 OpenApi or Swagger integration to Laravel
defuse/php-encryption                v2.2.1             Secure PHP Encryption Library
dnoegel/php-xdg-base-dir             v0.1.1             implementation of xdg base directory specification for php
doctrine/annotations                 1.11.1             Docblock Annotations Parser
doctrine/cache                       1.10.2             PHP Doctrine Cache library is a popular cache implementation that supports many different drivers such as redis, memcache,...
doctrine/dbal                        2.12.1             Powerful PHP database abstraction layer (DBAL) with many features for database schema introspection and management.
doctrine/event-manager               1.1.1              The Doctrine Event Manager is a simple PHP event system that was built to be used with the various Doctrine projects.
doctrine/inflector                   2.0.3              PHP Doctrine Inflector is a small library that can perform string manipulations with regard to upper/lowercase and singula...
doctrine/instantiator                1.4.0              A small, lightweight utility to instantiate objects in PHP without invoking their constructors
doctrine/lexer                       1.2.1              PHP Doctrine Lexer parser library that can be used in Top-Down, Recursive Descent Parsers.
dragonmantank/cron-expression        v3.1.0             CRON for PHP: Calculate the next or previous run date and determine if a CRON expression is due
egulias/email-validator              2.1.24             A library for validating emails against several RFCs
erusev/parsedown                     1.7.4              Parser for Markdown.
facade/flare-client-php              1.3.7              Send PHP errors to Flare
facade/ignition                      2.5.2              A beautiful error page for Laravel applications.
facade/ignition-contracts            1.0.2              Solution contracts for Ignition
fakerphp/faker                       v1.9.2             Faker is a PHP library that generates fake data for you.
fideloper/proxy                      4.4.1              Set trusted proxies for Laravel
filp/whoops                          2.9.1              php error handling for cool kids
firebase/php-jwt                     v5.2.0             A simple library to encode and decode JSON Web Tokens (JWT) in PHP. Should conform to the current spec.
friendsofphp/php-cs-fixer            v2.18.2            A tool to automatically fix PHP code style
fruitcake/laravel-cors               v2.0.3             Adds CORS (Cross-Origin Resource Sharing) headers support in your Laravel application
graham-campbell/result-type          v1.0.1             An Implementation Of The Result Type
guzzlehttp/guzzle                    7.2.0              Guzzle is a PHP HTTP client library
guzzlehttp/promises                  1.4.0              Guzzle promises library
guzzlehttp/psr7                      1.7.0              PSR-7 message implementation that also provides common utility methods
hamcrest/hamcrest-php                v2.0.1             This is the PHP port of Hamcrest Matchers
http-interop/http-factory-guzzle     1.0.0              An HTTP Factory using Guzzle PSR7
intervention/image                   2.5.1              Image handling and manipulation library with support for Laravel integration
jean85/pretty-package-versions       1.5.1              A wrapper for ocramius/package-versions to get pretty versions strings
justinrainbow/json-schema            5.2.10             A library to validate a json schema.
knuckleswtf/pastel                   1.3.6              Write your API docs in Markdown and get them converted into pretty HTML🎨
knuckleswtf/scribe                   2.5.1              Generate API documentation for humans from your Laravel codebase.✍
lanin/laravel-api-debugger           4.0.0              Easily debug your JSON API.
laravel/framework                    v8.17.1            The Laravel Framework.
laravel/nova                         3.16.2             A wonderful administration interface for Laravel.
laravel/passport                     v10.1.2            Laravel Passport provides OAuth2 server support to Laravel.
laravel/telescope                    v4.3.1             An elegant debug assistant for the Laravel framework.
laravel/tinker                       v2.5.0             Powerful REPL for the Laravel framework.
laravel/ui                           v3.1.0             Laravel UI utilities and presets.
lcobucci/clock                       2.0.0              Yet another clock abstraction
lcobucci/jwt                         4.1.2              A simple library to work with JSON Web Token and JSON Web Signature
league/commonmark                    1.5.7              Highly-extensible PHP Markdown parser which fully supports the CommonMark spec and Github-Flavored Markdown (GFM)
league/event                         2.2.0              Event package
league/flysystem                     1.1.3              Filesystem abstraction: Many filesystems, one API.
league/flysystem-aws-s3-v3           1.0.29             Flysystem adapter for the AWS S3 SDK v3.x
league/flysystem-memory              1.0.2              An in-memory adapter for Flysystem.
league/mime-type-detection           1.5.1              Mime-type detection for Flysystem
league/oauth2-server                 8.2.4              A lightweight and powerful OAuth 2.0 authorization and resource server library with support for all the core specification...
maatwebsite/excel                    3.1.26             Supercharged Excel exports and imports in Laravel
maennchen/zipstream-php              2.1.0              ZipStream is a library for dynamically streaming dynamic zip files from PHP without writing to the disk at all on the server.
markbaker/complex                    2.0.0              PHP Class for working with complex numbers
markbaker/matrix                     2.0.0              PHP Class for working with matrices
mnapoli/front-yaml                   1.7.0             
mnapoli/silly                        1.7.2              Silly CLI micro-framework based on Symfony Console
mockery/mockery                      1.4.2              Mockery is a simple yet flexible PHP mock object framework
monolog/monolog                      2.1.1              Sends your logs to files, sockets, inboxes, databases and various web services
moontoast/math                       1.2.1              A mathematics library, providing functionality for large numbers
mpociot/reflection-docblock          1.0.1             
mtdowling/jmespath.php               2.6.0              Declaratively specify how to extract elements from a JSON document
myclabs/deep-copy                    1.10.2             Create deep copies (clones) of your objects
myclabs/php-enum                     1.7.7              PHP Enum implementation
nesbot/carbon                        2.42.0             An API extension for DateTime that supports 281 different languages.
nikic/php-parser                     v4.10.2            A PHP parser written in PHP
nunomaduro/collision                 v5.1.0             Cli error handling for console/command-line PHP applications.
nyholm/psr7                          1.4.0              A fast PHP7 implementation of PSR-7
opis/closure                         3.6.1              A library that can be used to serialize closures (anonymous functions) and arbitrary objects.
paragonie/random_compat              v9.99.100          PHP 5.x polyfill for random_bytes() and random_int() from PHP 7
phar-io/manifest                     2.0.1              Component for reading phar.io manifest information from a PHP Archive (PHAR)
phar-io/version                      3.0.3              Library for handling version information and constraints
php-cs-fixer/diff                    v1.3.1             sebastian/diff v2 backport support for PHP5.6
php-di/invoker                       2.2.0              Generic and extensible callable invoker
phpdocumentor/reflection-common      2.2.0              Common reflection classes used by phpdocumentor to reflect the code structure
phpdocumentor/reflection-docblock    5.2.2              With this component, a library can provide support for annotations via DocBlocks or otherwise retrieve information that is...
phpdocumentor/type-resolver          1.4.0              A PSR-5 based resolver of Class names, Types and Structural Element Names
php-http/client-common               2.3.0              Common HTTP Client implementations and tools for HTTPlug
php-http/discovery                   1.13.0             Finds installed HTTPlug implementations and PSR-7 message factories
php-http/httplug                     2.2.0              HTTPlug, the HTTP client abstraction for PHP
php-http/message                     1.10.0             HTTP Message related tools
php-http/message-factory             v1.0.2             Factory interfaces for PSR-7 HTTP Message
php-http/promise                     1.1.0              Promise used for asynchronous HTTP requests
phpoffice/phpspreadsheet             1.15.0             PHPSpreadsheet - Read, Create and Write Spreadsheet documents in PHP - Spreadsheet engine
phpoption/phpoption                  1.7.5              Option Type for PHP
phpseclib/bcmath_compat              1.0.5              PHP 5.x/7.x polyfill for bcmath extension
phpseclib/phpseclib                  2.0.29             PHP Secure Communications Library - Pure-PHP implementations of RSA, AES, SSH2, SFTP, X.509 etc.
phpspec/prophecy                     1.12.1             Highly opinionated mocking framework for PHP 5.3+
phpunit/php-code-coverage            9.2.5              Library that provides collection, processing, and rendering functionality for PHP code coverage information.
phpunit/php-file-iterator            3.0.5              FilterIterator implementation that filters files based on a list of suffixes.
phpunit/php-invoker                  3.1.1              Invoke callables with a timeout
phpunit/php-text-template            2.0.4              Simple template engine.
phpunit/php-timer                    5.0.3              Utility class for timing
phpunit/phpunit                      9.5.2              The PHP Unit Testing framework.
predis/predis                        v1.1.6             Flexible and feature-complete Redis client for PHP and HHVM
psr/container                        1.0.0              Common Container Interface (PHP FIG PSR-11)
psr/event-dispatcher                 1.0.0              Standard interfaces for event handling.
psr/http-client                      1.0.1              Common interface for HTTP clients
psr/http-factory                     1.0.1              Common interfaces for PSR-7 HTTP message factories
psr/http-message                     1.0.1              Common interface for HTTP messages
psr/log                              1.1.3              Common interface for logging libraries
psr/simple-cache                     1.0.1              Common interfaces for simple caching
psy/psysh                            v0.10.4            An interactive shell for modern PHP.
ralouphie/getallheaders              3.0.3              A polyfill for getallheaders.
ramsey/collection                    1.1.1              A PHP 7.2+ library for representing and manipulating collections.
ramsey/uuid                          4.1.1              A PHP library for generating and working with universally unique identifiers (UUIDs).
react/promise                        v2.8.0             A lightweight implementation of CommonJS Promises/A for PHP
roave/security-advisories            dev-master 676668e Prevents installation of composer packages with known security vulnerabilities: no API, simply require it
sebastian/cli-parser                 1.0.1              Library for parsing CLI options
sebastian/code-unit                  1.0.8              Collection of value objects that represent the PHP code units
sebastian/code-unit-reverse-lookup   2.0.3              Looks up which function or method a line of code belongs to
sebastian/comparator                 4.0.6              Provides the functionality to compare PHP values for equality
sebastian/complexity                 2.0.2              Library for calculating the complexity of PHP code units
sebastian/diff                       4.0.4              Diff implementation
sebastian/environment                5.1.3              Provides functionality to handle HHVM/PHP environments
sebastian/exporter                   4.0.3              Provides the functionality to export PHP variables for visualization
sebastian/global-state               5.0.2              Snapshotting of global state
sebastian/lines-of-code              1.0.3              Library for counting the lines of code in PHP source code
sebastian/object-enumerator          4.0.4              Traverses array structures and object graphs to enumerate all referenced objects
sebastian/object-reflector           2.0.4              Allows reflection of object attributes, including inherited and non-public ones
sebastian/recursion-context          4.0.4              Provides functionality to recursively process PHP variables
sebastian/resource-operations        3.0.3              Provides a list of PHP built-in functions that operate on resources
sebastian/type                       2.3.1              Collection of value objects that represent the types of the PHP type system
sebastian/version                    3.0.2              Library that helps with managing the version number of Git-hosted PHP projects
seld/jsonlint                        1.8.3              JSON Linter
seld/phar-utils                      1.1.1              PHAR file format utilities, for when PHP phars you up
sentry/sdk                           3.1.0              This is a metapackage shipping sentry/sentry with a recommended HTTP client.
sentry/sentry                        3.1.0              A PHP SDK for Sentry (http://sentry.io)
sentry/sentry-laravel                2.3.0              Laravel SDK for Sentry (https://sentry.io)
shalvah/clara                        2.6.0              🔊 Simple, pretty, testable console output for CLI apps.
spatie/laravel-query-builder         3.3.4              Easily build Eloquent queries from API requests
spatie/once                          2.2.1              A magic memoization function
spatie/phpunit-snapshot-assertions   4.2.4              Snapshot testing with PHPUnit
swagger-api/swagger-ui               v3.37.2             Swagger UI is a collection of HTML, Javascript, and CSS assets that dynamically generate beautiful documentation from a S...
swiftmailer/swiftmailer              v6.2.3             Swiftmailer, free feature-rich PHP mailer
swooletw/laravel-swoole              v2.6.68            High performance HTTP server based on Swoole. Speed up your Laravel and Lumen applications.
symfony/console                      v5.2.0             Symfony Console Component
symfony/css-selector                 v5.2.0             Symfony CssSelector Component
symfony/deprecation-contracts        v2.2.0             A generic function and convention to trigger deprecation notices
symfony/error-handler                v5.2.0             Symfony ErrorHandler Component
symfony/event-dispatcher-contracts   v2.2.0             Generic abstractions related to dispatching event
symfony/event-dispatcher             v5.2.0             Symfony EventDispatcher Component
symfony/filesystem                   v5.2.0             Symfony Filesystem Component
symfony/finder                       v5.2.0             Symfony Finder Component
symfony/http-client-contracts        v2.3.1             Generic abstractions related to HTTP clients
symfony/http-client                  v5.2.0             Symfony HttpClient component
symfony/http-foundation              v5.2.0             Symfony HttpFoundation Component
symfony/http-kernel                  v5.2.0             Symfony HttpKernel Component
symfony/intl                         v5.2.0             A PHP replacement layer for the C intl extension that includes additional data from the ICU library.
symfony/mime                         v5.2.0             A library to manipulate MIME messages
symfony/options-resolver             v5.2.0             Symfony OptionsResolver Component
symfony/polyfill-ctype               v1.20.0            Symfony polyfill for ctype functions
symfony/polyfill-iconv               v1.20.0            Symfony polyfill for the Iconv extension
symfony/polyfill-intl-grapheme       v1.20.0            Symfony polyfill for intl's grapheme_* functions
symfony/polyfill-intl-icu            v1.20.0            Symfony polyfill for intl's ICU-related data and classes
symfony/polyfill-intl-idn            v1.20.0            Symfony polyfill for intl's idn_to_ascii and idn_to_utf8 functions
symfony/polyfill-intl-normalizer     v1.20.0            Symfony polyfill for intl's Normalizer class and related functions
symfony/polyfill-mbstring            v1.20.0            Symfony polyfill for the Mbstring extension
symfony/polyfill-php70               v1.20.0            Symfony polyfill backporting some PHP 7.0+ features to lower PHP versions
symfony/polyfill-php72               v1.20.0            Symfony polyfill backporting some PHP 7.2+ features to lower PHP versions
symfony/polyfill-php73               v1.20.0            Symfony polyfill backporting some PHP 7.3+ features to lower PHP versions
symfony/polyfill-php80               v1.20.0            Symfony polyfill backporting some PHP 8.0+ features to lower PHP versions
symfony/polyfill-uuid                v1.20.0            Symfony polyfill for uuid functions
symfony/process                      v5.2.0             Symfony Process Component
symfony/property-access              v5.2.0             Symfony PropertyAccess Component
symfony/property-info                v5.2.0             Symfony Property Info Component
symfony/psr-http-message-bridge      v2.1.0             PSR HTTP message bridge
symfony/routing                      v5.2.0             Symfony Routing Component
symfony/serializer                   v5.2.0             Symfony Serializer Component
symfony/service-contracts            v2.2.0             Generic abstractions related to writing services
symfony/stopwatch                    v5.2.0             Symfony Stopwatch Component
symfony/string                       v5.2.0             Symfony String component
symfony/translation-contracts        v2.3.0             Generic abstractions related to translation
symfony/translation                  v5.2.0             Symfony Translation Component
symfony/var-dumper                   v5.2.0             Symfony mechanism for exploring and dumping PHP variables
symfony/var-exporter                 v5.2.0             A blend of var_export() + serialize() to turn any serializable data structure to plain PHP code
symfony/yaml                         v5.2.0             Symfony Yaml Component
theseer/tokenizer                    1.2.0              A small library for converting tokenized PHP source code into XML and potentially other formats
tijsverkoyen/css-to-inline-styles    2.2.3              CssToInlineStyles is a class that enables you to convert HTML-pages/files into HTML-pages/files with inline styles. This i...
vlucas/phpdotenv                     v5.2.0             Loads environment variables from `.env` to `getenv()`, `$_ENV` and `$_SERVER` automagically.
voku/portable-ascii                  1.5.6              Portable ASCII library - performance optimized (ascii) string functions for php.
webmozart/assert                     1.9.1              Assertions to validate method input/output with nice error messages.
windwalker/renderer                  3.5.22             Windwalker Renderer package
windwalker/structure                 3.5.22             Windwalker Structure package
zircote/swagger-php                  3.1.0              swagger-php - Generate interactive documentation for your RESTful API using phpdoc annotations

Summary

I noticed while debugging a little that for some reason PHPUnit seems to take the list of tests in a testsuite and further wrap each test in another testsuite. So you get this hierarcy:

- Testsuite A
  |- TestSuite 1
     |- Test 1
  |- TestSuite 2
     |- Test 2
  |- TestSuite 3
     |- Test 3
- Testsuite B
  |- TestSuite 4
     |- Test 4
  |- TestSuite 5
     |- Test 5
  |- TestSuite 6
     |- Test 6

You can see here this list of "testsuites". They're actually supposed to be the "tests", but each is wrapped in a testsuite:

The problem is that the logic for sorting the execution order seems to be done locally to the testsuite. Since each test ends up along wrapped in a single dedicated "test suite", what ends up happening is each test gets "sorted" alone separately, so basically nothing gets sorted. Here is an example of "sorting" about to happen for a single test (I have 10+ tests).

I'm expecting for

Current behavior

Each test gets wrapped in its own test suite, and thus sorting does not work.

How to reproduce

Create any two PHPUnit tests under the same test suite. Run them with --order-by=duration. Debug at \PHPUnit\Runner\TestSuiteSorter::sortByDuration() and see that the tests array always has a single element in it. So sorting would do nothing.

Expected behavior

The tree should appear like so:

- Testsuite A
     |- Test 1
     |- Test 2
     |- Test 3
- Testsuite B
     |- Test 4
     |- Test 5
     |- Test 6

Without the additional wrapping of tests into testsuites. And putting a breakpoint at a sorting method, multiple tests should appear, not only just one.

sebastianbergmann / phpunit