Using a normal composer install means that while the build was testing against various PHP versions, it would only ever test against the latest PHPCS 3.x version (3.5.3 at the time of writing) for PHP 5.4-7.4 and PHPCS 2.9.2 for PHP 5.3.
This PR changes the build matrix to test against a variety of PHPCS versions to try to ensure compatibility with all versions this plugin claims to support.
Some explanation is needed about the choices made in this changeset:
Adding an install section
The install section is intended for installing prerequisites for test build tests.
AFAICS, this wasn't previously used to avoid the PHP lint command running over the vendor directory, but that can be solved by just removing the vendor directory from the list PHP lint is run over.
Having the install instructions in the install section makes it clearer that the installation is not part of the tests and what's being installed.
Changes to the PHP lint command
As mentioned above, the command now excludes the vendor directory.
Also, as most PHP versions now have a second build, adding a LINT environment variable so the PHP linting is only done on one build per PHP version.
Changes to the PHPCS run test command
The PSR12 ruleset which is part of the phpcs.xml.dist ruleset was only added in PHPCS 3.3.0.
As testing is now done against a wider range of PHPCS versions, this PSR12 ruleset won't always be available, so I've added a PHPCS environment variable to only run the PHPCS check against the repo specific ruleset once per build.
I've added this variable to a build against the latest PHPCS version. While I'd like to use dev-master for this, that may be prone to intermittent bugs in PHPCS, so using a stable PHPCS version instead makes the build more sane.
On all the other builds, only a check against PHPCompatibility is run which then tests whether the setting of the installed_paths for external standards worked correctly.
Now there are two caveats to this:
PHPCompatibility 9.0.0 dropped support for PHPCS < 2.3.0.
For testing against PHPCS 2.2.0 - 2.3.0 we need to explicitly install PHPCompatibility 8.x.
For testing against PHPCS < 2.2.0 we need to explicitly install PHPCompatibility 7.x, however PHPCompatibility 7.x wasn't compatible with Composer installs yet, so in that case we also need to rename the directory within vendor to make things work.
Note/advance notice: PHPCompatibility 10.0.0, which is expected over the next few months, will drop support for PHPCS < 2.6.0 and PHP < 5.4.
Also see: https://github.com/PHPCompatibility/PHPCompatibility/issues/835
The PHPCS --exclude option previously used to exclude the PHPCompatibility.Upgrade.LowPHPCS sniff is only available since PHPCS 2.6.2, so cannot be used to exclude that sniff on older PHPCS versions.
With that in mind, switching to using the --sniffs= command line option to limit the PHPCS run to just one sniff.
This option has been available since PHPCS 1.x (or even before), so can be used without problem.
Additionally, limiting the test run to just one sniff will make it faster as well.
I've just selected one of the sniffs from PHPCompatibility which has been around for a while for this. The actual sniff doesn't matter after all, as this test is just about PHPCS recognizing the external standard correctly.
Custom PHP 5.3 script
As the script for the PHPCS run has changed now, there is no need for the custom script section for the PHP 5.3 build anymore. This can now revert back to using the normal test script again, so to that end, I've remove the custom script.
Future improvements
This is a first iteration to improve the build script.
In a further iteration, I'd like to suggest exploring the following additional variants:
Having some builds with a global install of PHPCS + global install of the external standard used for testing.
Having some builds with a mix of global/local installs of external standards(s).
Having some builds using Windows as OS instead of Linux.
Proposed Changes
Using a normal
composer installmeans that while the build was testing against various PHP versions, it would only ever test against the latest PHPCS 3.x version (3.5.3at the time of writing) for PHP 5.4-7.4 and PHPCS 2.9.2 for PHP 5.3.This PR changes the build matrix to test against a variety of PHPCS versions to try to ensure compatibility with all versions this plugin claims to support.
Some explanation is needed about the choices made in this changeset:
Adding an
installsectionThe
installsection is intended for installing prerequisites for test build tests.AFAICS, this wasn't previously used to avoid the PHP
lintcommand running over thevendordirectory, but that can be solved by just removing thevendordirectory from the list PHPlintis run over.Having the install instructions in the
installsection makes it clearer that the installation is not part of the tests and what's being installed.Changes to the PHP
lintcommandAs mentioned above, the command now excludes the
vendordirectory.Also, as most PHP versions now have a second build, adding a
LINTenvironment variable so the PHP linting is only done on one build per PHP version.Changes to the PHPCS run test command
The PSR12 ruleset which is part of the
phpcs.xml.distruleset was only added in PHPCS 3.3.0.As testing is now done against a wider range of PHPCS versions, this
PSR12ruleset won't always be available, so I've added aPHPCSenvironment variable to only run the PHPCS check against the repo specific ruleset once per build.I've added this variable to a build against the latest PHPCS version. While I'd like to use
dev-masterfor this, that may be prone to intermittent bugs in PHPCS, so using a stable PHPCS version instead makes the build more sane.On all the other builds, only a check against
PHPCompatibilityis run which then tests whether the setting of theinstalled_pathsfor external standards worked correctly.Now there are two caveats to this:
vendorto make things work. Note/advance notice: PHPCompatibility 10.0.0, which is expected over the next few months, will drop support for PHPCS < 2.6.0 and PHP < 5.4. Also see: https://github.com/PHPCompatibility/PHPCompatibility/issues/835--excludeoption previously used to exclude thePHPCompatibility.Upgrade.LowPHPCSsniff is only available since PHPCS 2.6.2, so cannot be used to exclude that sniff on older PHPCS versions. With that in mind, switching to using the--sniffs=command line option to limit the PHPCS run to just one sniff. This option has been available since PHPCS 1.x (or even before), so can be used without problem. Additionally, limiting the test run to just one sniff will make it faster as well. I've just selected one of the sniffs from PHPCompatibility which has been around for a while for this. The actual sniff doesn't matter after all, as this test is just about PHPCS recognizing the external standard correctly.Custom PHP 5.3 script
As the script for the PHPCS run has changed now, there is no need for the custom
scriptsection for the PHP 5.3 build anymore. This can now revert back to using the normal testscriptagain, so to that end, I've remove the custom script.Future improvements
This is a first iteration to improve the build script.
In a further iteration, I'd like to suggest exploring the following additional variants: