Add support for PHPUnit testing using WordPress-develop-tests

[bobbingwide]

WordPress core is unit tested using PHPUnit. A number of popular plugins contain unit tests which are run under PHPUnit, using the core's test framework.

Traditionally, testing is performed in an empty sand box. The WordPress approach tests core functions not plugins or themes. It uses factory functions to create minimal amounts of test data.

With oik-batch I want to take a slightly different approach; test with live data, This means testing with plugins and themes active and data in the database: posts, users, comments, taxonomies, options, etcetera.

The primary purpose of the oik-batch PHPUnit logic is to provide a consistent framework that can be used by other plugins and themes. Each plugin and theme will use oik-batch's bootstrap logic to load the current WordPress environment, and the required functionality from WordPress-develop-tests.

In the event of a test failure, on restart it will also perform some basic teardown logic, attempting to restore the database to the original starting point. There will of course be limitations. In the event of a catastrope a database backup will be required. BAU.

[bobbingwide]

One of the challenges of working with PHPUnit is that it doesn't respect the fact that you're running the code from a symlink'ed directory.

Note: I'm using Windows so sometimes I'll be writing backslashes.

My development projects are installed in sub-directories of Apache's document root ( C:/apache/htdocs )

e.g. wordpress, wpms, src, oikcom are all different WordPress installations. For most of them the wp-config.php is where you'd expect it to be. C:/apache/htdocs/wordpress/

In order to run a batch routine under oik-wp.php the steps are:

1. change directory into the required installation's subdirectory
2. run oik-wp.php passing parameters to indicate what to run

oik-wp.php looks up the directory tree to find the wp-config.php file it should use and sets ABSPATH accordingly.

Now, in each installation, I use symlinks for each of the plugins back to the original Git clone of my plugin's repositories, which are in the wordpress installation e.g.

<JUNCTION>  bbboing [C:\apache\htdocs\wordpress\wp-content\plugins\bbboing]
<JUNCTION>  oik [C:\apache\htdocs\wordpress\wp-content\plugins\oik]
 <DIR>          oik-batch
<JUNCTION>  wordpress-develop-tests [C:\apache\htdocs\wordpress\wp-content\plugins\wordpress-develop-tests]

Note: As a first attempt at resolving the problem oik-batch is not symlinked; it's another clone of a Git repo.

When I want to run PHPUnit in situ the steps are:

1. change directory into the required installation's subdirectory
2. run phpunit

For each plugin with PHPUnit tests the phpunit.xml file is expected to be the same, starting

<phpunit bootstrap="../oik-batch/oik-wp.php" ...>
<testsuites>
        <testsuite>
            <directory prefix="test-" suffix=".php">tests/</directory>

oik-wp.php ( similar in some respects to WP-cli ) determines that it's being run under PHPUnit and performs the bootstrapping of the WordPress develop test functions. It'll look for them in a variety of locations - the wordpress-develop-tests plugin being the preferred starting point.

The trouble is... PHPUnit uses stream_resolve_include_path() which resolves the symlinks. Even if there's a local, non-symlinked version of oik-batch, if the plugin you're trying to test is symlinked then ../oik-batch/oik-wp.php resolves to the wordpress installation.

So for the following commands

cd \apache\htdocs\src\wp-content\plugins\bbboing
phpunit

we end up running C:\apache\htdocs\wordpress\wp-content\plugins\oik-batch\oik-wp.php with the current directory set to that path.

[bobbingwide]

My pragmatic solution is to try to use an environment variable ( PRE_PHPUNIT_CD ) to store the current directory before PHPUnit was started.

Since I'm using Windows I have a batch file called phpunit.bat which sets this environment variable before invoking PHPUnit.

setlocal
rem set WP_TESTS_DIR=C:\svn\wordpress-develop\tests\phpunit
rem WP_DEVELOP_DIR is basically the same thing
rem but we're going to rely on the 'wordpress-develop-tests' plugin
set PRE_PHPUNIT_CD=%CD%
rem Had problems with PHPUnit 4.8.0 and PHP 7 so trying PHPUnit 5.5.0
rem php c:\apache\htdocs\phpLibraries\phpunit\phpunit-4.8.0.phar "--verbose" "--log-json=phpunit.json" %*
php c:\apache\htdocs\phpLibraries\phpunit\phpunit-5.5.0.phar "--verbose" "--log-json=phpunit.json" "--disallow-test-output" %*
endlocal

[bobbingwide]

It wasn't just a simple case of performing a chdir() to the original directory before calling oik_batch_locate_wp_config() since the getcwd() will return the symlinked directory not the one we first thought of. So, to find the wp-config.php file, instead of working up the directory tree from the bottom ( which is the wrong bottom as far as we're concerned ) we start from the top and work down.

I added a new function called oik_batch_locate_wp_config_for_phpunit() which checks if oik-wp is running under phpunit before getting the PRE_PHPUNIT_CD environment variable. If it's set then we determine the path of the wp-config.php file by working downwards and using the last file found. See oik_batch_cd_drill_down()..

For my slightly complex installation, with multiple symlinks, and the following commands

cd \apache\htdocs\src\wp-content\plugins\bbboing
phpunit

we see the following results.

chdir() to	getcwd() returns	note
C:	C:\
apache	C:\apache
htdocs	C:\apache\htdocs	There's a wp-config.php file here but it's not the right one
src	c:\svn\wordpress-develop\src	symlink to WordPress source code. This is where the real wp-config.php is.
wp-content	C:\svn\wordpress-develop\src\wp-content
plugins	C:\svn\wordpress-develop\src\wp-content\plugins
bbboing	C:\apache\htdocs\wordpress\wp-content\plugins\bbboing	the symlinked plugin

oik-wp determines that the configuration file to use is the one that I can see in C:\apache\htdocs\src The correct one. Hooray! Now I can go back to having oik-batch symlinked rather than cloned.

[bobbingwide]

Since this isn't yet closed here are some more requirements.

• [x] Add support for PHPUnit testing WordPress themes.
• [x] Add support for PHPUnit testing in a WordPress Multisite environment.
• [x] Add support for PHPUnit 6.0.x ( currently 6.0.8 )

Note: In PHPUnit 6, the units of code are now namespaced. So PHPUnit_Framework_TestCase is now PHPUnit\Framework\TestCase. WordPress provides a solution to cater for this.

[bobbingwide]

For WordPress Multsite sub-sites we need to cater for the fact that attached files don't come from wp-content/uploads but from the blogs.dir directory. e.g. for qw/wpms/phphants ( site ID 9 )

[basedir] => C:\apache\htdocs\wpms/wp-content/blogs.dir/9/files
[baseurl] => https://qw/wpms/phphants/files

quickest fix is to change BW_UnitTestCase::replace_home_url() , adding

$upload_dir = wp_upload_dir();
$expected = str_replace( $upload_dir['baseurl'] , "https://qw/src/wp-content/uploads", $expected );

before the other replacements.

[bobbingwide]

I'm going to close this now.