Setup VisualPHPUnit to run new PHPUnit tests in browser

[mattab]

It would be very useful to run unit tests in the browsers:

• No need to start console line and interpret the output which can be hard to read & understand
• Easier to debug (since debugger already setup to debug browser, but not necessarily to debug cli PHP)
• we tell new devs to visit piwik/tests/ which returns a page linking to the various tests. This way it is friendly to the devs who can click & see results. If the PHPUnit must be run on command line only it adds complexity.

It seems VisualPHPUnit is a good candidate to run our PHPUnit tests. Benaka has already started to use it and is happy with it, therefore I suggest we include the lib in tests/libs/ and link it from piwik/tests/ HTML page.

[mattab]

When running tests in the browser, would it be possible to bypass the need to edit the phpunit.xml to set the HOST and REQUEST_URI or are they still required?>

Proposed:

• If the file phpunit.xml is there then use it
• If the file phpunit.xml is not found, then use HOST and REQUEST_URI found in the browser request

This way, developpers checking out fresh SVN could directly run tests in their browsers without editing any file which would be nice. Note: let's do this only if trivial, since it's not critical at all :)

[diosmosis]

Replying to matt:

When running tests in the browser, would it be possible to bypass the need to edit the phpunit.xml to set the HOST and REQUEST_URI or are they still required?>

A phpunit.xml file is not necessary if using VisualPHPUnit. It includes a file explorer widget so you can just ctrl+click to select the Core/Plugins/Integration directories then click Run Tests. See the screenshot here: https://github.com/NSinopoli/VisualPHPUnit

[diosmosis]

(In [6640]) Refs #3290, initial commit of visualphpunit. also modified tests/index.php to point to visualphpunit.

[mattab]

looks great. Some feedback:

• can you hide .svn directories from the listing
• Do you know if there is a feature to run one single test via a URL it was quite useful to just go to: http://localhost/trunk/tests/core/Period/Range.test.php

[diosmosis]

(In [6672]) Refs #3290, augmented install of visualphpunit to provide following features:

• Run tests one by one as opposed to all in one AJAX request so status of operation can be seen.
• Display error if data received from server is incorrect (which can happen when stray ob_start/exit/etc exist).
• Moved overall statistics to top of page and disable showing successes by default so running lots of tests are easier to manage.

[diosmosis]

(In [6687]) Refs #3290, allow visualphpunit to hide hidden folders (ie, .svn) in browser-based file explorer & allow users to specify tests to run via hash in URL (eg, /tests/lib/visualphpunit/#/Core/DataTableTest.php .

[diosmosis]

(In [6704]) Refs #3290, added VisualPHPUnit section to README.

[diosmosis]

(In [6726]) Refs #3290, use $.ajax instead of $.postJSON to avoid issue when jQuery tries & fails parsing.

[diosmosis]

(In [6733]) Refs #3290, show http response when visualphpunit gives HTTP error.

[mattab]

(In [6815]) Refs #3290 README tweak

[diosmosis]

(In [6818]) Refs #3290, use local JS/CSS instead of files delivered over content delivery network so internet access is not required.

[mattab]

(In [6821]) Refs #3290

• readme tweak
• not using absolute URLs since piwik can be installed in sub-dir... duplicating jquery/jquery-ui in visualphpunit

[diosmosis]

(In [6822]) Refs #3290, move 'Run Tests' button to top nav bar so it's always visible & made sure .ini.php files don't get run as tests.

[diosmosis]

(In [6838]) Refs #3290, display total execution time below test statistics.

[diosmosis]

(In [6839]) Refs #3290, avoid fadeOut flash when hiding test results after running tests.

[diosmosis]

(In [6840]) Refs #3290, fix small bug in last commit

[diosmosis]

(In [6841]) Refs #3290, fix another bug in displaying suites bug.

[diosmosis]

(In [6847]) Refs #3290, work around bug in phpunit's JSON logger & added ability to specify/override GLOBAL variables before running tests in visualphpunit.

[mattab]

Very nice and useful work on this nice addition to the piwik dev toolkit!

Awesome work. Can you please close the ticket as the task seems complete (you can still commit and Refs# the ticket once it's closed)

[diosmosis]

(In [6940]) Refs #3290, show more execution statistics including peak memory usage, memory delta, total memory delta and total elapsed time.

[diosmosis]

(In [6952]) Refs #3290, get ReleaseCheckListTest to work w/ visualphpunit.

[diosmosis]

(In [7027]) Refs #3290, #3177 added XHProf support to VisualPHPUnit. Can now profile tests & benchmarks through browser.

[diosmosis]

(In [7028]) Refs #3177, #3290 add XHProf to tests/lib and tweak visualphpunit display.

[mattab]

(In [7055]) Refs #3290 Fix link to xhprof if piwik installed in sub directory. Btw the xhprof integration is an AWESOME feature, it just worked following the tests/README instructions!! loving it :)

[diosmosis]

(In [7540]) Refs #3290, added rudimentary (but usable) diffing feature to VisualPHPUnit for use with Piwik integration tests.

[diosmosis]

(In [7541]) Refs #3290, use test suite filtering option (ie, show only successes) when showing/hiding individual tests.

[diosmosis]

(In [7610]) Refs #3290, make diffs more visible.