No systematic (regression) testing done yet

[GoogleCodeExporter]

We don't really do any really good systematic testing yet, especially 
regression testing is hard because not yet automated.

We should introduce automated test cases. The only solution for 
web-applications I know is Selenium http://seleniumhq.org/.

Once we have a good base of test cases, it is a lot easier to check whether a 
certain change breaks something (else).

And I think Selenium is really cool I'd really like to use it for this project 
;-)

Original issue reported on code.google.com by crazy4ch...@gmail.com on 30 Oct 2012 at 11:19

[GoogleCodeExporter]

We could also do non-web testing, once we have a bit more structure in the 
code. We can add a noop parameter so we can include() the main file in other 
scripts, and run a normal suite of tests.

Original comment by dreadnaut on 22 Nov 2012 at 7:54

[GoogleCodeExporter]

Although normal "non-web-testing" is fine to test a bug part of the code, some 
things in the end cannot be tested.

The interplay of JavaScript and PHP for example is almost impossible to test.
And all the other UI-stuff is a lot easier to test with browser-based tests 
like Selenium.

Of course we could have unit-tests to test our classes and Selenium-tests to 
test the UI.
But we need to restructure the code quite a lot before we can do good 
unit-testing I'm afraid.

Original comment by crazy4ch...@gmail.com on 22 Nov 2012 at 10:47

[GoogleCodeExporter]

Lol. "a bug part" was meant to be "a big part" ;-)

Original comment by crazy4ch...@gmail.com on 23 Nov 2012 at 9:11

[GoogleCodeExporter]

Just noticed that PHPUnit can work with Selenium¹. I was then going to suggest 
PHPUnit as a testing framework, but the latest versions requires either php 
5.3.3 (3.7) or 5.4.7 (3.8). It's actually becoming difficult to find 
non-namespaced code (< 5.3).

Until now, I've tried to keep everything 5.2 friendly, but it would be easier 
to have different version requirements for main (single file/split file) code 
and code that is only used for development or support (build script, tests).

Preferences?

[1] http://www.phpunit.de/manual/3.7/en/selenium.html

Original comment by dreadnaut on 6 Apr 2013 at 2:12

[GoogleCodeExporter]

I think it is fine to require php 5.3.3 or even 5.4.7 for test scripts. The 
problem with php versions is that you cannot update php yourself _on shared 
hosting webservers_. But you usually do not _develop_ code on a shared hosting 
webserver but on a localhost. Therefore, you can easily update php yourself in 
your development environment.
So in my opinion, it is definitely okay to require recent php versions for test 
scripts. I mean, you can still modify phpliteadmin yourself with older php 
version, you just cannot run the tests.

I am not sure if the build script is exactly the same. Although only used by 
developers, not being able to run the build script means that you'll have a 
hard time modifying parts of PLA like the JS. I'd have a better feeling if the 
build script did not require php 5.4, but at most 5.3. Otherwise we might stop 
some people from modifying PLA.

PHPUnit definitely is the right way to test php units. I did not know so far 
that it also supports selenium. This makes it even cooler, so we cannot only 
test logic units with it but also test GUI including browser behaviour.

Original comment by crazy4ch...@gmail.com on 6 Apr 2013 at 2:25

[GoogleCodeExporter]

I'll start using PHPUnit for new classes then, we'll see how it goes. In case 
anyone in interested in the setup, I'm installing it through Composer and 
keeping tests files under 'trunk/tests/'.

composer.json
{
    "require-dev": {
        "phpunit/phpunit": "3.7.*"
    }
}

In the future, I won't commit anything under vendor/ to repository.

[For fun]
We could also experiment with BDD and Behat (http://docs.behat.org/) which also 
supports Selenium :)

Original comment by dreadnaut on 6 Apr 2013 at 3:02

[GoogleCodeExporter]

Okay. Let's see.
I think when I find the time, I will need to create a test suite for the 
alterTable function (currently part of the Database class - famous issue #12). 
When I modify it, I always fear that changing a regex slightly might crush 
other cases which I currently don't have in mind. Having a good test suite 
would be really helpful for regression testing there. In fact I consider this 
method the only one in PLA that really has some kind of complicated logic ;-)

Test cases for this should be easy to create, a CREATE TABLE statement of the 
initial table, an ALTER TABLE statement for the alter operation and a couple of 
post-checks, mainly comparing the actual result of SELECT table_info() with the 
expected one.

Original comment by crazy4ch...@gmail.com on 6 Apr 2013 at 4:00