[DX] Replace Simpletest with PHPUnit

[ghost]

Unit/functional testing in Backdrop is based on the core Testing module, which was the Simpletest contrib module in Drupal 6. This appears to be originally based on https://github.com/simpletest/simpletest.

PHPUnit seems to be the de-facto standard for unit testing for PHP nowadays, and is what Drupal 8+ uses by default (Simpletest having been deprecated).

I wonder if it's possible/worthwhile looking to eventually migrate Backdrop's tests from Simpletest to PHPUnit...?

Benefits I see in doing so:

• PHPUnit is more common (especially with Drupal's move to it), so we'd be less reliant on a custom, Drupal-specific solution
• With more documentation available for PHPUnit (and possible familiarity with it in other projects), Backdrop developers might be able to more easily write/fix tests for Backdrop (something that seems to be a real struggle for a few people currently)

[ghost]

Not sure if this might be useful... https://www.drupal.org/docs/automated-testing/converting-simpletests-to-phpunit-tests

[klonos]

I went through the instructions and the change records in Drupal, and I think that things are pretty straight-forward. Perhaps we could even whip out something like https://github.com/backdrop-contrib/coder_upgrade to help us with the conversion ...or even the existing coder_upgrade module could be used to handle the conversions?

What I haven't checked out yet is the difference in actual code between simpletest tests and PHPUnit tests. Is there still the notion of assertions, and if so are there any changes, and if so then are these documented?

I'd also like to know how long it would take for the PHPUnit tests to run compared to the current simpletest tests, as if they are considerably slower, then that might be a concern 🤔

As a final note: I have never created tests from scratch myself - I either copied and tweaked existing tests, or fixed them if code changes in PRs required that in order to get tests to pass again (all these were trivial/minor changes though). So my opinion shouldn't have as much "gravity" as the opinion of people that are more experienced and invested in writing tests.

Other than that, I have no objection to this if others also think that this would be to the benefit of Backdrop and our contributors (given that this will be a rather big refactoring of our code that is).

[klonos]

...another thing to note is that PHPUnit seems to have very frequent releases (as frequent as every day or every other day): https://github.com/sebastianbergmann/phpunit/tags. This is a good thing I guess, but we'd need to decide on a plan to be updating our version I guess.

[herbdool]

I agree at some point. It would be useful to have a proof of concept first. And then there might be a period where both libraries exist side by side.

[ghost]

@herbdool I was thinking along the same lines... Get PHPUnit working in the Testing module/UI alongside Simpletest, and via the commandline in core as part of the run-tests.sh script, then start slowly adding/migrating tests.

[docwilmot]

Re write every single Backdrop test using a new library/language? Sounds painful.

[ghost]

@docwilmot Yep, no one's saying it won't be a lot of work. But I think I'll ultimately be worthwhile (see points above).