Unit Test Structuring

[robertmain]

Apologies if this isn't the right place to bring this up, but I wondered if we could talk about the testing setup and configuration?

Looking around there seem to be scripts for configuring the database and downloading WP etc. and I was curious as to why for unit tests. Surely we could just mock out/stub the relevant system calls needed in each unit?

This would also make the tests much faster to run.

Examples:

• https://github.com/WP-for-Church/Sermon-Manager/blob/dev/bin/install-wp-tests.sh
• https://github.com/WP-for-Church/Sermon-Manager/blob/dev/tests/bootstrap.php

[nikola3244]

By using those scripts, we can easily set up a complete WordPress instance for testing, so we have calls for any WordPress functions - and they would work across all tests. I.e. if one test modifies meta field data, the other test would have access to it later.

Yeah, we could mock out things, but I see more downsides than advantages of doing so. If nothing else, it would take time to create, than to use the exisitng WordPress functions.

• https://premium.wpmudev.org/blog/unit-testing-wordpress-plugins-phpunit/
• https://make.wordpress.org/cli/handbook/plugin-unit-tests/

[robertmain]

Right, but you don't "create" the wordpress functions - you just stub the function out for that particular scenario.

Also, what you're proposing isn't a unit test, but a system test, except that because you're including wordpress like that your tests are actually covering wordpress functions too. Ideally you want to mock those out so that you aren't covering third-party code.

[nikola3244]

True, but most functions we use are depending on WordPress - since it's a WordPress plugin. :) For example, we can't create sermon post type/taxonomies without WordPress, or test meta fields, etc...

If this was a plain PHP application, it would be much easier, but alas - it's not.

Anyway, if you think that you have a better idea (that would take less time/effort), I would love to hear it and try it. :slightly_smiling_face: (some working prototype would be great too)

[robertmain]

Right, but you don't have to actually create a sermon post. You just want to test that your code calls the correct WP functions with the correct arguments. If those aren't working, then we have much bigger problems that are outside the scope of this plugin to fix...

You can see an example of this here.

I'm stubbing out the get_by function my user model inherits from the parent class (since I don't need to test the parent class's code). I'm asserting that:

1. The get_by function will be called with exactly one argument
2. That argument will be an array
3. This argument will contain the key of "username" and the value of "mrtesting"
4. Then, I go on and basically "frig it" to return a predetermined user object I've configured above

So let's say you have a hypothetical function that returns an array of sermons. Here's how you could do it. The problem is though that in order to test that, you have to have sermons in the database to start with... Not only that but your test is dependent upon either setup code to insert a bunch of sermons, or (worse) another test that tests the "Add" functionality. It's bad practice for tests to be dependent upon one another. It's also bad practice to have tests that run slowly. Why? Because, if they take a long time to run, you'll run them less often. So mocking out the wordpress function calls makes your tests faster and more robust. See here. I'm happy to put a simple example together for you if you like.

So what you could do is mock out the $wpdb->query() function in your test so that it returns an array of sermon objects. Now your test doesn't need to be connected to the database - in fact it may not even require wordpress. This is good because you can run your tests with less setup and faster.

I hope all that makes sense :)

[nikola3244]

You have definitely convinced me now :wink:

Honestly, I have been avoiding setting up WordPress testing for a while, because it looks like it would take a lot of time, but when you put it like this - doesn't sound like it would take much

Let's do it soon!

[robertmain]

Yeah :)

Forgot to also say that tests like you describe do have merit. But they aren't unit tests. They're system tests.

Kevlin Henney has some really good talks on the subject of tests