Tests - what to check and what to test?

[nannau]

I'm trying to identify the differences between checks and tests. Since I am new to testing, I've added a few checks and tests to the data loading part of climpyrical.

My understanding of tests vs. checks is rudimentary, however, is this in essence:

• A test is meant to verify that code is working the way you expect it to be and is producing outputs that are logical and as expected

• A check is meant to verify that the code is being used properly and protects it from wandering inputs

In this PR I have two places where I implement checks and tests. The checks remain in datacube.py and are at the top of the file.

Please let me know if you have any input on this!

[corviday]

I really, really like pytest. If it fits with the way you have code organized at all, I'd suggest it.

EDIT: I see you are using it, it just wasn't listed in requirements.txt where I checked. Never mind!

[rod-glover]

Yes:

• A test is code that exercises other (application) code and compares its actual behaviour to expected/specified behaviour. It's easiest when the behaviour is "return this value" but it can, more or less, be anything, e.g., "insert this value in a database."

• What you are calling a check is perhaps more often called input validation: Checks input (external info, from command line or an input file) for validity, and usually emits a helpful ( :) ) error message and stops the program.

  • This is pretty much what you have here.
  • The error should be correctable by the user, e.g., by correcting an invalid input value.

• Finally, a second type of "check": Sanity checking or assertions: Checks inserted into the code that verify that you are doing what you think you are doing.

  • Example: In the course of your code's exection you compute a value x that should be between 0 and 1. You could insert a sanity check just after it is computed reading assert 0 <= x < 1, and this would raise an AssertionError should x not be in the expected range. This is not user-friendly, and should only be part of development and debugging and not expected to be encountered in the normal flow of usage.
    • Better yet would be to isolate the computation of x in a function and add tests of that function to verify that it works correctly for all (valid) inputs.
    • Using Assertions Effectively is worth a quick read.

[rod-glover]

Well, @corviday got there ahead of me re. parametrizing, but note he is suggesting parametrizing the path, and not the var. In fact you should parametrize both.