Test GloFAS processing step

[turnerm]

A couple of questions:

• Is there a better way to organize this without the class structure just using pytest fixtures?
• Is there a way to put tmp_pth in the decorator or somewhere else to avoid putting everything in with self.mock_data_dir?

[mcarans]

Sorry that I have taken so long to look at this PR. I need access to the Google Drive folder (and have requested it) in order to download the data needed to run the test (AA_DATA_DIR).

On your questions: pytest does not require the use of classes. You can just use functions. However, I'd suggest that if there are multiple functions, that they be grouped in a class.

Something that may be useful is that pytest allows for a file called conftest.py that can contain fixtures which are shared between different sets of tests. It can be placed at the root of the tests folder and fixtures in there are accessible to all tests.

I have utilities to deal with temporary folders. See: https://github.com/OCHA-DAP/hdx-python-utilities#path-utilities

Specifically I'd suggest something like the codebelow. The temporary folder AATest is created inside the tmp folder (or the folder defined in the environment variable TEMP_DIR if you have set it). If it already exists, it is recreated. Since the scope of the fixture is class, it is called on class setup (so the same temp_dir can be used in all functions in the class). If the tests exit successfully AATest is deleted. If not, it remains (which can be helpful for debugging test failures).

The pytest scopes are: function: the default scope, the fixture is destroyed at the end of the test.

class: the fixture is destroyed during teardown of the last test in the class.

module: the fixture is destroyed during teardown of the last test in the module.

package: the fixture is destroyed during teardown of the last test in the package.

session: the fixture is destroyed at the end of the test session

(Fixtures are created the first time they are accessed and then cached unless you specify autouse=True in which case they are created as soon as the scope they are in is in scope.)

@pytest.fixture(scope="class")
def temp_dir(self):
    tempdir = get_temp_dir("AATest", delete_if_exists=True)
    yield tempdir
    rmtree(tempdir)

[mcarans]

I've made the tests use pytest fixtures and pytest-mock, made pytest recognise the src folder, changed to relative imports and removing "src." from imports. I've also ensured that there is no dependency on the existence of the environment variable AA_DATA_DIR (which is automatically set for all tests to a folder in /tmp) or on .cdsapirc in the home folder to run the tests.

[mcarans]

I've merged all the changes from the develop branch into this branch and the tests pass. This PR is ready to be merged.

[turnerm]

Superseded by #229