Decide on testing framework (not urgent)

[damiansteiger]

We should decide on which testing framework we want to support over the long term. It is not urgent but we shouldn't forget about it. Does somebody already have a preference? As we stick for the moment to unittest style only, we can switch at any point in time.

ProjectQ uses pytest instead of unittest. pytest has some nice features like parametrized tests:

@pytest.mark.parametrize("angle", [0, 0.2, 2.1, 4.1, 2* math.pi,
    4 * math.pi])
def test_rx(angle):
    gate = _gates.Rx(angle)
    expected_matrix = np.matrix([[math.cos(0.5 * angle),
                       -1j * math.sin(0.5 * angle)],
                      [-1j * math.sin(0.5 * angle),
                       math.cos(0.5 * angle)]])
    assert gate.matrix.shape == expected_matrix.shape
    assert np.allclose(gate.matrix, expected_matrix)

and does not require (though still possible) the boilerplate of unittest with deriving from unittest.TestCase:

class LocalOperatorsTest(unittest.TestCase):

pytest supports pytest and unittest style unittest cannot run pytest style

[maffoo]

I'd vote for pytest. I think the test parametrization and some other features are nice, but the biggest advantage IMO is the reduced boilerplate, both in defining tests and in being able to use plain assert statements.

[jarrodmcc]

Would this allow better organization of tests like hydrogen_integration_test also? Currently everything is in one test there to avoid multiple calls to the set up procedure, but that makes it annoying to understand what failed when part of it actually does, or if multiple parts fail to display all of them rather than just the first. This will be even more true as we add tests for larger systems. If it allows this, then I also vote for the move to pytest.

[maffoo]

Even with the existing unittest library we could change that test to call setUpClass instead of setUp. The former gets run once for the entire TestCase instance and can store state on the instance, but you can have separate test methods that are more granular but reuse the same state.

[babbush]

That is a really good idea. Is this setUpClass for the slow test cases such as hydrogen_integration_test something you can help us with @maffoo ? By the way, if not much seems to be going on with FermiLib it is because the integration with ProjectQ is currently happening in a development repo of ProjectQ that Damian will update Tuesday. We will then add everybody with a CLA. At that point, @jarrodmcc , @idk3 and myself will be working to add a couple modules and a bunch of examples which show how to use VQE and Trotter/PEA with FermiLib/ProjectQ.