Changing to use native pytest

[damiansteiger]

We have discussed this issue before the release and decided to switch to pytest because

• No boilerplate
• Better options share test fixtures (more advances version of unittest's setup)
• Parametrized tests
• More sensible error messages in certain cases

For example code, browse the tests in ProjectQ.

Our continuous integration testing is therefore since the beginning using pytest instead of unittest. Pytest allows to run the standard unittest files for backwards compatibility and unfortunately our tests are still using unittest.

Could we decide that if we write completely new test files, that these use pytest and not unittest? Obviously it would also be great if the other tests at some point are refactored but this is not an important issue but we should start somewhere now. I know this means some people will have to learn pytest, but it is easy (see https://docs.pytest.org/en/latest/) and there are already great test examples in ProjectQ for pretty much all cases.

It would also help new users that they see that the best way to testing is going with pytest. For example in #123

self.assertTrue(self.assertTrue(evensector == 2**(n_qubits - 1))

was used and naturally it doesn't give a sensible error message (False not equal True). Of course one can use self.assertEqual instead to get a better error message but such a thing would not have happened with pytest as it is without boilerplate:

assert evensector == 2**(n_qubits - 1)

@maffoo: I remember you were also in favour of pytest. Anymore good reasons to switch?

[maffoo]

I think the reasons you've outlined capture the things I like about pytest fairly well. In particular, the ability to use bare asserts makes the test code quite a bit more clear IMO.

One thing I would advise is to be careful about not using too much pytest "magic"; in particular, fixtures can sometimes be overused in ways that make test code hard to follow because too much is happening behind the scenes.

On a related note, pytest gives a lot of flexibility, for example it allows writing tests as module-level functions or in a more unittest-like style where tests are methods on a class. This goes somewhat against the "there's only one way to do it" ethos of python, and it's easy to accumulate tests in a project that use different styles and "look" very different from each other. When adopting pytest it's good to pick a testing style and limit use of some of the more esoteric pytest features so the test code stays reasonably consistent.

[babbush]

I am fine with this so long as we are consistent. Is somebody willing to go through and change all the unittests to pytests?

[damiansteiger]

I would give it a shot if people are okay with changing...

Thanks @maffoo for your suggestions. I would go with module-level functions style because it has less boilerplate and is what we are already doing in ProjectQ.

[babbush]

Yeah I think everyone is cool with this. Go for it Damian!

[damiansteiger]

Cool, so we all agree to use pytest. Unfortunately, a couple of other projects popped up which need to be done before I can work on this...