Robust testing (e.g. randomness)

As pointed out in #88 there can be issues related to testing (locally and in CI) randomness in mitiq. In https://github.com/unitaryfund/mitiq/pull/88#issuecomment-613135104 one issue seems related to seeds.

There are various options that can be considered, such as setting the seeds for testing.

stick to pytest and be creative

A quick fix in testing may be to add randomly generated data that then is pointed at or fix seeds, https://github.com/unitaryfund/mitiq/pull/88#issuecomment-613410866. Pros: fast (?) Cons: technical debt.

pytest-randomly plugin

pytest-randomly is a pytest plugin that addresses this kind of issues. It allows to control random.seed, rather than numpy.random.seed.
Pros: pytest plugin, supports doctest. Cons: not super popular, not designed for numpy

property-based testing with Hypothesis

Hypothesis is a library that aims at changing the way tests are designed, allegedly: it should go from testing an instance to designing a test that applies to a domain of instances (property-based testing). It is not super clear to me right now.
It contains various randomness-related features, including a seed function.

Pros: sounds powerful and clever, popular and growing, well documented, more robust even beyond this randomness problems. Cons: radical change of testing framework (?), steep learning curve (?), overkill (?).

I also admit I used nose until recently / used with pytest tests thought for nose, without taking advantage of pytest full power.

I have probably found how to fix the seed problem with our tests. This is just a quick solution to fix our problems and doesn't preclude investigating also the alternative proposals of @nathanshammah.

The origin of our problem is that we should not use np.random.seed(SOME_SEED) in our code. The reason is that this will set a seed for the global state of numpy which is shared between different tests and will create problems. (see e.g. this)
The safe way of setting the seed is to initialize a local variable rng = np.random.RandomState(SOME_SEED) and then use rng to generate random numbers. E.g. random_number = rng.normal().

I locally tested this approach in test_factories.py of the PR #88, it solved the problem of the failing test.

unitaryfund / mitiq