Yelp / MOE

A global, black box optimization engine for real world metric optimization.
Other
1.31k stars 139 forks source link

[Python] set up common pytest fixtures #409

Open suntzu86 opened 9 years ago

suntzu86 commented 9 years ago

pytest provides a powerful fixtures framework: http://pytest.org/latest/fixture.html

We should place common ones (e.g., disable/re-enable logging) in a common location (similar to #407). For example, this

@pytest.fixture()
def disable_logging(self, request):
  """Disable logging (for the duration of this test case)."""
  logging.disable(logging.CRITICAL)
  def finalize():
    """Re-enable logging (so other test cases are unaffected)."""
    logging.disable(logging.NOTSET)
  request.addfinalizer(finalize)

is defined like 5 times.

edit: It's probably worth making the 'scope' argument selectable since users may want to disable just for 1 function, class, module, or session: http://pytest.org/latest/builtin.html#fixtures-and-requests

edit2: for the particular example above, be careful w/scope. pytest doesn't guarantee that fixture-using tests are run like:

build fixture
  fixture using tests
destroy fixture
other tests

So if you set disable_logging to scope='class', it will disable logging for every test in the class, whether the fixture is explicitly used or not.

This would be true for any fixture that touches global state (yet another reason why that's typically bad).