Yelp / MOE

A global, black box optimization engine for real world metric optimization.
Other
1.31k stars 139 forks source link

Moved from testify to pytest #406

Closed sc932 closed 9 years ago

sc932 commented 9 years ago

***** PEOPLE ***** Primary reviewer: @suntzu86

Reviewers:

***** DESCRIPTION ** Branch Name: sclark_36_testify_to_pytest Ticket(s)/Issue(s): Closes #36

-transitioned from testify to pytest, nothing fancy yet. -single fcn vs class setup/teardown being handled properly -fixed some scoping issues with instance v class v static methods -applied hidetraceback where appropriate

-future cleanup/pytest improvements (it's a lot more powerful than testify!) to be tracked in #408

***** TESTING DONE ***** make test make style-test

suntzu86 commented 9 years ago

haven't gone through the code, but could we get something like: http://pytest.org/latest/example/simple.html#control-skipping-of-tests-according-to-command-line-option or http://pytest.org/latest/skipping.html for marking tests as 'exclude'? Before this was straightforward in testify but would not be now. Marking 'slow' is an interesting idea too.

Also we may want our test helper functions (like relative error checks) to do something more like: http://pytest.org/latest/example/simple.html#writing-well-integrated-assertion-helpers i.e., add the tracebackhide thing.

sc932 commented 9 years ago

Yes, we can do all sorts of cool things with this framework.

If you want anything in particular feel free to ticket it. This is just the 0 level switch over to the new system. From here we can optimize.

suntzu86 commented 9 years ago

Added a tracking tickets for more pytest improvements: https://github.com/Yelp/MOE/issues/408

couple of things:

  1. the tracebackhide is generally useful and a one-liner so I'm adding those
  2. pytest.fixture's default scope is "function". for things that were previously class_setup, they should be "class" (and in the future some of these should be module or even session)
  3. making a "fin" function doesn't cause cleanup to happen nor does the name matter. You have to register it in a request object:
@pytest.fixture(autouse=True)
def myfixture(request):
  ...stuff...
  def finalize():
    ...cleanup...
  request.addfinalizer(finalize)

I'm making these changes now and will commit a diff to this branch.

suntzu86 commented 9 years ago

cleaned some stuff up, shipit from me

sc932 commented 9 years ago

:sheep: it

Thanks @suntzu86!