Since 97569e1, setUpClass is called once per test

[smspillaz]

Since 97569e1 setUpClass, tearDownClass setUpModule and tearDownModule get called once per unit test. This is due to running each suite's component individually, as the subprocess mode does.

According to the unittest documentation the order and number of times setUpClass and tearDownClass are called isn't strictly-speaking defined. It just so happens that by convention on unittest's own single-threaded mode that they only get called once, since the tests run in order. No guarantee is made about what happens when tests are run in a randomised order or in parallel.

Personally, I use those methods to do "expensive" set up and tear down where I know I can maintain the invariant between tests in setUp and tearDown. As such, the current behaviour makes my tests run a lot slower.

I would submit that the correct behaviour is to detect where tests are setting those methods and only divide up suites to the level where any of those methods are defined. So, for instance, if setUpClass was defined on a TestCase, then the entire TestCase should run in serial, but could be parallelized amongst other tests.

Here's some output to show the problem and a testcase:

from testtools import TestCase

import sys

class TestSetUpClass(TestCase):

    """Test set up class."""

    @classmethod
    def setUpClass(cls):
        sys.stderr.write("setUpClass\n")

    def test_one(self):
        """One"""
        sys.stderr.write("method one on two called\n")
        pass

    def test_two(self):
        """Two"""
        sys.stderr.write("method two on two called\n")
        pass

    def test_three(self):
        """Three"""
        sys.stderr.write("method three on two called\n")
        pass

Green 1.11.0, Coverage 3.7.1, Python 2.7.9

test.test_two.test_set_up_class
  TestSetUpClass
    OnesetUpClass
method one on two called
setUpClass
method three on two called
.   One
.   Three
    TwosetUpClass
method two on two called
.   Two

Ran 3 tests in 0.108s

OK (passes=3)

[CleanCut]

(See comment on #69)

[CleanCut]

Fixed, thanks to you! I really appreciate all the time you put into your pull requests. We wouldn't be where we are without your effort. :-) 2.0.0 should be out soon (tomorrow...Monday...maybe?) when I can finish up all the busywork that goes along with a release (changelog, cleanup, etc.)

[CleanCut]

2.0.0 has been released!!!